Computer Science and Convergence: CSA 2011 & WCC 2011 Proceedings

Lecture Notes in Electrical Engineering Volume 114

For further volumes: http://www.springer.com/series/7818

James J. (Jong Hyuk) Park
Han-Chieh Chao Mohammad S. Obaidat Jongsung Kim •

Editors

Computer Science and Convergence CSA 2011 & WCC 2011 Proceedings

123

Prof. James J. (Jong Hyuk) Park Department of Computer Science and Engineering Seoul National University of Science & Technology Gongreung 2-dong 172 Seoul 139-742, Republic of South Korea e-mail: [email protected] Prof. Han-Chieh Chao Institute of Computer Science & Information Engineering and Department of Electronic Engineering National Ilan University 1 Sec. 1, Shen-Lung Rd. 260, I-Lan, Taiwan e-mail: [email protected]

ISSN 1876-1100 ISBN 978-94-007-2791-5 DOI 10.1007/978-94-007-2792-2

Prof. Mohammad S. Obaidat Department of Computer Science & Software Engineering Monmouth University W. Long Branch NJ USA e-mail: [email protected] Prof. Jongsung Kim Department of e-Business Kyungnam University Masan Changwon Gyeongnam Republic of South Korea e-mail: [email protected]

e-ISSN 1876-1119 e-ISBN 978-94-007-2792-2

Springer Dordrecht Heidelberg London New York Library of Congress Control Number: 2011943213  Springer Science+Business Media B.V. 2012 No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Message from the CSA 2011 General Chairs

Welcome to the 3rd International Conference on Computer Science and its Applications (CSA 2011), held in the Jeju Island, Korea. CSA 2011 is the next event in a series of highly successful International Conference on Computer Science and its Applications, previously held as CSA 2008 (Australia, October, 2008) and CSA 2009 (Jeju, December, 2009). The CSA 2011 will be the most comprehensive conference focused on the various aspects of advances in computer science and its applications. The CSA 2011 will provide an opportunity for academic and industry professionals to discuss the latest issues and progress in the area of CSA. We would like to sincerely thank the following speakers who kindly accepted our invitations, and, in this way, helped to meet the objectives of the conference: • • • •

Mohammad S. Obaidat, Monmouth University, USA Habib F. Rashvand, University of Warwick, UK Han-Chieh Chao, National Ilan University, Taiwan Hai Jin, Huazhong University of Science and Technology (HUST), China

We sincerely thank the CSA 2011 honorary chiar, Makoto Takizawa and also thank the CSA 2011 steering chairs, James J. (Jong Hyuk) Park, Han-Chieh Chao and Mohammad S. Obaidat. It would have been impossible to organize our program without the hard work of all our chairs and committees. Finally, we would like to thank the DCIT 2011, HNCA 2011, MIST 2011, CC 2011 and MCM 2011 workshop chairs for contributions that have added variety to the topics represented. Sang-Soo Yeo and Victor Leung CSA 2011 General Chairs

v

Message from the CSA 2011 Program Chairs

On behalf of the CSA 2011 organizing committee, it is our pleasure to welcome you to the 3rd International Conference on Computer Science and its Applications (CSA 2011). The success of a conference is mainly determined by the quality of its technical program. This year’s program will live up to high expectations due to the careful selection by the Program Committee. They have spent long hours in putting together an excellent program and deserve a big applaud. The conference received 133 submissions, and all were reviewed by the Program Committee. In the review process, we assigned at least three Program Committee members to each paper. After careful deliberation and peer-reviews, we selected 42 papers for presentation and inclusion of the conference proceedings, whose acceptance rate is around 31.5%. There are many people who contributed to the success of CSA 2011. We would like to thank the many authors from around the world for submitting their papers. We are deeply grateful to the Program Committee for their hard work and enthusiasm that each paper received a thorough and fair review. Finally, we would like to thank all the participants for their contribution to the conference. Jongsung Kim, Isaac Woungang, Ray (Yueh-Min) Huang, Irfan Awan, Vincenzo Loia CSA 2011 Program Chairs

vii

CSA 2011 Organization

Honorary Chair Makoto Takizawa, Seikei University, Japan Steering Chairs James J. (Jong Hyuk) Park, SeoulTech, Korea Han-Chieh Chao, National Ilan University, Taiwan Mohammad S. Obaidat, Monmouth University, USA General Chairs Sang-Soo Yeo, Mokwon University, Korea Victor Leung, University of British Columbia, Canada Program Chairs Jongsung Kim, Kyungnam University, Korea Isaac Woungang, Ryerson University, Canada Ray (Yueh-Min) Huang, National Cheng-Kung University, Taiwan Irfan Awan, University of Bradford, UK Vincenzo Loia, University of Salerno, Italy Workshop Chairs Young-Sik Jeong, Wonkwang University, Korea Damien Sauveron, University of Limoges, France Der-Jiunn Deng, National Changhua University of Education, Taiwan Jalal Al Muhtadi, King Saud University, Saudi Arabia International Advisory Board Hamid R. Arabnia, The University of Georgia, USA Doo-soon Park, SoonChunHyang University, Korea Hsiao-Hwa Chen, Sun Yat-Sen University, Taiwan Philip S. Yu, University of Illinois at Chicago, USA Yi Pan, Georgia State University, USA ix

x

CSA 2011 Organization

Salim Hariri, University of Arizona, USA Leonard Barolli, Fukuoka Institute of Technology, Japan Jiankun Hu, RMIT University, Australia Shu-Ching Chen, Florida International University, USA Lei Li, Hose University, Japan Kevin (Jar-Ferr) Yang, National Cheng Kung University, Taiwan Local Arrangement Chairs Sang Yep Nam, Kookje College, Korea Deok Gyu Lee, ETRI, Korea Taeshik Shon, Samsung Electronics, Korea Publicity Chairs David Taniar, Monash University, Australia Joel Rodrigues, University of Beira Interior, Portugal Markus Aleksy, ABB Corporate Research Center, Germany Chengcui Zhang, The University of Alabama at Birmingham, USA Senol Z. Erdogan, Maltepe University, Turkey Jose A. Onieva, University of Malaga, Spain Seong O. Hwang, Hongik University, Korea Sang Oh ParK, Chungang University, Korea Finance & Registration Chair Changhoon Lee, Hanshin University, Korea Registration and Finance Chair Changhoon Lee, Hanshin University, Korea Web and system management chair Yang Sun Lee, Chosun University, Korea Jaeik Cho, Korea University, Korea

Track Chairs Track 1. Mobile and ubiquitous computing Deok Gyu Lee, ETRI, Korea Sanjay Kumar Dhurandher, University of Delhi, India Hoon Choi, Chungnam National University, Korea Track 2. Dependable, reliable and autonomic computing Kyusuk Han, KAIST, Korea Deqing Zou, Huazhong University of Science & Technology, China Jiman Hong, Soongsil University, Korea

CSA 2011 Organization

xi

Track 3. Security and trust management Namsu Chang, Sejong Cyber University, Korea Ahmad AlMulhem, King Fahd University of Petroleum & Minerals, Saudi Arabia Isaac Woungang, Ryerson University, Toronto, Canada Track 4. Graphics, multimedia systems and visual computing Chih-Lin Hu, National Central University, Taiwan Liang Zhou, Institute of Media Technology, Germany Lei Ye, University of Wollongong, Australia Track 5. Networking and communications Naveen Chilamkurti, La Trobe University, Australia Yuh-Shyan Chen, National Taipei University, Taiwan Yang Sun Lee, Chosun University, Korea Track 6. Database and data mining Takahiro Hara, Osaka University, Japan Richi Nayak, Queensland University of Technology, Australia Taeshik Shon, Ajou University, Korea Track 7. Game and software engineering Shu-Ching Chen, Florida International University, USA Ashkan Sami, Shiraz University, Iran Roman Neruda, Academy of Sciences of the Czech Republic, Czech Republic Track 8. Grid, cloud and scalable computing Christophe Cerin, Universite de Paris Nord, Frence Jen-Wen Ding, National Kaohsiung University of Applied Sciences, Taiwan Yeong-Deok Kim, Woosong University, Korea Track 9. Embedded system and software Chin-Fu Kuo, the National Kaohsiung University, Taiwan Jian-Jia Chen, Swiss Federal Institute of Technology (ETH) Zurich, Switzerland Qing Tan, Athabasca University, Canada Chin Feng Lai, National Ilan University, Taiwan Track 10. Artificial intelligence Vincenzo Loia, University of Salerno, Italy Yo-Ping Huang, National Taipei University of Technology, Taiwan Simone Ludwig, North Dakota State University, USA Alex Kuo, School of Health Information Science, University of Victoria, Canada Ruey-Maw Chen, National Chin-Yi University of Technology, Taiwan

xii

CSA 2011 Organization

Track 11. Distributed and parallel algorithms Ching-Hsien Hsu, Chung Hua University, Taiwan Jemal H. Abbawajy, Deakin University, Austrailia Constantinos Mourlas, University of Athens, Greece Track 12. Web and internet computing Ruay-Shiung Chang, National Dong Hwa University, Taiwan SangKeun Lee, Korea University, Korea Guandong Xu, Victoria University, Australia Track 13. IT policy and business management Hangbae Chang, Daejin University, Korea Andrew Kusiak, The University of Iowa, USA Ying Jiang, School of Information Management, Wuhan University, China Program Committee Track 1. Mobile and ubiquitous computing Andry Rakotonirainy, Queensland University of Technology, Australia David Taniar, Monash University, Australia Evi Syukur, University of New South Wales, Australia Jan Newmarch, BoxHill Institute, Australia Sudip Misra,Indian Institute of Technology, Kharagpur, India G. V. Singh,Jawaharlal Nehru University, New Delhi, India R. K. Pateriya, Maulana Azad National Institute of Technology, Bhopal, India Vijeta Mehra, University of Delhi, India M. Dominguez-Morales, University of Seville, SPAIN Taek Jin Kwon, Telchordia Technologies, USA Kae-Won Choi, SeoulTech, Korea Min Choi, Department of Computer Engineering, Wonkwang University, Korea Seung-Ho Lim, Hankuk University of Foreign Studies, Korea Nak-Seon Seong, Electronics and Telecommunications Research Institute, Korea Kilhung Lee, Seoul National University of Science and technology, Korea Hyo Hyun Choi, Department of Computer Science, Inha Technical College, Korea Track 2. Dependable, reliable and autonomic computing Masakazu Soshi, Hiroshima City University, Japan Gregorio Martinez, University of Murcia, Spain Jianxun Liu, Hunan University of Science and Technology, China Wei Chen, Nanjing University of Posts and Telecommunication, China Dennis Pfisterer, University of Luebeck, Germany Bo Zhao, Wuhan University, China Fei Yan, Wuhan University, China

CSA 2011 Organization

xiii

Weizhong Qiang, Huazhong University of Science and Technology, China Lansheng Han, Huazhong University of Science and Technology, China Neal N. Xiong, Georgia State University, USA Yuan Luo, Shanghai Jiaotong University, China Chuliang Weng, Shanghai Jiaotong University, China Konidahla M. Divyan, Singapore Chan Yeob Yeun, KUSTAR, UAE Dang Nguyen Duc, Samsung Electronics, Korea Hyeran Mun, ETRI, Korea Track 3. Security and trust management Dieter Gollmann, TUHH, Germany Danfeng Yao, Rutgers University, USA Claudio Ardagna, University of Milan, Italy Jordi Forne, Technical University of Catalonia, Spain Jose Onieva, University of Malaga, Spain Willy Susilo, University of Wollongong, Australia Aljosa Pasic, Atos Origin, Spain Stefanos Gritzalis, University of the Aegean, Greece Jan de Meer, Brandenburg Technical University, Germany Seog Chung Seo, Korea University, Korea Young In Cho, Korea University, Korea HeeSeok Kim, Korea University, Korea Tae Hyun Kim, Korea University, Korea Taek-Young Youn, ETRI, Korea Chang Han Kim, Semyung University, Korea Young-Ho Park, Sejong Cyber University, Korea Zubair Baig, King Fahd University of Petroleum and Minerals, Saudi Arabia El-Sayed El-Alfy, King Fahd University of Petroleum and Minerals, Saudi Arabia Mohammed Houssaini Sqalli, King Fahd University of Petroleum and Minerals, Saudi Arabia Sami Zhioua, King Fahd University of Petroleum and Minerals, Saudi Arabia Talal Mousa Alkharobi, King Fahd University of Petroleum and Minerals, Saudi Arabia Min Zhang, University of Nottingham, UK Soltan Alharbi, University of Victoria, Canada Peter Herrmann, NTNU, Norway Sjouke Mauw, University of Luxemburg, LU Stig F. Mjoelsnes, NTNU, Norway Stefanos Gritzalis, University of the Aegean, Greece Ketil Stoelen, SINTEF, Norway Christian Damsgaard Jensen, DTU, Denmark Jean-Marc Seigneur, University of Geneva, Switzerland

xiv

CSA 2011 Organization

Track 4. Graphics, multimedia systems and visual computing Atul Sajjanhar, Deakin University, Australia Lin Liu, University of South Australia, Australia Tanveer Zia, Charles Sturt University, Australia Xiaofei Liao, Huazhong University of Science and Technology, China Wei-Shinn Ku, Auburn University, USA Chuan-Ming Liu, National Taipei University of Technology, Taiwan Yeong-Sheng Chen, National Taipei University of Education, Taiwan Yao-Nan Lien, National Chengchi University, Taiwan Chih-Herng Ke, National Quemoy University, Taiwan Jiun-Long Huang, National Chiao Tung University, Taiwan Wei-Guang Teng, National Cheng Kung University, Taiwan Honggang Wang, University of Massachusetts, USA Rongbo Zhu, South-Central University for Nationalities, China Chin-Feng Lai, National Ilan University, Taiwan Kai Lin, Dalian University of Technology, China Sherali Zeadally, University of the District of Columbia, USA Dalei Wu, University of Nebraska-Lincoln, USA Xingang Liu, Yonsei University, Korea Track 5. Networking and communications Lain-Chyr Hwang, I-Shou University, Taiwan Ilsun You, Korean Bible University, Korea Rongxing Lu, University of Waterloo, Canada Chang Wu Yu, Chung Hua University, Taiwan ROC Tzung-Shi Chen, National University of Tainan, Taiwan ROC Chih-Yung Chang, Tamkang University, Taiwan ROC Chih-Shun Hsu, Shih Hsin University, Taiwan ROC Ling-Jyh Chen, Academia Sinica, Taiwan ROC Zeadally Sherali, University of the District of Columbia, USA Wang Yu, University of North Carolina at Charlotte, USA Cliff Zou, University of Central Florida, USA Hongyi Wu, University of Louisiana at Lafayette, USA Damla Turgut, University of Central Florida, USA Track 6. Database and data mining Shichao Zhang, University of Technology, Australia Chunping Li, Tsinghua University, China Qingyuan Bai, Fuzhou University, China Yasuhiko Morimoto, Hiroshima University, Japan Young-Ho Park, Sookmyung Women’s University, Korea Sang-Won Lee, Sungkyunkwan University, Korea Masashi Toyoda, University of Tokyo, Japan Jun Miyazaki, NAIST, Japan Yasuhiko Morimoto, Hiroshima University, Japan

CSA 2011 Organization

xv

Toshiyuki Amagasa, Tsukuba University, Japan Sanjay Madria, Missouri University of Science and Technology, USA Anirban Mondal, Indraprastha Institute of Information Technology, Delhi, India Sourav Bhowmick, Nanyang Technological University, Singapore Ramamohanarao Kotagiri, The University of Melbourne, Australia Hua Wang, University of Southern Queensland, Australia Raymond Lau, City University of Hong Kong, Hong Kong Sharma Chakravarthy, The University of Texas at Arlington, USA Simeon Simoff, University of Western Sydney, Australia Debajyoti Mukhopadhyay, Balaji Institute of Telecom & Management Pune, India Navneet Goyal, Department of Computer Science & Information Systems BITS Pilani Rajasthan, India Shlomo Geva, Queensland University of Technology, Australia Longbing Cao, University of Technology Sydney, Australia Track 7. Game and software engineering Son Bao Pham, Vietnam National University Hanoi, Vietnam Mei-Ling Shyu, University of Miami, USA Min Chen, University of Montana, USA Lin Lin, American National Standards Institute, USA Kasturi Chatterjee, hi5 Networks, Inc., USA Choochart Haruechaiyasak, NECTEC, Thailand Hongli Luo, Indiana University - Purdue University Fort Wayne, USA Jianhua Yan, Mintec Inc., USA Zhenyu Yang, Florida International University, USA Chengcui Zhang, University of Alabama at Birmingham, USA David LO, Singapore Management University, Singapore Masao Ohira, Nara Institute of Science and Technology, Japan Farhad Daneshgar, The University of New South Wales, Australia Tomoharu NAKASHIMA, Osaka Prefecture University, Japan Apostolos Ampatzoglou, Aristotle University of Thessaloniki, Greece Mazeiar Salehie, Lero—The Irish Software Engineering Center, Ireland Track 8. Grid, cloud and scalable computing Chao-Tung Yang, Tunghai University, Taiwan Keqiu Li, Dalian University of Technology, China Ramin Yahyapour, University of Dortmund, Germany Yangwoo Kim, Dongguk University, Korea Jong-Hyouk Lee, IMARA Team, INRIA, France Rajkumar Buyya, University of Melbourne, Australia Andrzej M. Goscinski, Deakin University, Geelong, Australia Uwe Schwiegelshohn, TU Dortmund University, Germany Jehn-Ruey Jiang, National Central University, Taiwan Dimitrios Lekkas, University of the Aegean, Greece Sebastien Goasguen, Clemson University, USA

xvi

CSA 2011 Organization

Namsoo Chang, Sejong Cyber University, Korea SeungTaek Ryoo, Hanshin University, Korea Hae Young Lee, Electronics and Telecommunications Research Institute, Korea Dong Kyoo Kim, Convergence Research Department, Electronics and Telecommunications Research Institute, Korea Dongho Kim, Halla University, Korea Yong-hee Lee, Halla University, Korea Hyunsung Kim, Kyungil University, Korea Namje Park, Jeju National University, Korea Track 9 . Embedded system and software Li-Pin Chang, National Chiao-Tung University, Taiwan Hiroyuki Tomiyama, Nagoya University, Japan Jun Wu, National Pingtung Institute of Commerce, Taiwan Shengquan Wang, University of Michigan Dearborn, USA Jian Li, Shanghai Jiaotong University, China Zili Shao, The Hong Kong Polytechnic University, Hong Kong Liang-Teh Lee, Tatung University, Taiwan Jen-Wei Hsieh, National Taiwan University of Science and Technology, Taiwan Yung-Feng Lu, National Tai-Chung, Institute of Technology, Taiwan Yuan-Tse Yu, National Kaohsiung Normal University, Taiwan Yashu Chen, National Taiwan University of Science and Technology, Taiwan Yuan-Hao Chang, National Taipei University of Technology, Taiwan Track 10. Artificial intelligence Babak Esfandiari, Carleton University, Canada Shu Chen Cheng, Southern Taiwan University, Taiwan Frode Eika Sandnes, Oslo University College, Norway Chuan-Kang Ting, National Chung Cheng University, Taiwan Antonina Dattolo, University of Udine, Italy Hakan Duman, British Telecom, UK Hani Hagras, University of Essex, UK Tzung-Pei Hong, National University of Kaohsiung, Taiwan Chang-Shing Lee, National University of Tainan, Taiwan Ahmad Lotfi, Nottingham Trent University, UK Marek Reformat, University of Alberta, Canada Thanos Vasilakos, University of Western acedonia, Greece Mario Vento, University of Salerno, Italy Yue-Shan Chang, National Taipei University, Taiwan Yen-Lin Chen, National Taipei University of Technology, Taiwan Hung-Chi Chu, Chaoyang University of Technology, Taiwan Shang-Lin Hsieh, Tatung University, Taiwan Li-Jen Kao, Hwa Hsia Institute of Technology, Taiwan Tsang-Long Pao, Tatung University, Taiwan Chwan-Lu Tseng, National Taipei University of Technology, Taiwan

CSA 2011 Organization

Shun-Yuan Wang, National Taipei University of Technology, Taiwan Don-Lin Yang, Feng Chia University, Taiwan Morris Chang, Iowa State University, USA Frode Eika Sandnes, Oslo University College, Norway Chang-Tien Lu, Virginia Tech, USA Jinglong Wu, Okayama University, Japan David Bell, Brunel University, UK Babak Esfandiari, Carleton University, Canada Kevin Grant, University of Lethbridge, Canada Wei Jin, North Dakota State University, USA Brigitte Krenn, Austrian Research Institute for Artificial Intelligence, Austria Jen Li, North Dakota State University, USA Ken Nygard, North Dakota State University, USA Julian Padget, University of Bath, UK Omer Rana, Cardiff University, UK Alireza Sadeghian, Ryerson University, Canada Track 11. Distributed and parallel algorithms Paolo Trunfio, University of Calabria, Italy Farhad Arbab, CWI, The Netherlands Kuan-Ching Li, Providence University, Taiwan Qishi Wu, University of Memphis, USA Jen-Wei Hsieh, National Taiwan University of Techonolgy, Taiwan Huaqun Guo, Institute for Infocomm Research, Singapore Massimo Cafaro, University of Salento, Italy Stefan Fischer, University of Luebeck, Germany Ragib Hasan, Johns Hopkins University, USA Mukaddim Pathan, CSIRO, AUSTRALIA Bahman Javadi, Melbourne University, AUSTRALIA Abdelkarim Erradi, Qatar University, Qatar Ekow Otoo, University of Witwatersrand, South Africa Sameer Bataineh, United Arab Emirates University, UAE Mustafa Mat Deris, UTHM, Malaysia Lionel Eyraud-Dubois, INRIA, France Chuan Wu, University of Hong Kong, China Sun-Yuan Hsieh, National Cheng Kung University, Taiwan Daniel Grosu, Wayne State University, USA Hae-Duck Joshua Jeong, Korean Bible University, Korea Xin Yuan, Florida State University, USA Cho-Chin Lin, National Yilan University, Taiwan Raj Kettimuthu, Argonne National Laboratory, USA Victor Malyshkin, ICMMG, Russia

xvii

xviii

CSA 2011 Organization

Track 12. Web and internet computing Chia-Hung Yeh , National Sun Yat-sen University, Taiwan Yuan-Cheng Lai, National Taiwan University of Science and Technology, Taiwan Guan-Ling Lee, National Dong Hwa University, Taiwan Yao-Chung Chang, National Taitung University, Taiwan Kuei-Ping Shih, Tamkang University, Taiwan Hazalina Hashim, Open University Malaysia, Malaysia Guanling Lee, National Dong Hwa University, Taiwan Chia-Min Wu, National Dong Hwa University, Taiwan Jui-Feng Yeh, National Chia-Yi University, Taiwan Hiroshi Esaki, University of Tokyo, Japan Daisuke Umehara, Kyoto University, Japan Huiping Guo, Cal State U, Los Angeles, USA Ilmi Yoon, Cal State U, San Francisco, USA Kyong-Ho Lee, Yonsei University, Korea Jae-Ho Choi, Yonsei University, Korea SungJin Choi, KT, Korea SungSuk Kim, Seokyeong University, Korea Hoh Peter In, Korea University, Korea Jaewoo Kang, Korea University, Korea Taek-Hun Kim, Korea University, Korea Juejie Yao, Peking University, China Toshiyuki AMAGASA, Tsukuba University, Japan Lin Li, Wuhan University of Technology, China Irene Garrigos, Universidad de Alicante, Spain Xiaohui Tao, University of Southern Queensland, Australia Zhenglu Yang, University of Tokyo, Japan JIanwei Zhang, Kyoto Sangyo University, Japan Yu Zong, University of Science and Technology of China, China Track 13. IT policy and business management Yang Shen, Wuhan University, P.R. China Han Xu, InfoQ, P.R. China Alton Chua Yeow Kuan, Nanyang Technological University, Singapore Margaret Tan, Nanyang Technological University, Singapore Theng Yin Leng, Nanyang Technological University, Singapore Qing Tan, Athabasca University, Canada Chin-Chen Chang, Computer Science Feng Chia University, Taiwan Johnnes Arreymbi, University of East London, UK Chia-Chen Lin, Providence University, Taiwan Neeli Prasad, Aalborg University, Denmark Jiqiang Lu, University of London, UK Theng Yin Leng, Nanyang Technological University, Singapore Benting Wan, Jiangxi University of Finance and. Economics, China Chu-Hsing Lin, Tunghai University, Taiwan

CSA 2011 Organization

xix

Margaret Tan, Nanyang Technological University, Singapore Alton Chua Yeow Kuan, Nanyang Technological University, Singapore Ernesto Cuadros-Vargas, San Pablo Catholic University, Peru Dong Seong Kim, Duke University, USA Jeng-Shyang Pan, National Kaohsiung University of Applied Sciences, Taiwan Seong-Moo Yoo, The University of Alabama in Huntsville, USA Chien-Liang(Jonathan) Liu, University of Florida, USA Stefanos Gritzalis, University of the Aegean, Greece

Message from the WCC 2011 General Chairs

As the General Chairs of the 2011 FTRA World Convergence Conference (FTRA WCC 2011), we have the pleasure of welcoming you to this conference, which will be held at the beautiful Jeju Island, Korea. FTRA WCC 2011 is the next event in a series of highly successful World Convergence Conference, previously held as FTRA WCC 2010 (Gwangju, December 2010). The conference is aimed at addressing key themes on ‘‘Technologies and Services for Converged environments’’. We would like to thank the founding steering chair, James J. (Jong Hyuk) Park. Finally, we sincerely thank all our chairs and committees. Without their hard work, the success of WCC 2011 would not have been possible. Mohammad S. Obaidat, Jin Kwak, Laurence T. Yang, Changhoon Lee, Rajkumar Buyya, Han-Chieh Chao, Wanlei Zhou, Sangjin Lee, Leonard Barolli, Young-Sik Jeong, Sang-Soo Yeo, Jianhua Ma, Hangbae Chang, Dong Yong Kwak, Hai Jin, Jungduk Kim WCC 2011 General Chairs

xxi

Message from the WCC 2011 Program Chairs

It is with great pleasure that we welcome you to the 2011 FTRA World Convergence Conference (FTRA WCC 2011), held on December 12–15, 2011 at Jeju, Korea. In WCC 2011, we received lots of paper submissions which have undergone blind review by at least three reviewers of technical program committee that consists of leading researchers around the globe. Without their hard work, achieving such a high quality of proceedings would have been impossible. We take this opportunity to thank them for their great support and cooperation. Finally, we would like to thank all the authors and participants for their contributions to the conference. Jongsung Kim, Hamid R. Arabnia, Naveen Chilamkurti, Sajid Hussain, Taeshik Shon, Dong-Seong Kim, Ching-Hsien Hsu, Nazim AGOULMINE, Ketan Mane, Yang Sun Lee, Dong Kyoo Kim, Seungmin Rho, Hangbae Chang WCC 2011 Program Chairs

xxiii

WCC 2011 Organization

Steering Chair James J. (Jong Hyuk) Park, Seoul National University of Science and Technology, Korea General / General-Vice Chairs Mohammad S. Obaidat, Monmouth University, USA Jin Kwak, Soonchunhyang University, Korea Laurence T. Yang, St Francis Xavier University, Canada Changhoon Lee, Hanshin University, Korea Rajkumar Buyya, University of Melbourne, Australia Han-Chieh Chao, National Ilan University, Taiwan Wanlei Zhou, Deakin University, Australia Sangjin Lee, Korea University, Korea Leonard Barolli , Fukuoka Institute of Technology, Japan Young-Sik Jeong , Wonkwang University, Korea Sang-Soo Yeo, Mokwon University, Korea Jianhua Ma, Hosei University, Japan Hangbae Chang, Daejin University, Korea Dong Yong Kwak, ETRI, Korea Hai Jin, Huazhong University of Science and Technology, China Jungduk Kim, Chung-Ang University, Korea Program Chairs Jongsung Kim, Kyungnam University, Korea Hamid R. Arabnia, University of Georgia, USA Naveen Chilamkurti, La Trobe University, Australia Sajid Hussain, Fisk University, USA Taeshik Shon, Ajou University, Korea Dong-Seong Kim, Duke University, USA Ching-Hsien Hsu, Chung Hua University, Taiwan Nazim AGOULMINE, University of Evry Val d’Essonne, France xxv

xxvi

Ketan Mane, University of North Carolina, USA Yang Sun Lee, Chosun University, Korea Dong Kyoo Kim, ETRI, Korea Seungmin Rho, Korea University, Korea Hangbae Chang, Daejin University, Korea Publicity Chairs Jin Kwak, Soonchunhyang University, Korea Sang Oh Park, Chungang University, Korea

WCC 2011 Organization

Message from the DCIT 2011 Chairs (CSA 2011 Workshop)

It is our great pleasure to welcome you to the 1st International Workshop on Digital Convergence, Information and Technology (DCIT-2011), which will be held in conjunction with the 3-rd International Conference on Computer Science and its Applications (CSA-2011) in Jeju, Korea, from December 12 to December 15, 2011. Digital Convergence and Information Technology of today are going through a rapid evolution. Different kinds of Technology and Learning, Science with different characteristics are emerging and they are integrating in Different Content and technology. Also known as digital convergence, the combining of several industries computers, communications, consumer electronics, entertainment, and mass media through various devices exchange data in digital form. Why it is important: Convergence has led to electronic products that perform multiple functions, such as TVs with internet access, cell phones that are also digital cameras, and a refrigerator that allows you to send email. The progress of electronic integration and wireless communications is going to pave the way to offer people the access to the wireless networks on the fly, based on which all electronic devices will be able to exchange the information with each other whenever necessary. Also, ubiquitous computing is an emerging field of research for computing paradigms. This emergence is the natural result of research and technological advances mainly in Digital Convergence and Information Technology, mobile computing, sensor networks, ad-hoc networks, IT service and application, Digital Convergence Policy , Telemedicine, Cloud Computing, Smart learning and application, Robotics, 3D animation and Multimedia application, Biometrics and System, GPS and GIS, Bioinformatics and agent technologies. The aim of this international workshop is to present the innovative research and technologies as well as developments related to Digital Convergence and Information Technology, mobile computing, Cloud Computing, Smart learning and

xxvii

xxviii

Message from the DCIT 2011 Chairs

application, Image Processing, GPS and GIS and their applications. The organizing committee of DCIT-2011 selected 10 papers for presentation in the workshop. The organization of an International Workshop requires the support and help of many people. First, we would like to thank all the authors for submitting their papers, the Program Committee Members, and the reviewers. We would like to give special thanks to Prof. Young-Sik Jeong, Wonkwang University, Korea as leading Workshops General and Program Chair of DCIT2011 for their encouragement and support to hold our workshop in conjunction with DCIT-2011. Finally, we would like to thank the Local Arrangement Chairs for the local arrangement of the workshop. We hope you will enjoy the workshop and have a great time in Jeju, Korea. Bonghwa Hong, Hwayoung Jeong, Seongsoo Cho DCIT 2011 Chairs

DCIT 2011 Organization

Workshop Organizer Bonghwa Hong, Kyunghee Cyber University, Seoul, Korea Program Chairs Hwa-Young Jeong, Kyunghee University, Seoul, Korea Seong-soo Cho, Kwangwoon University, Seoul, Korea Program Committee Members Won-bong Lee, Kyung Hee Cyber University, Korea Dong-Keon Kim, Kyung Hee University, Korea Un-ho Choi, Seoul National University, Korea Hye-jung Jung, Pyeongtaek University, Korea Byeong Kug Yoon, Kyunghee Cyber University, Korea Jin-Mook Kim, SunMoon University, Korea Juyeon Jo, University of Nevada, USA Jungsun (Sunny) Kim, Texas Tech University, USA Sun Dan Ning, Shandong Institute of Business and Technology, China Andrew Thompson, Kyung Hee University, Humanitas College, Korea Frelyn Pagente, Speechpower Inc. Philippines HaeJong Joo, Dongguk University, Korea Heagil Choi, Kyunghee Cyber University, Korea JangMook Kang, Dongguk University, Korea

xxix

Message from the HNCA 2011 Chairs (CSA 2011 Workshop)

It is our great pleasure to welcome you to the 1st International Workshop on Heterogeneous Networks, Computing and Applications (HNCA-2011), which will be held in conjunction with the 3rd International Conference on Computer Science and its Applications (CSA-2011) in Jeju, Korea, from December 12 to December 15, 2011. Networks of today are going through a rapid evolution. Different kinds of networks with different characteristics are emerging and they are integrating in heterogeneous networks. For these reasons, there are many interconnection problems which may occur at different levels in the hardware and software design of communicating entities and communication networks. These kinds of networks need to manage an increasing usage demand, provide support for a significant number of services, guarantee their QoS, and optimize the utilization of network resources. The progress of electronic integration and wireless communications is going to pave the way to offer people the access to the wireless networks on the fly, based on which all electronic devices will be able to exchange the information with each other whenever necessary. Also, ubiquitous computing is an emerging field of research for computing paradigms. This emergence is the natural result of research and technological advances mainly in wireless communications, mobile computing, sensor networks, ad-hoc networks, P2P networks and agent technologies. The aim of this international workshop is to present the innovative research and technologies as well as developments related to heterogenous networking, wireless communications, mobile computing and their applications. The organizing committee of HNCA-2011 selected sixpapers for presentation in the workshop. The organization of an International Workshop requires the support and help of many people. First, we would like to thank all the authors for submitting their papers, the Program Committee Members, and the reviewers.

xxxi

xxxii

Message from the HNCA 2011 Chairs

We would like to gice special thanks to Prof. Young-Sik Jeong, Wonkwang University, Korea as leading Workshops Co-Chair of CSA-2011 for his encourangement and support to held our workshop in conjunction with CSA-2011. Finally, we would like to thank the Local Arrangement Chairs for the local arrangement of the workshop. We hope you will enjoy the workshop and have a great time in Jeju, Korea. Leonard Barolli, Makoto Takizawa, Makoto Ikeda, Tomoya Enokido HNCA 2011 Chairs

HNCA 2011 Organization

Workshop Organizers Leonard Barolli, Fukuoka Institute of Technology, Japan Makoto Takizawa, Seikei University, Japan Program Chairs Makoto Ikeda, Fukuoka Institute of Technology, Japan Tomoya Enokido, Rissho University, Japan Program Committee Members Jiro Iwashige, Fukuoka Institute of Technology, Japan Arjan Durresi, Indiana University Purdue University Indianapolis (IUPUI), USA Hiroshi Maeda, Fukuoka Institute of Technology, Japan Fatos Xhafa, Technical University of Catalonia, Spain Joan Arnedo Moreno, Open University of Catalonia, Spain Akio Koyama, Yamagata University, Japan Toshiaki Matsunaga, Fukuoka Institute of Technology, Japan Vladi Kolici, Tirana Polytechnic University, Albania Kazunori Uchida, Fukuoka Institute of Technology, Japan Valbona Barolli, Seikei University, Japan Toshihiko Wakahara, Fukuoka Institute of Technology, Japan Rozeta Miho, Tirana Polytechnic University, Albania Koki Watanabe, Fukuoka Institute of Technology, Japan Tao Yang, Fukuoka Institute of Technology, Japan

xxxiii

Message from the MIST 2011 Chair (CSA 2011 Workshop)

Welcome to the First International Workshop on Multimedia and Information Security Technology (MIST 2011). The MIST 2011 will be held in conjunction with the Third International Conference on Computer Science and its Applications in Jeju, Korea. The MIST 2011 is a comprehensive workshop focused on the various aspects of information security including multimedia. The MIST 2011 will provide an opportunity for academic and industry professionals to discuss the latest issues and progress in the areas of multimedia and information security as well as related issues in innovative multimedia technology and public policy. In this year, there were large number of good papers submitted to the MIST 2011. All the submitted papers had undergone rigorous review process by the program committee members and some external reviewers. Only four papers were accepted for presentation and publication in the Conference proceedings. For organizing an international workshop, there were the support and help of many people. First, we would like to thank all the authors for submitting their outstanding papers. Also, we thank the program committee members and reviewers who carefully evaluated the submitted papers. We hope that the MIST 2011 will provide you with opportunity to interact, share ideas with, and learn from the other researchers working in the fields of multimedia and information security. We also encourage you to continue to participate in future editions of MIST workshop in order to contribute to the growth of this community. Thank you. Hae-Yeoun Lee MIST 2011 Chair

xxxv

MIST 2011 Organization

Workshop Organizer Hae-Yeoun Lee, Kumoh National Institute of Technology, Korea Workshop Committee Heung-Kyu Lee, Korea Advanced Institute of Science and Technology (KAIST), Korea Yongjian Hu, South China University of Technology, China Kyung-Su Kim, KT Network R&D Laboratory, Korea Min-Jeong Lee, Korea Advanced Institute of Science and Technology (KAIST), Korea Program Committee Members Hae-Yeoun Lee, Kumoh National Institute of Technology, Korea Yongjian Hu, South China University of Technology, China Hyungshin Kim, Chungnam National University, Korea Kyung-Su Kim, KT Network R&D Laboratory, Korea Dong-Hyuck Im, KT Network R&D Laboratory, Korea

xxxvii

Message from the CC 2011 Chairs (CSA 2011 Workshop)

It is our pleasure to welcome to the 2011 International Workshop on Computing and Communications (CC-11). The MIST 2011 will be held in conjunction with the Third International Conference on Computer Science and its Applications in Jeju, Korea. Recently, advances in computing and communication technologies are creating exciting new paradigms and are emerging as disciplines to provide intelligent computing and communication services all the time, everywhere, such as ubiquitous/pervasive computing and communication, smart/Intelligent computing and communication, green computing and communication, and so on. This emergence is a natural outcome of research and technological advances in wireless networks, computer communication, mobile computing and distributed computing. And its explosive impact on our daily life can be compared to that of computer over the past century. So, we must realize computing and communications are an important base and trend for present and future information technologies. The primary goal of this workshop is to provide researchers and academic communities around the world with the highest quality articles while reporting the state-of-the-art research results and scientific findings. We would like to take this opportunity to thank all the authors and participants for your contributions to make CC 2011 a great success. Changhoon Lee, Ching-Hsien Hsu CC 2011 Chairs

xxxix

CC 2011 Organization

Workshop Organizers Changhoon Lee, Hanshin University, Korea Ching-Hsien Hsu, Chung Hua University, Taiwan Program Committee Members Jung-Sik Cho, Chung-Ang University, Korea Hoojin Lee, Hansung University, Korea Eun-Young Lee, Dongduk Women’s University, Seoul, Korea

xli

Message from the MCM 2011 Chairs (CSA 2011 Workshop)

The 2011 International Workshop on Mobile platform, Consumer electronics and Multimedia services (MCM-11) is a merged workshop of the following three workshops: • International Workshop on Mobile Platform, Computing and Applications (MPCA 2011) • The 2011 International Workshop on Consumer Electronics and Digital Convergence (CEDC-11) • The 2011 International Workshop on Web and Multimedia Services (WMS-11) The workshop MCM-11 will comprehensively provide an interdisciplinary opportunity for academic and industry professionals to discuss the latest issues, progress, projects, research cooperation, and new idea in the areas of Mobile platform, Consumer electronics and Multimedia services. On behalf of the CSA-11, we would like to welcome all the delegates and their guests to the workshop MCM-11. We hope that you will enjoy the workshop and find it fruitful. Sang-Soo Yeo, Wen-Chung Kao, Sang Oh Park, Won Woo Ro, James J. (Jong Hyuk) Park, Young-Sik Jeong, Seung-Ho Lim, Min Choi, Seungmin Rho, Sethuraman Panchanathan MCM 2011 Chairs

xliii

MCM 2011 Organization

Workshop Organizers Sang-Soo Yeo, Mokwon University, Korea (Chair) Wen-Chung Kao, NTNU, Taiwan (Co-Chair) Sang Oh Park, Chung-Ang University, Korea Won Woo Ro, Yonsei University, Korea James J. (Jong Hyuk) Park, Seoul National University of Science and Technology, Korea Young-Sik Jeong, Wonkwang University, Korea Seung-Ho Lim, Hankuk University of Foreign Studies, Korea Min Choi, Wonkwang University, Korea Seungmin Rho, Korea University, Korea Sethuraman Panchanathan, Arizona State University, USA Program Committee Members Davide Proserpio, Carlos III University, Spain Zhaocheng Wang, Tsinghua University, China, Yueh-Min Huang, National Cheng Kung University Jong-Ok Kim, Korea University, Korea Tonghong Li, University of Madrid, Spain Da-Wei Chang, National Cheng Kung University, Taiwan Yawgeng A.Chau, Yuan-Ze University, Taiwan Youngmo Han, Hanyang Cyber University, Korea Haidi Ibrahim, Universiti Sains Malaysia, Malaysia Yonghong Kuo, Xidian University, China Zi-Wei Zheng, Ningbo University, China Juan Carlos Lopez, University of Castilla-La Mancha, Spain Omer ileri, Technology and Innovation Funding Directorate, Turkey Thomas Wook Choi, Hankuk University of Foreign Studies, Korea Chan Yeun Yeob, Khalifa University of Science Technology and Research, UAE Hyohyun Choi, Inha Technical College, Korea

xlv

xlvi

MCM 2011 Organization

Rodrigo Roman Castro, University of Malaga, Spain Yong Lee, ChungJu University, Korea Ting-Wei Hou, National Cheng-Kung University, Taiwan Paul S. Fisher, Winston-Salem State University, USA Stefan Mozar, CCM Consulting, Germany Mohammad. A. Karim, Old Dominion University, USA Dong Keun Kim, Sangmyung University, Korea Raed Mesleh, Jacobs University Bremen, German G. Fernández-Escribano, University of Castilla-La Mancha, Spain Mohammed Golam Sarwer, University of Windsor, Canada Samuel H. Russ, University of South Alabama, USA Giancarlo Iannizzotto, University of Messina, Italy Liu Wei, Samsung Electronics Jiakun Liu, University of Pensilvenia Horry Hojung Nam, University of California, San Diego Youngjae Kim, Oak Ridge Research Center Namgi Kim, Kyonggi University Wonjae Lee, ETRI Young-Gab Kim, Korea University, Korea Marco Bertini, Universita di Firenze, Italy Stephan Kopf, University of Mannheim, Germany

Contents

Part I

Computer Science and its Applications

A Large-Scale Terrain Rendering Method With GPU-Based Geometry Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zihou Ge and Wenhui Li

3

Cots Sparse Matrix Utilization in Distribution Power Flow Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dino Ablakovic, Izudin Dzafic and Hans-Theo Neisius

11

A New Graph-Based Algorithm for Persian Text Summarization . . . . Hassan Shakeri, Saeedeh Gholamrezazadeh, Mohsen Amini Salehi and Fatemeh Ghadamyari The Factors Affect User Behaviour in a Picture-Based User Authentication System: PixelPin . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhang Min, Brendan Ryan and Sarah Atkinson Analysis of Smartphone-Based Location Information . . . . . . . . . . . . . Dohyun Kim, Jewan Bang and Sangjin Lee

21

31

43

Dynamic Visualization for Sensing Information Based on Mobile Sensor Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Su-Hyun Yang, Eun-Ha Song and Young-Sik Jeong

55

Model Driven Real-Time Power System Application Development Using Just-In-Time Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . Izudin Dzafic, Priyanka Mohapatra and Nenad Lecek

67

xlvii

xlviii

Contents

An Efficient Distribution Sensitive Privacy for Real-Time Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arijit Ukil, Jaydip Sen and Sumit Ghosh DeSPoT: A Method for the Development and Specification of Policies for Trust Negotiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tormod Håvaldsrud, Birger Møller-Pedersen, Bjørnar Solhaug and Ketil Stølen Extended NIST Prime Family for Efficient Modular Reduction. . . . . . Young In Cho, Nam Su Chang, Chang Han Kim and Seokhie Hong

81

93

105

High Attack Cost: Hash Based RFID Tag Mutual Authentication Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jung-Sik Cho and James J. (Jong Hyuk) Park

113

Energy-Efficient Multi-Granularity Resource Reservations for Multimedia Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Joohyung Sun and Hyeonjoong Cho

121

Single Camera Vehicle Detection Using Edges and Bag-of-Features. . . Syed Jahanzeb Hussain Pirzada, Ehsan Ul Haq and Hyunchul Shin

135

A New BiGaussian Edge Filter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ehsan Ul Haq, Syed Jahanzeb Hussain Pirzada and Hyunchu Shin

145

Self-Healing in Smart Distribution Networks Using Data Exchange Model and Reliable Multicast Protocol. . . . . . . . . . . . . . . . . . . . . . . . I. Dzafic, N. Lecek and T. Donlagic

155

Applying Event-Based Framework to Support Power System Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nenad Lecek and Izudin Dzafic

167

Utilizing Converged Networks for Green Network Operations . . . . . . Xiaodong Yang A Ring and Clustering Routing Scheme Based on Data Field in Sensor Network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wenwen Dai, Kaihua Xu, James J. (Jong Hyuk) Park, Naixue Xiong and Yuhua Liu

183

193

Contents

A Case Study on Cooperative Real-time Quality Monitoring Service Utilizing Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . Hangbae Chang, Jonggu Kang, Yanghoon Kim, Jaehwan Lim and Youngsub Na High Availability Archive for Time-Based Data . . . . . . . . . . . . . . . . . Tobias Haag, Thomas Meier and Izudin Dzafic A Multivariate Spatial Outlier Detection Method Based on Semantic Similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fatemeh Azam and Ahmad Baraani-Dastjerdi Cohesion Between Size of Commit and Type of Commit. . . . . . . . . . . Maryam Marzban, Zahra Khoshmanesh and Ashkan Sami

xlix

203

211

221

231

Software Defect Prediction Using Transitive Dependencies on Software Dependency Graph. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Javad Kamyabi, Farzaneh Maleki and Ashkan Sami

241

Test Case Generation for Classes in Objects-Oriented Programming Using Grammatical Evolution . . . . . . . . . . . . . . . . . . . Jirawat Chaiareerat, Peraphon Sophatsathit and Chidchanok Lursinsap

251

An Efficient Access Control Scheme for Online Gaming Server . . . . . Hye-Young Kim

259

Performance Evaluation of a New Scheduling Algorithm in Data Grid Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. Mansouri, Gh. Dastghaibyfard and A. Horri

269

A Novel Dynamic Replication Algorithm in Hierarchical Data Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. Mansouri and Gh. Dastghaibyfard

281

Intelligent Application Migration within a Self-Provisioned Hybrid Cloud Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei-Jen Wang, Yo-Ming Lo, Shao-Jui Chen and Yue-Shan Chang

295

Multi-Band OFDM and OFDM Simulation Software Using MATLABÒ Graphical User Interface . . . . . . . . . . . . . . . . . . . . Roslina Mohamad, Wan Mohd Hafeez, Wan Mohd Salleh and Nuzli Mohdmad Anas

305

l

Contents

A Comparison on PSO Variants in Calculating HEPWM Switching Angles for a Cascaded H-Bridge Multilevel Inverter. . . . . . . . . . . . . . Norkharziana Mohd Nayan, Naziha Ahmad Azli and Shahrin Md. Ayob

315

A Hybrid Artificial Potential Field: Genetic Algorithm Approach to Mobile Robot Path Planning in Dynamic Environments . . . . . . . . . Yanping Liu and Kamal K. Bharadwaj

325

Energy-Constrained Scheduling for Weakly-Hard Real-Time Tasks on Multiprocessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yeonhwa Kong and Hyeonjoong Cho

335

A New Software Process Model: The Insect Model . . . . . . . . . . . . . . . Mahmood Vahedi Moghaddam and Mehdi Zand Akbari

349

Improving Recommendations in Tag-Based Systems with Spectral Clustering of Tag Neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rong Pan, Guandong Xu and Peter Dolog

355

Event-Centric Test Case Scripting Method for SOA Execution Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Youngkon Lee

365

bQoS (Business QoS) Parameters for SOA Quality Rating . . . . . . . . . Youngkon Lee

375

Business-Centric Test Assertion Model for SOA . . . . . . . . . . . . . . . . . Youngkon Lee

385

A Classification of Model-Driven Approaches to Develop Semantic Web Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alaeddin Kalantari, Suhaimi Ibrahim and Hamed Taherdoost A Conceptual Model Explaining IT/IS Usage Behavior . . . . . . . . . . . . Sabine Hoesch and Siegfried Voessner Offering a Conceptual Model of Technology Acceptance in e-Banking Based on the Customers’ Satisfaction and Trust. . . . . . . Alireza Dehghan, Parham Arjomand, Amirahmed Nayyeri and Masoud Tabatabaey

395

405

415

Contents

Part II

li

Computer Convergence

Architecture for Secure SCADA System Based on IF-MAP Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chuan Deng, Lin Yang, Linru Ma and Yuan Cao

427

Research on the Model of the Survivable Storage System (SSS) and Its Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao-yuan Yang, Jing-jing Wang and Xiao-feng Chen

437

New Improved Impossible Differential Attack on Reduced-Round AES-128 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ya Liu, Dawu Gu, Zhiqiang Liu, Wei Li and Weihao Kong

453

Smart Space Model for Facilitating Dynamic Service Binding. . . . . . . Changgyu Bak, Euna Choi, Jung-Won Lee and Nammee Moon Functionality Evaluation of Ubiquitous Service Ontology for Dynamic Service Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . Meeyeon Lee, Jung-Won Lee, Nammee Moon and Seung Soo Park OO Divide and Conquer Pattern Suitable for Parallel, Grid and Cloud Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manouchehr ZadahmadJafarlou, Bahman Arasteh and Parisa YousefzadehFard A Dynamic and Reliable Failure Detection and Failure Recovery Services in the Grid Systems. . . . . . . . . . . . . . . . . . . . . . . . Bahman arasteh, Manouchehr ZadahmadJafarlou and Mohammad Javad Hosseini Vehicle Positioning Technology Using Infra-based Laser Scanner Sensors for Autonomous Driving Service . . . . . . . . . . . . . . . . . . . . . . Kyoungwook Min and Jeongdan Choi Security Manpower Scheduling for Smart Airports . . . . . . . . . . . . . . Sangwon Seo, Sanggyun Choi and Chulung Lee A Study on Static Analysis Model of Mobile Application for Privacy Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Seil Kim, Jae Ik Cho, Hee Won Myeong and Dong Hoon Lee

463

475

487

497

511

519

529

lii

Part III

Contents

Digital Convergence Information Technology

China’s Industrial Policy: Focusing on the . . . . . . . . . . . Won-bong Lee

543

A Dual CPU Based Fault Tolerance Technique for Manufacturing System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HwaYoung Jeong and BongHwa Hong

553

A Computational Clustering of Korean Classical Texts: Focusing on the Gogocheonbyeon Passage of Sugungga . . . . . . . . . . . . Woonho Choi and Dong Keon Kim

561

The Survey of Quality Model for Software and System . . . . . . . . . . . Hye-Jeong Jeong and Suck-Joo Hong

569

Improvement of Retinex Algorithm for Backlight Image Efficiency . . . Seongsoo Cho, Bhanu Shrestha, Hae-Jong Joo and Bonghwa Hong

579

A User Authentication Method for M2M Environments . . . . . . . . . . . Jin-Mook Kim, Hwa-Young Jeong and Bong-Hwa Hong

589

Test Driven Development of Model Transformation with Reusable Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jong-won Ko and Young-jae Song An Advanced E-learning System Using Web Services . . . . . . . . . . . . . Hae-Gill Choi, Jungsun Kim and JuYeon Jo A Study of Mobile Architecture Web Design for Virtual Ecology Learning System Using LED (Light-Emitting Diodes) Lights in Smart-Phone Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jang-Mook Kang, Bong-Hwa Hong, You-Jin Song and Jae-sang Cha A Study on the Tag Performance Test for International Standards Using RFID Emulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hae-Gill Choi, Sang-Soo Kim, Moon-Taek Cho, Hae-Jong Joo and Euy-Soo Lee

597

607

613

621

Contents

Part IV

liii

Heterogenous Networks, Computing and Applications

Simulation and Performance Analysis of Ad-hoc Networks Using NS-2 and NS-3 Simulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . Makoto Ikeda, Elis Kulla, Masahiro Hiyama and Leonard Barolli A Comparison Study for Different Settings of Crossover and Mutation Rates Using WMN-GA Simulation System . . . . . . . . . . Evjola Spaho, Tetsuya Oda, Admir Barolli, Fatos Xhafa, Leonard Barolli and Makoto Takizawa A Survey on MANET Testbeds and Mobility Models . . . . . . . . . . . . . Elis Kulla, Makoto Ikeda, Leonard Barolli, Fatos Xhafa and Jiro Iwashige Evaluation of Wireless Sensor Networks for Multi Mobile Events Using Different Topologies, Protocols and Radio Propagation Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tao Yang, Leonard Barolli, Jiro Iwashige, Arjan Durresi and Fatos Xhafa

635

643

651

659

Trustworthiness-Based Broadcast Protocols in Peer-to-Peer Overlay Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ailixier Aikebaier, Tomoya Enokido and Makoto Takizawa

669

Energy Efficient Distributed Systems for Storage-Based Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takuro Inoue, Ailixier Aikebaier, Tomoya Enokido and Makoto Takizawa

677

Part V

Multimedia and Information Security Technology

Forged Region Detection for Scanned Images. . . . . . . . . . . . . . . . . . . Chang-Hee Choi, Min-Jeong Lee, Dai-Kyung Hyun and Heung-Kyu Lee

687

Camcorder Identification for Heavily Compressed Low Resolution Videos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dai-Kyung Hyun, Chang-Hee Choi and Heung-Kyu Lee

695

Block-based Image Authentication Algorithm Using Reversible Watermarking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dong-Gyu Yeo and Hae-Yeoun Lee

703

liv

Contents

Digital Forgery Detection Scheme Incorporating Imaging Device Characteristics Using Wiener Filter . . . . . . . . . . . . . . . . . . . . Jae Youn Shim and Seong-Whan Kim

Part VI

713

Computing and Communications

Dynamic Multi-channel Multi-path Routing Protocol for Smart Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jih-Ching Chiu, Kai-Ming Yang, Yao-Chin Huang and Ming-Shiou Wu

725

Speed and Surge Control for an Unmanned Aircraft Vehicle with Turbojet Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Min-Seok Jie and Beyong-In Jung

735

Influence of G729 Speech Coding on Automatic Speaker Recognition in VoIP Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . Dalila Yessad, Abderrahmane Amrouche, Mohamed Debyeche and Nabil Ramou Scheduling DAGs with Parallel Tasks in Multi-Clusters Based on Parallel Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Silvio Luiz Stanzani and Líria Matsumoto Sato Tomographic Reconstruction of a Multi-Attenuation Phantom by Means of Ultrasonic Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mohd Hafiz Fazalul Rahiman, Ruzairi Abdul Rahim and Herlina Abdul Rahim A Healthcare Cloud Computing Strategic Planning Model . . . . . . . . . Kuo Mu-Hsing

Part VII

745

753

761

769

Mobile Platform, Consumer Electronics and Multimedia Services

Multimedia Data Placement and Retrieval for Mobile Platform . . . . . Seung-Ho Lim A Platform-Independent Smartphone Application Development Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Min Choi

779

787

Contents

lv

Energy-Efficient Scheduling for Mobile Sensor Networks with Renewable Energy Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kae Won Choi

795

Issues and Challenges in Determining Quality of Service Requirements in Wired, Wireless and Mobile Networks . . . . . . . . . . . P. Calduwel Newton and L. Arockiam

805

Implementation of the River Monitoring System Using Embedded RTU Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dae-Sik Ko

813

A Study on Message Batch Verification Scheme Using Bloom Filter in VANET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Su-Hyun Kim and Im-Yeong Lee

821

Computing Text-to-Text Semantic Relatedness Based on Building and Analyzing Enriched Concept Graph . . . . . . . . . . . . . Zoleikha Jahanbakhsh Nagadeh, Fariborz Mahmoudi and Amir Hossein Jadidinejad Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

831

841

Part I

Computer Science and its Applications

A Large-Scale Terrain Rendering Method With GPU-Based Geometry Compression Zihou Ge and Wenhui Li

Abstract We present a large-scale terrain rendering method, which is based on restricted quadtree meshes. The error metric method corresponds to parallel computing requires, and processes the error metric and triangulation by GPU, to improve the rending efficiency of image hardware, and save the running time of CPU. For large terrain data, terrain data are compressed and stored in special method, and dynamically transmitted to graphic hardware according to rendering demand, and decoded by GPU too. Our compression method significantly reduces bandwidth requirements that is the bottleneck of terrain rendering.



Large-scale terrain rendering Keywords GPU-based meshes Nested error metric Geometry compression







Restricted quadtree

1 Introduction Real-time terrain rendering is an important area of computer graphics, which is used for flight simulators, computer games, virtual reality and visualization. With the development of technology of remote sensing, the size and resolution of digital Z. Ge (&)  W. Li College of Computer Science and Technology, JiLin University, Changchun, Jilin, China e-mail: [email protected] W. Li e-mail: [email protected] Z. Ge Aeronautical Engineering department, Aeronautical University of Air Force, Changchun, Jilin, China

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_1, Ó Springer Science+Business Media B.V. 2012

3

4

Z. Ge and W. Li

elevation models (DEMs) has quickly increased. Today, spatial resolutions as high as 1 m are available for some areas, and raw data amount is even more than 100 GB. As the data is too large to fit into main memory, recently terrain rendering algorithm mainly utilize out-of-core method to save tiled data set in hard disk, preload required data to main memory and send data to GPU for rendering, which rendering process involves frequent disk access and large amount of data transmission. So, terrain rendering performance is limited by bandwidth restrictions rather than computational or rendering power. From this observation, several compression schemes for DEMs have been proposed, for instance, based on wavelet decomposition [1], space-filling curves [2], or tiled quadtrees [3]. And the geometric clipmap approach proposed by Losasso and Hoppe [4], which compressed large DEMs to a regular height map pyramid. It should be noted that the aforementioned approaches utilize CPU-based decoding of the compressed data streams. As a consequence, their throughput is limited by the computational power and memory bandwidth of the CPU as well as the bandwidth of the graphics bus. In this paper, we present geometry compression method for large-scale terrain rendering method, which dynamically transmit compressing data to graphic hardware. The domain was first tiled and computed into a nested geometric space error map and stored into displaying memory with terrain map. In order to obtain the seamless connection, the edge errors of two connecting tiles are consistent with each other. At run time, screen space error is projected from geometric error, and compare with error threshold according to view-point, to select terrain grid points by GPU. At last, triangle strip is constructed according to RQT method. The error metric method corresponds to parallel computing requires, and processes the error metric and triangulation by GPU, to improve the rending efficiency of image hardware, and save the running time of CPU.

2 Related Work The restricted constraint of restricted quadtree is that neighboring regions must be within one level of each other in the quadtree hierarchy. It was first presented by Von Herzen, and was applied for terrain visualization by Lindstrom in [5, 6]. Pajarola proposed an geometric error metric computing method with RQT In [7]. Error values of vertexes in triangle grid T are propagated and maximized along the dependency graph. The maximum error ensures that every vertex conform to error saturation and obtain accurate object-space geometric error. Then, Lindstrom presented an error metric algorithm, which is based on the accurate object-space geometric error of [7] and nested sphere hierarchy of [8], to transform the nested object-space geometric error into nested screen-space error for terrain simplification. This algorithm ensures both object-space geometric errors and screen-space errors are saturated. The same method also can be seen in paper [9], and so on.

A Large-Scale Terrain Rendering Method With GPU-Based Geometry Compression

5

Moreover, Today’s GPUs are able to sustain speeds of hundreds of millions of triangles per second. So, terrain rending algorithms mostly utilizes the performance of GPU. References [10, 11] partition the terrain into square patches tessellated at different resolutions, [4] directly loads compressed terrain data into graphics memory, and [12] still provides a seamless connecting method between patches. Those algorithms use batch technology of CPU and cache of graphics memory, largely improve rending efficiency. But [10] only computes static error, [11] largely simplifies error, and [4] even takes the distance to view point for error metric. So, it is still difficulty to take quick, GPU-based, optimized terrain rending effect. As scanned DEMs are typically much larger than the main memory available on desktop computers, data layout schemes on external storage devices [11, 13] in combination with visibility-based and speculative pre-fetching strategies as well as GPU occlusion queries have been developed [14]. Common to all these approaches is the goal to reduce both the amount of data to be transferred from disk and the number of seek operations to be performed. [15] provided quantitative evidence that a circle-shaped pre-fetching region is superior to a fan-shaped region, as it only requires slightly more memory but does not considerably increase bandwidth requirements, yet it allows fast changes of the viewing direction. It should be noted that although the aforementioned approaches can achieve similar or even better compression rates than our method, CPU-based decoding of the compressed data streams is inherent to all of them. As a consequence, their throughput is limited by the computational power and memory bandwidth of the CPU as well as the bandwidth of the graphics bus.

3 GPU-Based Error Metric Rendering Method On the basis of paper [13], our method computes saturated error of each vertex in terrain grid patch, and compose error map, which is loaded into graphic memory with terrain elevation map. Then we project geometric error to view-dependent screen space error of each vertex, and select vertexes to compose triangle strip by GPU according to the screen space error.

3.1 Error Metric Based on GPU In [13], by defining nested geometric error and nested sphere hierarchy, we obtain activeðiÞ , qðei ; Bi ; eÞ [ s , ðvei þ ri Þ2 [ di2

ð1Þ

The above expression involves only six additions and five multiplications, and corresponds to parallel computing requires. Therefore, error metric and active vertexes can be computed by GPU.

6

Z. Ge and W. Li

(vεi + ri) 2 cos θ εi

Warping & Adding

di

2

Comparing

ri

Fig. 1 GPU-based error metric and LOD selecting

For massive terrain data, we partition the terrain mesh into equal square tiles (size of 257 9 257), which elevation data are saved by terms of array. In preprocessing stage, we compute the nested geometric error of each vertex of every tile according to Eq. 2 during pre-processing, and saved by array too. Moreover, in order to project nested screen space error, we compute a nested bounding sphere radius array according to Eq. 1 for all tiles. During rendering, load DEM, geometric error, bounding sphere radius data of required tiles into display memory first according to view point parameter. At runtime, we take view point (e) and view angle(h) into account, Eq. 1 turn to: activeðiÞ , ðvei þ ri Þ2 cos h [ di2

ð2Þ

The process is shown in Fig. 1. Therefore, we compare one by one to get the LOD active vertexes according to the view point, and construct the triangle strip by means of RQT to render quickly.

3.2 Geometry Compression Our geometry compression method works on restricted quadtree meshes [10]. Mesh construction and compression is performed in a pre-processing step: Firstly, we tile the entire DEM. into square regions of 257 9 257 samples. Then we compute nested geometry error of each node, and organize the tile data in such a way that arrange each quad node according to nested geometry error and form an array, the front data have larger error. Because array is sorted as nested error, so when a node is loaded into memory, all of its ancestors must be in memory as well. The memory layout for any given node is shown in Fig. 2.

A Large-Scale Terrain Rendering Method With GPU-Based Geometry Compression

7

Fig. 2 Node order sorted by nested geometry error

After a tile is partitioned, the tile’s height values can be quantized. Using a uniform quantization at k bits, which can be computed as k ¼ max f dlog2 ðhmax  hmin Þe ; 0g where hmin and hmax denote the tile’s minimum and maximum height value. In all of our experiments, the smaller tile, the smaller number of bits k. But too small tile causes lower rendering efficiency, so in our method, we partition the terrain mesh into 257 9 257 tiles. Normally k is smaller than 16, and height can be stored in 16 bits. The same as height value, geometric error of tile nodes can be quantized too. Because ei \ hmax - hmin, so bit number of geometric error is smaller than that of height value, and geometric error can be stored in 16 bits too. Because our tile data are stored in special order as Fig. 2, thus we should store x/y-coordinates with height value for decoding. In our method, the x/y-coordinates can be stored in 8 bits. As a result, all data of a node use 48 bits, and data decoding can be performed entirely on the GPU using geometry shades as well as bit and integer arithmetic and rendering as the method of Sect. 3.1. The propose of our method is that data can be transmitted to display hardware as require, and save bandwidth from CPU to GPU, which is just the bottleneck of terrain rendering nowadays.

4 Results We used a 2.8 GHz Pentium Dual-Core E7400 PC, with 2G DDR2 of RAM, GeForce 9800GT graphics with 1G of graphics RAM, and SATA 500G disk. For all results, a data set over the Puget Sound area in Washington was used, which is

8

Z. Ge and W. Li

Fig. 3 Puget sound area, generated with our method (a), and its triangle construction (b)

Fig. 4 Examination result

made up of 16,384 9 16,384 vertices at 10 m horizontal and 0.1 m vertical resolution, and the texture data used pseudo-color. Our application is implemented by VC and OpenGL, and the window size is in all cases 1,024 9 768 pixels with 1 pixel screen space error thresholds. Application image is as Fig. 3, and the result of 1,000 frames is shown in Fig. 4. The frame rates are between 60 and 120 fps, and the number of triangles is 150,000 in average. Results from our analyses suggest that our method largely reduce the triangles number, and frame rates reached the real-time requirement of our travels. In summary, a GPU-based dynamic error metric and geometry compression technique for large, textured terrain rendering is presented here. By this method, the works of error metric computing is transferred from CPU to GPU without any rendering quantity reduction, which sufficiently utilizes the computing ability of GPU while leaving the CPU free to work on other important tasks. In order to obtain the seamless connection, we also keep the error consistency of two

A Large-Scale Terrain Rendering Method With GPU-Based Geometry Compression

9

connecting tiles. The result meets our rendering requirement. But, our method increases about 10% sizes of out-of-core files, which restricts the rendering efficiency in a certain extent. For the future, a method to improve loading speed of terrain files is necessary for the purpose of increase farther rendering speed.

References 1. Gobbetti E, Marton F, Cignoni P, DI Benedetto M, Ganovelli F (2006) C-BDAM— compressed batched dynamic adaptive meshes for terrain rendering. Comput Graphics Forum 25(3):333–342 2. Hoppe H (1998) Smooth view-dependent level-of-detail control and its application to terrain rendering. In: Proceedings of IEEE visualization’98 [C], New Caledonia, pp 135–142 3. Gerstner T, Rumpf M, Weikard U (2000) Error indicators for multilevel visualization and computing on nested grids. Comput Graphics 24(3):363–373 4. Losasso F, Hoppe H (2004) Geometry clipmaps: terrain rendering using nested regular grids. ACM Trans Graphics 23(3):769–776 5. Koller D, Lindstrom P, Ribarsky W, Hodges LF, Faust N, Turner G (1995) Virtual GIS: a real-time 3D geographic information system. In: Proceedings visualization 95, IEEE, Computer Society Press, Los Alamitos, California, pp 94–100 6. Lindstrom P, Koller D, Ribarsky W, Hodges LF, Faust N, Turner GA (1996) Real-time, continuous level of detail rendering of height fields. In: Proceedings SIGGRAPH 96, ACM SIGGRAPH, pp 109–118 7. Pajarola R (1998) Large scale terrain visualization using the restricted quadtree triangulation. In: Proceedings IEEE visualization, pp 19–26 8. Thomas gerstner: multiresolution visualization and compression of global topographic data (1999) Geoinformatica, 2002, To appear. Available as SFB 256 Report 29, University of Bonn 9. Bao X, Pajarola R, Shafae M (2004) SMART: an efficient technique for massive terrain visualization from out-of-core. In: Proceedings vision, modeling and visualization (VMV), pp 413–420 10. Ulrich T (2000) Rendering massive terrains using chunked level of detail. In: Super-size-it! Scaling up to massive virtualworlds (ACM SIGGRAPH Tutorial Notes), ACM SIGGRAPH 11. Cignoni P, Ganovelli F, Gobbetti E, Marton F, Ponchio F, Scopigno R (2003) BDAMbatched dynamic adaptive meshes for high performance terrain visualization. In: Proceedings EUROGRAPHICS, pp 505–514 12. Livny Y, Kogan Z, El-Sana J (2007) Seamless patches for GPU-based terrain rendering. In: Proceedings of WSCG, pp 201–208 13. Lindstrom P, Pascucci V (2002) Terrain simplification simplified: a general framework for view-dependent out-of-core visualization. IEEE Trans Visual Comput Graphics 8(3):239–254 14. Correa WT, Klosowski JT, Silva CT (2003) Visibility-based prefetching for interactive outof-core rendering. In: Proceedings of IEEE symposium on parallel and large-data visualization and graphics, pp 1–8 15. Ng C-M, Nguyen C-T, Tran D-N, Tan T-S, Yeow S-W (2005) Analyzing pre-fetching in large-scale visual simulation. In: Proceedings of computer graphics international, pp 100–107

Cots Sparse Matrix Utilization in Distribution Power Flow Applications Dino Ablakovic, Izudin Dzafic and Hans-Theo Neisius

Abstract Sparse matrix is used in many modern industrial applications, to provide low cost and high performance solution. Unlike a number of previous works in the field of distribution networks, which analyzed the sparse matrix calculations as integral part of its calculation engine, this paper presents the considerations and analysis for the utilization of commercial of-the-shelf sparse matrix solvers (COTS SMS). Analysis is given for the parallel direct sparse solver utilized in unbalanced and unsymmetrical Distribution System Power Flow solution, which maximizes parallelization on multi-core processors. Optimal matrix population algorithm and container class from the power network object model is proposed for the fast population, search and access to the matrix elements. Optimal use cases of symmetrical and unsymmetrical matrices for the given method are analyzed. Test results for the Current Injection—Power Flow algorithm on a Three-Phase large scale system are presented which show the performance scales for symmetric and unsymmetrical matrices over a scope of system sizes and for different network topologies. Keywords Power flow

 Sparse matrix  Distribution network  Three-phase

D. Ablakovic (&)  I. Dzafic  H.-T. Neisius Siemens AG, Humboldtstrasse 59, 90459 Nuernberg, Germany e-mail: [email protected] I. Dzafic e-mail: [email protected] H.-T. Neisius e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_2,  Springer Science+Business Media B.V. 2012

11

12

D. Ablakovic et al.

1 Introduction Distribution Power Networks have changed immensely in recent years, becoming larger and more complex every day. Distribution generation increased the problem complexity by far. Networks became much larger with the need for interconnection between different networks on distribution and medium voltage level. Three-Phase unbalanced and unsymmetrical networks, typical for the USA, require three-phase analysis. All of this requires better, and more performant DSPF application. Advanced DSPF solutions can utilize sparse matrix as the system impedance/ admittance matrix depending on the methods used. Introduced in 1960s to the power systems calculation [1], sparse matrix utilization evolved to present times as the crucial element of DSPF. Nodal matrix method, Newton–Raphson and FastDecoupled DSPF algorithms in all different variations make a great use of sparse matrix. However, almost all previous DSPF solutions utilize sparse matrix internally, and all operations such as factorization and eventually forward backward substitution are deeply integrated into the DSPF algorithm. It has long been realized that DSPF parallelization with different technologies and hardware can provide significant performance improvements [2–4]. Different factorization algorithms can be differently parallelized. From the perspective of DSPF application, it has become very hard to choose the right algorithm, hardware and implementation technique. The problem for the industrial real-time DSPF application is exponentially more complex when design and development time, and costs are considered. The aim of this paper is to analyze the utilization of Commercial Of-the-shelf Sparse Matrix Solvers (COTS SMS) for the DMS and more specifically DSPF, which are now more than ever, to maximize the power of parallel processing through multi-core CPU processors or even GPUs [5].

2 Cots Sparse Solvers in DSPF Unlike so many previous works in research and industry which implement their own direct sparse solvers for the DSPF, COTS solvers offer equally performant, but much faster to implement and more feasible solution. A SWOT analysis of using COTS SS, which are elaborated here, is given in Table 1. Two types of sparse solvers in general use in industry are: Direct and Iterative solvers. Both types have advantages and disadvantages, depending on the application. For the Power Systems in general, research up-to-date shows that Iterative method is still not developed enough, having convergence problems with different pre-conditioners. Direct sparse solvers have more predictable convergence, but require large memory to hold the factorized matrix. They are however, more stable, robust and reliable, and for those reasons found greater acceptance in DSPF applications and power systems in general.

Cots Sparse Matrix Utilization Table 1 SWOT analysis of COTS SMS Strengths Developed by dedicated sparse solver entities Very feasible, no special development needed Clear interface to network applications Updateable without additional development Opportunities Make best use of parallel processing, multi-core CPU or GPU, FPGA Ability to chose different SS for different hardware, operating systems and projects on demand Updateable without additional development at small cost

13

Weaknesses Special interface needed Wrappers for all used COTS SS necessary Special container class needed, additional memory used Dependency on 3rd party entity Threats Eventual bugs in 3rd party software Product update and support discontinuity Compatibility issues Portability issues

Numerical evaluation of sparse direct solvers for the solution of large sparse symmetric linear systems of equations [5], analyses 10 different COTS solvers and provides the performance profile for different matrix types for each of them. Although the referenced benchmarking work is done only for symmetrical solvers with real numbers, many of the tested ones support also unsymmetrical matrices, and almost all support complex numbers. This is crucial consideration for choosing the right solver, depending on the DSPF implementation. A few direct sparse solvers are developed directly by the processor vendors which can only emphasize the maximum output from their most deep knowledge of the hardware on which the solver runs. According to the Moore’s law the processor performance is doubled every 18–24 months [6]. With currently available 8 and 12-core processors, parallel sparse solvers are inevitable in serious DSPF application. Other than processor vendors, there are serious research entities dedicated to the sparse solver development which are more concentrated on the algorithms for improving the solvers, by constantly finding better solutions for ordering, pivoting and factorization algorithms. This dedicated development of sparse solvers is very time and resource consuming and can be too big task for Power Systems development groups. Given that hardware development is so fast that it constantly requires revision of the sparse solver algorithms and re-implementation and the above stated arguments it can be concluded that COTS SMS can be feasible, performant, up-to-date solution for DSPF. From all of the above, requirements for the COTS SMS in DSPF are defined as follows: 1. 2. 3. 4. 5. 6.

Direct solver type Support float and double complex numbers Must support symmetrical and unsymmetrical matrices Platform independent (Windows/Linux) Acceptable licensing/pricing model Acceptable upgrade/development roadmap

14

D. Ablakovic et al.

Fig. 1 COTS SMS interface

cNetObjModel

cDSPFSolver

«interface» iCOTS_SS

cIterativeSS

1 * Iterative SS library

cDirectSS

cMultigridSS

1 * Direct SS library

1 * Multigrid SS library

cFPGASS

1 * FPGA SS library

3 Interface and Wrappers of Cots Sparse Solver The aim is to enable DSPF and DMS to integrate any COTS SMS, and to replace it at lowest possible cost when better one is available, or even to be able to use more than one solver depending on the available hardware platform, operating system. Solution must be so robust to integrate iterative, direct and even GPU and FPGA solvers. Since there are a number of different solvers encapsulated in different libraries, which are created in different programming languages, the Interface must be defined with specific minimum functionalities, each solver must provide. Class wrappers must be used to encapsulate the solvers, which will then in Object Orientated Design OOD inherit from the defined interface. If C++ is the chosen language, as it is widely used in the DMS development, Fig. 1 presents the proposed interface. Template capabilities of C++ are to be used for the input of data types of the solver. The first step is defining the iCOTS_SS interface and its functions, which means that all inheriting wrappers will have to provide those. Solvers main functions are obviously factorization and then solving the system of linear equations with factorized matrix.

4 Sparse Matrix Container There are many methods for storing the matrix data, i.e.: compressed row, compressed column, block compressed row storage, diagonal storage, jagged diagonal storage etc. All of those sparse storage schemes use the continuous storage

Cots Sparse Matrix Utilization

15

memory allocation for the nonzero elements of the matrix, or eventually a limited number of zeros when indefinite matrices are used. Overview of the standard formats given in [8, 9] shows that the compressed row and column storage formats are the simplest because they don’t consider the scarcity structure of the matrix, and they don’t store any unnecessary elements. Because of it, they are on the other hand not very efficient, needing an indirect addressing step for every single scalar operation in a matrix–vector product or preconditioned solve. An example of compressed row format for the symmetric matrix with real numbers is given in (1) where only upper triangular elements are stored saving the space of both 0 and symmetric values.

The length of the value and column index arrays is actually the number of non-zero elements in the upper triangular matrix, and the row pointer array gives the location of the first non-zero element within a row. Power networks topology is dynamic in its nature. Switching equipment changing its statuses define the network topology, combining and separating sub-networks. Having different sub-networks and topologies requires an implementation of tracing or graph theory coloring algorithm to collect all galvanically connected network elements, from which a sub-network is composed, at one point in time, by which the network object model is created. Bus bars are essentially graph vertices and branches, i.e. lines and transformers are graph edges. Switches are in general DSPF calculation ignored because their impedances are, when compared to branches, negligible. When one sub-network object model is created, sparse admittance matrix vectors need to be populated. Because of its stacked sparse format, population process requires special sparse matrix container class which will provide fast population and access to matrix elements. Since the benchmark solver is using compressed row format shown in (1), it means for both algorithms that matrix must be populated by sweeping columns for each row, one by one. Because of virtually unknown number of non-zero elements the container must have dynamic size and grow as elements are inserted. This means no continuous memory allocation, but use of single linked lists. This container class is essentially made of one vector-array containing the first column of struct elements defined above. The array itself uses continual memory allocation and therefore the access to each row can be done in a single step. Later when row is found, iteration over the elements in the row by using a pointer to each next element and comparing the column value is needed. It is represented on Fig. 2.

16

D. Ablakovic et al.

Fig. 2 Advanced sparse matrix container structure

5 Symmetrical Matrix in DSPF Unbalanced and unsymmetrical DSPF calculations in phase domain require admittance sparse matrix YABC of size 3n, where n is the number of electrical nodes and 3 is a phase multiplicator. Unsymmetrical matrix factorization is in general slower than the symmetrical matrix factorization because of the more complex permutation calculation and larger fill-in calculation. Full unsymmetrical matrix size is n2 and symmetrical (n2 ? n)/2. To achieve the best utilization of the matrix in DSPF, with the proposed modeling two main goals are: 1. Make only one Y admittance matrix factorization 2. Make the Y admittance matrix symmetrical Nodal admittance matrix YABC contains the branch admittances, i.e. lines and transformers. Loads are modeled in an injection vector, because they are dependent on the calculated voltage, which in the end makes this method iterative calculation process. When local control algorithm is implemented with DSPF, tap changer positions and capacitor banks are also considered in an injection vector. To improve the method solvability, matrix numerical conditioning and decrease the number of iterations, advance modeling can be utilized in following: 3. Model constant impedance loads in the matrix 4. Model initial tap positions and capacitor banks in the matrix, and final tap positions simulate as current injections Therefore general equation for the Current Injection method is presented in (3) ½YABC   ½V  ¼ ½IL  þ ½ID   ½IS  ¼ ½I   ½IS 

ð2Þ

Cots Sparse Matrix Utilization

17

where: IL ID IS

—Load injection vector calculated from the voltage dependent loads. —Injection vector calculated from tap positions of load tap changers and capacitor calculated in local control simulation. —Injection source (Slack) compensation vector.

To avoid first calculation iteration, instead of filling the slack voltage values to (V) and doing one forward–backward calculation, IL vector is initially set as given in (4). Because of mutual coupling between phases and the affect on the angle, initial Slack values at node k must be set with 120 phase shift. IkA ¼ 1\0 ¼ 1 þ j0p:u IkB ¼ 1\  120 ¼ 0:5 þ j  0:8660p:u IkC ¼ 1\120 ¼ 0:5 þ j0:8660p:u

ð3Þ

Is is a compensation vector which ensures that Injection sources—Slack Buses are kept at the referent voltage during the DSPF iterations, keeping the matrix symmetrical at the same time. Isj ¼ YABCjk  Ikn¼0 n k

ð4Þ

Where: —iteration number = 0 —Index of the Slack Bus 

1 0

       1 IS1  0   VS ¼  Y~ABC V1n IS1n I1n

ð5Þ

When all Current injection vectors are added it still has to be ensured that values for Slack Buses are as given in (5), which means all branches and loads on the Slack Buses are ignored. Important to note is that Is vector must be calculated only once and in the first iteration of the Power Flow when only Slack values are populated. The Load vector IL is calculated in each Power Flow iteration and Delta ID in each Local Control Iteration.

6 Result and Analysis This paper presents the results of DSPF Admittance matrix population and factorization with implemented proposed sparse matrix container against the balanced tree container found in C++ Standard Template Library (STL). The test network is created from modified IEEE 34 test feeder [10] by multiplying the feeder 148

18 Table 2 Number of equipment units in the network

D. Ablakovic et al. Type of equipment

Number of equipments units

Bus bars Transformers Lines Loads Capacitors Switching devices

5.071 390 4.770 2.250 180 10.381

(ms)

Fig. 3 Matrix population and factorization times for radial network

350 300 250 200 150 100 50 0

313

196 107 70

Unsymmetric

60

std::map

66

AdvSparse

matrix type Matrix population

Factorization

times and adding them in parallel to the same feeder head bus bar. Additionally connection line segments on Buses 864 and 840 are added in order to provide the meshed network topology. In Meshed configuration two loops are made between each two feeders on the given buses. The network size in total is given in Table 2. Test Application: DSPF—Distribution System Power Flow Benchmark Test Server: 2 9 Intel Xenon(R) E5420 CPU 2.5 GHz, RAM 24 GB Num of processor cores: 2 9 4 = 8 cores Operating System: Windows Server 2003 SP2 Sparse solver: Intel MKL Pardiso [7] Test Tool/Methods: • Win32 API timing functions for profiling used directly in code utilizing number of processor ticks with precision to millisecond (ms). • No code optimization and no SSE2 instructions in compiler optimization. • Average calculation times on 10 runs are presented Results presented show that both matrix population and factorization times can be significantly improved with usage of symmetrical matrix alone, even with usage of Standard Template library collections. Utilizing the Advanced Sparse Matrix container, population time is far better and shows the importance of giving so much attention to the entire process of sparse matrix utilization. Results are presented on Figs. 3 and 4, Table 3.

Cots Sparse Matrix Utilization

19

Fig. 4 Matrix population and factorization times for meshed network

350

320

300 (ms)

250

211

200 150

111 78

100

64

74

50 0 Unsymmetric

std::map

AdvSparse

matrix type Matrix population Factorization

Table 3 Results of comparison tests for radial and meshed networks Unsymmetrical Symmetric Symmetric std::map AdvSparseMat Radial networks Population Factorization Meshed Population networks Factorization

313 107 320 111

196 70 211 78

60 66 64 74

Finally, from the presented results it can also be seen that results are worse for Meshed network types, which is result of more off-diagonal elements and spread matrix structure. A topic for further research is to determine exactly how the network topology affects the matrix population and factorization times.

References 1. Tinney WF, Walker JW (1967) Direct solutions of sparse network equations by optimally ordered triangular factorization. In: Proceedings of the IEEE, Nov 1967 2. Wu JQ, Bose A (1995) Parallel solution of large sparse matrix equations and parallel power flow. IEEE Trans Power Syst 10(3):1343 (August) 3. Feng Z, Zeng Z, Li P (2010) Parallel on-chip power distribution network analysis on multicore-multi-GPU platforms. IEEE Trans Very Large Scale Integr VLSI Syst, vol PP, Issue: 99 4. Johnson J, Nagvajara P, Nwankpa C (2004) Sparse linear solver for power system analysis using FPGA, Drexel University, HPEC 5. Gould NIM, Hu Y, Scott JA (2005) A numerical evaluation of sparse direct solvers for the solution of large sparse, symmetric linear systems of equations, Technical report, RAL-TR2005-005, CCLRC Rutherford Appleton laboratory 6. Ramanathan RM Intel Multi-core processors: making the move to quad-core and beyond, White Paper. Intel Corporation 7. Intel Math Kernel library reference manual—March 2009 630813-031US 8. Templates for the solution of algebraic eigenvalue problems: a practical guide (2000) In: Bai Z, Demmel J, Dongarra J, Ruhe A, van der Vorst H (eds) SIAM, Philadelphia

20

D. Ablakovic et al.

9. Distributed sparse data structures for linear algebra operations (1992) Technical report CS 92–169, Computer science department, University of Tennessee, Knoxville, TN, LAPACK working note 10. IEEE 34 node test feeder, IEEE distribution system analysis subcommittee

A New Graph-Based Algorithm for Persian Text Summarization Hassan Shakeri, Saeedeh Gholamrezazadeh, Mohsen Amini Salehi and Fatemeh Ghadamyari

Abstract Nowadays, with increasing volume of electronic text information, the need for production of summary systems becomes essential. Summary systems capture and summarize the most important concepts of the documents and help the user to go through the main points of the text faster and make the processing of information much easier. An important class of such systems is the ones that produce extractive summaries. This summary is produced by selecting most important parts of the document without doing any modification on the main text. One approach for producing this kind of summary is using the graph theory. In this paper a new algorithm based on the graph theory is introduced to select the most important sentences of the document. In this algorithm the nodes and edges will be assigned with different weights and then the final weight of each one will be defined by combining these values. This final weight indicates the importance of the sentence and the probability of appearing this sentence in the final summary. The results show that considering simultaneous different criteria generate a summary which is more similar to human one.









Keywords Summarization Persian texts Sentence’s graph Extract Cohesion

H. Shakeri (&)  S. Gholamrezazadeh  M. A. Salehi  F. Ghadamyari Islamic Azad University, Mashhad Branch, Mashhad, Iran e-mail: [email protected] S. Gholamrezazadeh e-mail: [email protected] M. A. Salehi e-mail: [email protected] F. Ghadamyari e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_3,  Springer Science+Business Media B.V. 2012

21

22

H. Shakeri et al.

1 Introduction Nowadays, human faces a large amount of information every day. Significant parts of this information are in text format. Due to this, there is a demand for tools that accelerate the reading and comprehending text documents. Summarization systems enable us to read the most important parts of each document and so increase the speed of reading and comprehending. There are various definitions for summary. Edward Hovy et al. [1] defines the summary as a text that is based on one or more texts; it has the most important information of the main texts and its content is less than half of the main texts. Mani [2], describes the text summarization as a process of finding the main source of information, finding the main important contents and presenting them as a concise text in the predefined template. Some factors such as the language of input text are so challengeable. For example in Persian texts, multi meaning and multi functional words are one of the challenges in summarizing these texts. In [3] the challenges in Persian texts processing is discussed completely. There are three main steps for summarizing texts [4]. These steps are topic identification, interpretation, and summary generation. In topic identification step, the most prominent information in the text is identified. Most of the systems, assign different precedence to different parts of the text (sentence, words, and phrases); then a fuser module mix the scores of each part in order to find the total score for a part. At last, the system presents the N highest score parts in final summary. Several techniques for topic identification have been reported such as methods based on Position, Cue Phrases, word frequency and content counting [1]. Abstract summaries need to go through interpretation step. In This step, related subjects are combined in order to form a general concise content [5] and the additional phrases are omitted. Inferring the topics is difficult; therefore most of the systems generate the extract summary. In summary generation, the system uses text generation method. This step includes a range of various generation methods from very simple word or phrase printing to more sophisticated phrase merging and sentence generation [9]. In other words, the natural language, which is understandable, by user is generated here. The summarization systems are categorized based on the type of generated summary. In this paper we focus on extractive summaries. An extractive summary is generated by selecting sentences from the main text to form a summary without any modification of their original wording. Up to now, many different techniques have proposed to select the most important part of the text such as statistical methods which includes Aggregation Similarity Method [6], Location Method [7], Frequency Method [8], TF-Based Query Method [9], linguistic methods which includes Graph Theory, Lexical Chain, WordNet and Clustering. Graph is an appropriate approach for presenting the relation between sentences in a way that the relation between each two sentences can be shown independent the other ones. Lexical chain uses WordNet in order to identify the relation between words and put them in a chain. Since there is not any WordNet for Persian words, we cannot

A New Graph-Based Algorithm

23

use this technique in our system. Clustering technique makes some clusters from sentences and relates them to each other. Clustering method is unable to examine the relation between sentences in a text whereas graph technique provides this relation for us. Considering these reasons, in this paper we propose a graph-based algorithm for summarizing Persian texts. The structure of this paper is as follows: in Sect. 2, the categorization criteria for summarized text will be discussed. In the third section, we discuss the graphbased approach for summarization and introduce systems which are designed based on it. In the forth section, we introduce our proposed algorithm. In Sect. 5, the experimental results will be analyzed and finally in Sect. 6, the result will be discussed.

2 Related Works Based on the output summary, the text summarization systems are classified. These categorizations are discussed in [10]. So far, many systems which generate a kind of summary that presented in Fig. 1 are designed. Most of these systems are designed for English texts and other language such as Japanese, Spanish and others. But unfortunately very few efforts have been done for Persian texts. The most important systems for Persian text summarization are introduced here. FarsiSum: FarsiSum [5, 11] is a text summarization system for Persian newspaper text/HTML. It uses modules implemented in SweSum [14], a Persian stop list in Unicode format and a small set of heuristic rules. The summarization process includes three phases: Tokenizing, Scoring and Keyword extraction. Words in the document are converted from ASCII to UTF-8 and then compared with the words in stop-list. Automatic Persian Text Summarizer: this system uses a hybrid method to summarize Persian texts automatically. In this system, the below techniques are used to select the sentences which should be presented in final summary: Lexical chains, summarization based on graphs, selecting important sentences based on cue words, number of similar sentences, similarity between sentences and similarity with topic and query. For more details see [12]. Hybrid Farsi text summarization: in [13] a technique based on term co-occurrence and conceptual property of the text is defined. In this method for each two words, the co-occurrence degree is computed. Then lexical chains are created and n top ranked words are selected. After that, a graph is created with words as its nodes. The edges are drawn based on co-occurrence degree between words. The sentence score is computed by summing the gain of all its words and finally n top ranked sentences are selected.

24

H. Shakeri et al. Constructing Graph Algorithm Input: sentences from the main text, Base node Output: sentences which make summary 1. For each sentences take part in constructing graph 1.1. Calculate number of edges connected 1.2. Calculate total frequency of words 1.3. Count number of key words 1.4. Count multiple selection criteria simultaneously 1.5. Calculate the rate of deviation from base node //base node is calculated earlier by formula number 1 and is identified for this algorithm as input 2. For each two sentences 2.1. Count shared words 2.2. Count shared key words 2.3. Count common English word 2.4. Count common word with explanation as footer 2.5. Determine if two sentences are located in one paragraph 3. For each sentences calculate formula number 4

Fig. 1 The proposed summarization algorithm based on graph

3 Applying Graph in Extractive Summaries Using the graph for displaying the structure of the text will help us to better understand the connection between different parts. [10]. Graph-based algorithms use a ranking algorithm to rank different sections of a text where each section is considered as a node. Ranking algorithms use various criteria in order to sort the nodes based on priority. The nature of nodes and edges will be defined by the type of text. For example, some sections of the text, words, or sentences can be considered as the nodes. Edges will represent the lexical or semantic connection, or commonalities between the two nodes. Regardless of the type and characteristics of the text that we want to draw the graph for it, a graph-based ranking algorithm includes the following basic steps: 1. Identify units of text-which can include phrase, word or other units- and to consider them as vertices in the graph. 2. Determine the relationships that these units were related to making and using these relationships to drawn edge between the vertices of the graph. The edges can be directed/undirected and weighted/unweighted. 3. Run the graph ranking algorithm repeatedly until all entities (nodes) are sorted according to priority. 4. Sort the vertices according to their rank. As it is stated in the fourth step, after specifying the final score for each node, the nodes are ranking based on their final score. Then, depending on compression rates—Compression Rate defines how the main text should be shortened or determines the length of summary text- in the desired text, sentences with the highest score are selected to attend in final summary. LexRank [15] and TextRank

A New Graph-Based Algorithm

25

[16] are two of the most important algorithms based on the graph. Following, we examined each of these algorithms briefly. LexRank: Erkan and Radov [15] designed LexRank in order to summarizing text in multi-document systems. It is assumed that a sentence which is similar with many other sentences in a cluster, is more central (more important) and closer to the subject. In this algorithm, a fully connected and undirected graph is plotted for the sentences of each cluster. If two sentences share similarities, an edge is drawn between them. The cosine similarity is used to calculate the similarity between two sentences. After the calculation of similarity between sentences and construct a graph, we specify the central sentence using graph for each cluster by the following order. They define a degree of centrality for each sentence which is the number of similar sentences to the desired one. The sentence with the highest degree of centrality is the central sentence. TextRank: TextRank is a graph-based ranking model which is used for all graphs that derived from natural language texts. TextRank is derived from Google page ranking [9] model and is designed for use in single document summarization systems. TextRank is used to extract key words and sentences. A fully connected and undirected graph is used to extract sentences. Each sentence is considered as a vertex (node) in graph. To make an edge between two sentences, a similarity relation is used which is measured as a function of joint concepts. Each edge is also weighted that indicates the importance of relationship. Sentences based on their scores are ranked and the sentences with the highest score are selected [16]. SUMGRAPH [8] and Time stamped Graph [17] are two other summarization systems which are designed based on the graph.

4 Proposed Method As mentioned earlier, in many of current methods, the most important sentences are selected based on final weight of nodes whereas the weight of edges is solely used to determine the weight of nodes. The method proposed in this paper tries to involve all existing relations between sentences in determining the most important sentences. Moreover, the importance of sentences independently is considered. In other words, we consider both the importance of each sentence and the importance of relation between sentences. The reason is that if the content of a sentence is not important, it is worthless for the system, no matter how close is the relation with other sentences. The strength of this algorithm is addressing the importance of sentences independently and simultaneously the relations between them. In this algorithm, a connected and undirected graph is used. We consider undirected graph because it is appropriate well for graphs with weak links [7]. Sentences considered as nodes and relation between them is shown by edges. We consider a weight for nodes and for each edge. Weight of each edge defines the

26

H. Shakeri et al.

degree of importance of the relation between two sentences. The following criteria are used for weighting nodes: • • • •

Number of edges connected to node. Frequency of words in one sentence. Number of keywords in a sentence. Having multiple selection criteria simultaneously (criteria based on them sentences are selected from original text to form a graph). • Rate of deviation from the base node: the base node is the one which is known as the most important and key sentence in the text, and contains the main subject of the text. To determine the base node, we combine weight of all nodes and work out a value for every node. Then we select the highest value and consider the sentence related to it as the base node. For combining weights we apply the following formula: X5 C Wi ð1Þ Tw ¼ i¼1 i where Tw is the overall weight, Ci is coefficient dedicated to ith criteria which indicates the percentage importance of this criterion and Wi is the numeric value of ith criteria which is obtained for a specific sentence. Ci is obtained experimentally and based on a research conducted on the characteristics of Persian texts. After identifying the base node, we calculate the deviation of each sentence from the base node by formula (2). diff ¼

NI D

ð2Þ

where diff is the deviation of each sentence from the base node, N is the number of words in sentences, I is the number of common words with base node and D is the diversity words of the original text. D is calculated by reading text one time and considers each word in its first presence in the text. After calculating the deviation, we reduce the obtained number from the P/D fraction where P is the number of words in base node. Thus, the similarity of each sentence with base node is obtained. S¼

P  diff D

ð3Þ

Indeed, the sentences which have more similarity with the base node are more related to the topic of the main text. Based on the following criteria, two sentences are related to each other and an edge should be considered between them. These criteria are: • Number of words shared between two sentences • Number of keywords shared between two sentences

A New Graph-Based Algorithm

27

• Having common English words (this system is designed for Persian texts) • Having words with common explanation as footer • Existence of two sentences in a paragraph After constructing the graph, ten weights are obtained. Five weights are for nodes—former criteria—and five weights are for edges—latter criteria. Then, all these ten weights for all of the nodes and edges are combined using formula (4) and consequently a final weight for each node is calculated. For each node, the weights assigned to itself and weights of edges which is connected to, are combined. X10 CW ð4Þ Tw ¼ j¼1 j j where Tw is final weight, Ci is coefficient for ith criteria, and Wi is the numerical value for ith criteria which is obtained for a specific node. The formula 4 is the same formula 1 but the boundaries are changed. In formula 1, we combine just five weights which are obtained by weighting nodes criteria in order to identify the base node whereas the formula 4, combines ten weights obtained by weighting edges criteria addition to weighting nodes, to calculate a final weight for each node. We assign higher values to the nodes weighting criteria because if one sentence has no importance itself, is not appropriate to attend in the final summary. Another benefit of the proposed graph to previous algorithms is taking into account the degree of importance for sentences and relationship between them, simultaneously. It enables us to choose the sentences which have main content and are related to the others. This cause the final summary has more cohesion and become more similar with the human one. The following pseudo-code shows the steps of our proposed algorithm.

5 Performance Metrics To evaluate a text summarization system, two widely used metrics are: Precision and Recall [18]. These two metrics are used just for evaluating extractive summaries. Recall is the fraction of sentences chosen by the person that were also correctly identified by the system. A person is asked to select sentences that seem to best convey the meaning of the text to be summarized. Then selected sentences automatically by system are evaluated against the human selection. Recall ¼

system  humanchoicesoverlap sentencechosenbyhuman

And Precision is the fraction of system sentences that were correct.

ð5Þ

28

H. Shakeri et al.

Table 1 Comparison result between proposed algorithm and FarsiSum

Precision Recall F1 ROUGE-1

FarsiSum

Proposed method

0.37 0.52 0.43 0.43

0.52 0.67 0.58 0.62

Precision ¼ ðsystem  human choice overlapÞ=ðsentence chosen by systemÞ ð6Þ F1 is a weighted average of the precision, recall and calculated by following formula [19]: F1 ¼ ð2  ðprecision  recallÞÞ=ððprecision þ recallÞÞ

ð7Þ

ROUGE-N [20] is another criteria which is widely used in evaluating summaries. ROUGE-N is calculated as follows: P P S2fRefrence summariesg gramn 2S Countmatch ðgramn Þ P ð8Þ ROUGE  N ¼ P S2fRefrence summariesg gramn 2S Countðgramn Þ where N stands for the length of the n-gram, gramn , and Countmatch ðgramn Þ is the maximum number of n-grams co-occurring in a candidate summary and a set of reference summaries.

6 Experimental Results In order to evaluate the proposed method, we compare the output of this system with FarsiSum [5] system. Evaluation criteria like Precision, Recall, standard F1and ROUGE-1 are used for comparison. For this purpose, ten scientific papers about computer technology were summarized by FarsiSum system and the algorithm which is presented in this paper. Also, these papers were summarized by a human expert and we consider this human summary as the reference. Compression rate is set to 50%. The results are listed in Table 1. As it is shown in Table 1, the precision and recall and ROUGE-1 are improved. We noticed that considering more criteria and specifically taking English words into account is the reason of difference between our approach and approaches applied in FarsiSum. In Persian texts to avoid mistakes and misunderstanding, specialized words are quoted in the original language. We consider these words as clues that express the prominent parts of a professional text. On the other hand, we consider relationship between two sentences in addition to the importance of sentences. This helps generated summary to be more similar with the human one. In fact, for the sake of cohesion and clarity, human summarizer (expert), chooses sentences which are related to each other.

A New Graph-Based Algorithm

29

7 Conclusion In this article we first review the text summarization systems as well as summary classification criteria. Then, we propose a new method based on graph theory to create an extractive summary for Persian texts. The aim of this method is to consider the importance of sentences independently and at the same time the importance of the relationship between them. Thus, the sentences are selected to attend in the final summary contains more important subjects, and also have more contact with other sentences. As a result, we notice that the sentences in summary text have relationship with each other and become closer to the human generated summary. Evaluation results indicate that the output of proposed method improves precision, recall and ROUGH-1 in comparison with FarsiSum. This algorithm is a part of text summarization system. In future we plan to fine tune the output of this algorithm. For this goal, we can add some additional processing steps as a post processing step to the system. This step can involve processes such as finding the reference of pronouns in the text and replace them, depends on the genre of the text some sentences can be omitted and reduce the redundancy. Additionally, having richer database improves the accuracy of summary.

References 1. Frankel, David S (2003) Model driven architecture: applying MDA to enterprise computing. OMG Press, Wiley, New York 2. Mani I (2001) Automatic summarization John Benjamin’s publishing Co, pp 1–22 3. Shamsfard M (2007) Processing persian texts and its challenges. In: The second workshop on Persian language and computer. pp 172–189. (in Persian) 4. Lin CY, Hovy EH (1997) Identify topic by position. In: Proceedings of 5th conference on applied natural language processing, March 1997 5. Mazdak N (2004) A Persian text summarizer, master thesis, department of linguistics, Stockholm University, Jan 2004 6. Kupiec, Jullian M, Schuetze, Hinrich (2004) System for genre specific summarization of documents, Xerox corporation 7. Rada M (2004) Graph-based ranking algorithms for sentence extraction, applied to text summarization, annual meeting of the ACL 2004, pp 170–173 8. Patil K, Brazdil P (2007) Sumgraph: Text summarization using centrality in the pathfinder network. IADIS Int J Comput Sci Info Sys 2:18–32 9. Wills RS (2006) Google’s pagerank: the math behind the search engine 10. Saeedeh G, Mohsen AS, Bahareh G (2009) A comprehensive survey on text summarization systems’’. CSA 2:462–467 11. Martin H, Nima M (2004) A Persian text summarizer. In: International conference on computational linguistics 12. Zohre K, Mehrnoush S (2007) A system for automatic persian text summarization. In: 12th international CSI computer conference, (in Persian)

30

H. Shakeri et al.

13. Azadeh Z, Behrouz M-B, Mohsen S (2008) A new hybrid farsi text summarization technique based on term co-occurrence and conceptual property of the text, In: 9th ACIS international conference on software engineering, artificial intelligence, networking and parallel/ distributed computing 14. Dalianis H (2000) SweSum—A text summarizer for Swedish, Technical report, TRITA-NAP0015, IPLab-174, NADA, KTH, Oct 2000 15. Erkan G, Radev DR (2004) LexRank: graph-based centrality as salience in text summarization, J Artif Intell Res 22, pp 457–459 16. Rada M, Tarau P (2004) TextRank: bringing order into texts. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP 2004) 17. Lin Z (2006–07) Graph-Based methods for automatic text summarization, Ph.D. thesis, school of computing National University of Singapore 2006–07 18. Nenkova A (2006) summarization evaluation for text and speech: issues and approaches, Stanford University 19. Norshuhani Z, Arian G (2010) A hybrid approach for malay text summarizer, The 3rd international multi-conference on engineering and technological innovation 2010 20. Lin C (2004) Rouge: a package for automatic evaluation of summaries. In: proceedings of the workshop on text summarization branches out, 42nd annual meeting of the association for computational linguistics. 25–26 July, Barcelona, Spain, pp 74–81

The Factors Affect User Behaviour in a Picture-Based User Authentication System: PixelPin Zhang Min, Brendan Ryan and Sarah Atkinson

Abstract Nowadays alphanumeric passwords are still widely used, even though it causes the well-known ‘password problem’. There are a lot of studies on alternative user authentication schemes which try to take place of the textual password. PixelPin is one of the picture-based authentication systems, which are base on Blonder’s patent (Graphical Passwords. United States Patent 55599611996). This paper is the report on our evaluation of the PixelPin by conducting four experiments. In specific, we summarized the issues of PixelPin’s usability and security via web-based user study and focus group. We also tried to understand users’ choice of picture and the way of their memorization of the click-points. A cursor effect was studies in the last experiment. Finally, some issues are proposed to be solved in the future work. Keywords User authentication


Cued recall-based picture passwords
PixelPin

1 Introduction User authentication is a pivotal part of the computer security. After first introduction in the late 1960s [3], alphanumerical passwords are ubiquitous authentication today, using in computer systems, networks, Internet-based systems etc.

Z. Min (&) Horizon DTC, School of Computer Science, University of Nottingham, NG8 1BB, Nottingham, UK e-mail: [email protected] B. Ryan
S. Atkinson Human Factor Research Group, University of Nottingham, Nottingham, UK

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_4,  Springer Science+Business Media B.V. 2012

31

32

Z. Min et al.

As the development of the information technology, more and more systems require users to remember their account names and passwords. In addition, some systems ask users to change passwords regularly. According to the studies conducted by Dhamija et al. [9], human beings are merely capable to remember a limited number of passwords, the proliferation of passwords-system increase the interference [42], therefore people often write down [1], share and use the same or similar passwords for all different system accounts [15]. Moreover, surveys show that frequent passwords are short and meaningful words [26], such as ‘password’, friends’ or family members’ names or names of pets, etc. [6, 32]. The alphanumeric password is thought to be in a dilemma between security and usability, that is, the well-known ‘the password problem’ [6, 19, 32]. Briefly, on one hand, passwords should be easy to remember and authenticated quickly; on the other hand, passwords should be random, should be different on one user’s different accounts and should be changed frequently. Many alternative solutions have been proposed with the aim of finding the best trade-off between security and usability, which can reduce the memory issues on the basis of security rather than at the expense of security. For instance, the token-based authentication (i.e. smart-card, bank card), biometric authentication [17] (i.e. the fingerprint, iris [30], face recognition [20], speech matching [22], etc.). However, these schemes are also accompanies by vulnerabilities and shortcomings, in specific, the expensive hardware and support including deployment and maintenance; the unresolved issues of privacy, impersonation [25], etc. The graphical passwords have been devised based on psychology theory that human can remember pictures better than alphanumeric characters [33] and the ‘picture superiority effect’ [26]. It is thought that the key cause for ‘the passwords problem’ lies in the memorability of the secure passwords, the graphical password will be more memorable, easier to use and therefore will be more secure [39]. In this paper we report on evaluation of a picture-based password system–PixelPin,1 and some findings The rest of the paper is divided as follows. In the next section, we briefly introduce the related research and description of PixelPin. Section 3 reports on our experiment, followed by discussion of results in Sect. 4. In Sect. 5, we summarize our findings and future work.

2 Related Work Graphical passwords were originally described by Blonder [4] in 1996. From 1999 onward, a great amount of graphical password schemes have been proposed. The

1

http://www.pixelpin.co.uk

The Factors Affect User Behaviour

33

early surveys in [23, 35] give the overviews of graphical password systems before 2005; the recent studies in [14] summarize and categorize 12 schemes and review the usability and security issues of those systems and Renaud [29] provides the usability guidelines for designing those systems. Biddle et al. [2] provide a more comprehensive review of published research on the graphical passwords during the first twelve years. In a nutshell, graphical password belongs to the knowledge-based authentication. There are three main categories of graphical passwords: (1) recognition-based (e.g., PassImages [13], Passface [27, 28]), (2) pure-recall (e.g., Draw-A-Secret (DAS) [18], Passdoodles [38]), and (3) cued-recall (e.g., PassPoints [40], Cued Click-Points (CCP) [7]). Some of them have already been commercialized. In Hafiz et al.’s [14] survey study, both usability and security features are recognized and reviewed. According to ISO 9241-11 standard, usability is defined as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [16]. The previous studies on the graphical passwords also consider the memorability issue [11], they suggest that the created password must be meaningful, frequently used and fun [5, 36]. The PixelPin As the PassPoints, PixelPin is also based on Blonder-style graphical password [4] of using multiple click-points (no limitation of the number of points) on a single image as the passwords. Owned by OEM partnership Ltd., PixelPin is a web-based authentication system. The basic functionalities are described as follows: Creation—users can use any images (e.g., the thumb nail images provided by PixelPin system, or personal images), then choose any places they prefer to click on, the sequenced points are supposed to be their passwords. Login—after users inputting their account name (registered email address), system displays users’ own images; users enter the same sequence of the points on their images. If correct (both places and order of points), system directly forward to the successfully-login page. Change passwords—there are two ways to change passwords: changing the picture (using another image and choosing several points on it), and changing the points on the same picture (choosing another points). Reset passwords—users click the ‘forgotten your passpoint?’ link, and input their email address. The reset-passwords link is sent to their typed email address, which would guide users to their previous picture to click on the new points as their new passwords. The aim of our evaluation is to find the issues of both usability and security aspects. We also try to understand user’s choice of image and their preference for using different cursors.

34

Z. Min et al.

Table 1 Nielsen’s five-point severity rating Rating description 0 1 2 3 4

I don’t agree that this is a usability problem at all. Cosmetic problem only—need not be fixed unless extra time is available on the project. Minor usability problem—fixing this should be given low priority Major usability problem—important to fix, should be given high priority Usability catastrophe–imperative to fix this before product can be released

3 Methodology 3.1 Experiment Design and Procedure Experiment 1—Web-based user study The task lists (including five tasks— creating an account, logging in, changing passwords by changing picture, changing passwords by changing points on the same picture, and resetting passwords) and questionnaires were sent to twelve subjects via Email. The questionnaires were used to collect demographic information, computer skill, Internet browser, task performance, 5-point likert for both difficulty-level of individual tasks and overall satisfaction of PixelPin, the number of their pass-points. One week later, the subjects were asked to log in their own accounts for the recall test and give the feedback about how many attempts they try before logging in successfully. Experiment 2—Focus group. Four human factor experts participated in the focus group session which lasted for two hours. Their interactions with PixelPin were video-recorded individually. Experts were asked to discussion and rate all possible problems we found in the Experiment 1 according to 5-point severity rating (shown by Table 1) from Nielsen’s heuristic evaluation [24]. Experiment 3—Paper-based user understanding. Sixteen participants were involved in this experiment. One sheet of paper (as Fig. 1 shows) with sixteen images printed on was shown to participant individually. They were instructed how to use the picture passwords. Then we ask them to choose one picture they prefer to use as password-picture and to mark the points on it (the number of points were not limited) as their ‘passwords’. We had subjects return after a week for a recall test. Experiment 4—Cursor-choice study. Eight participants took part in our study voluntarily. They were asked to try seven different kinds of cursors (refer to Fig. 6) individually on the laptop when clicking the points on the picture, other than the original ‘Finger’ used by the PixePin. After that they chose only one cursor they would like to use most. To ensure that the reproduction of your illustrations is of a reasonable quality, we advise against the use of shading. The contrast should be as pronounced as possible.

The Factors Affect User Behaviour

35

Fig. 1 Paper-based image for understanding user

If screenshots are necessary, please make sure that you are happy with the print quality before you send the files.

3.2 Participants There are totally 20 subjects (21–45 years old) who participates at least one of our experiments. Eighteen of them are students or stuff (13 subjects with computer science background) in University of Nottingham. None of all participants have used the picture-password before.

3.3 Materials Digital task list sheet, questionnaires were used in Experiment 1; Experiment 2 used video-camera, four problems sheets for severity rating, four laptops for individual interaction with PixelPin; one-page of pictures sheet was required in Experiment 3; a Samsung Q330 laptop was used in Experiment 4.

4 Result and Discussion Experiment 1 Ten questionnaires are received. The performance of each task is shown by Fig. 2. We found only 30% subjects could log in successfully at their first attempt.

36

Z. Min et al.

Fig. 2 Task performance

Fig. 3 Subjects’ perception of tasks, following instructions, and overall satisfaction of PixelPin (1 = very easy/good, 5 = very difficult/bad)

On average, people have to try 2.4 attempts before they successfully login. Resetting the password is also suffered the low success rate (60%). The number of participants’ pass-points is 3–4 points mostly, one people even choose only one point. The average number of the points is 3.2 points. According to subject’s 5-point likert on the difficulty-level of tasks and their own satisfaction of PixelPin and following its instruction (as shown by Fig. 3), 50% of subjects find it is difficult to log in, 40% find difficulty in following the PixelPin’s instruction. 10% find it is very difficult to reset the passwords (the reset link does not work). Figure 3 also indicate that most of people’s satisfaction of the PixelPin is on the moderate level (50%), using the picture as the password seems to be more

The Factors Affect User Behaviour

37

Fig. 4 Result of recall test

interesting than the text password, however, the user experience with PixelPin is impaired by the flashing-refresh screen after each clicking, annoying error tolerance, etc. more details will be presented in the next part. Figure 4 displays the memorability of PixelPin passwords once creating and a week later. Only 10% of subjects can log in successfully at the first attempt, 30% can log in after several times; most subjects could not remember their points at all, they have to reset the passwords to enter the system. This might be due to their infrequent use of the PixelPin, and a lack of learning their points after creating. Experiment 2 In the focus group discussion, we focus on both usability and security issues. a. Usability Usability issues often significantly impact the real-world security. We list the PixelPin’s usability issues which are against to Nielsen’s ten usability heuristics [] as follows: #1. Visibility of system status Lack of feedback to keep users informed about what is going on. e.g., • users do not know if their point have been clicked or not; • no message when user inputting wrong email address; • no tips after user clicking wrong sequence of points or incorrect place of points. #3. User control and freedom • ‘‘I don’t like textual passwords, but at least I feel in control with that. But I don’t understand this.’’ • When errors happen, users have to start from the scratch (input the email address again); • Users could not change picture after they choosing one picture during creation phase.

38

Z. Min et al.

#4. Consistency and standards • Input the email address firstly and click ‘Sign up’ or ‘Log in’; while this doesn’t work for the resetting password — users have to click the ‘forgotten the Passpoint’ link firstly, then input the email address. #5. Error prevention Users do not know how exact they should click on the points; #8. Aesthetic and minimalist design The ‘Email’ tip is still in the textbox when textbox is activated; The instruction is not clear, e.g., ‘Sign up’ and ‘Log in’ are too closed to be distinguished easily; The picture is not displayed adaptively to the full screen, users have to drag up and down for looking the whole picture; After clicking each point, the picture flashes to the original position, users have to scroll it down again. #9. Help users recognise, diagnose and recover from errors Users do not know when and what errors happened; The only way recovering from errors is re-doing it from the scratch. #10. Help and documentation No help document or video tutorial available. b. Security We find the potential security issues in phases of login, changing password, and resetting passwords. For example, during the resetting passwords session, if user open the resetting link, there is no double-check about ‘who you are’, rather displaying user’s current picture and is allowed to click points as new passwords. Shoulder surfing is classical problem for the graphical passwords. Some other issues, like using PixelPin on the small-screen mobile device, are still open questions. According to the interview of focus group, sharing passwords is not a desired feature for the expected user authentication. There are three kinds of reasons for recall errors: • Password-length error; • Position error: users are confused by the positions of points, or out of the region tolerance. • Order error: users always mix up the clicked points order. Experiment 3 The purpose of this experiment is to understand users’ choice of picture and how can they remember their points, that is, learnability and memorability of different images.

The Factors Affect User Behaviour

39

In terms of the choice of image, earlier studies [36] suggested that image should have semantically meaningful content for high memorability, and the coherent images [3] will be better for visual recognition therefore increase the speed of image perception. From our experiment, we find some initial indications of users’ strategies as follows: • Users prefer to select simple pictures or pictures containing subjects easy to recognize (like people or corner, etc.) • Users click pass-points based on their preference to specific points in the picture, and those preference will be influenced by how much they are naturally attracted to those points. • Different people have their own different preference points or attentions on the same picture, and if applicable, they will use the same pattern to choose points. For example, people who choose eyes and nose on one picture will intend to use the same strategy when using another picture, no matter the gist of picture is human being or an animal; someone prefer to select the corner, peak or crossing points. Nobody in our experiment likes to select the blue sky without cloud as their password. • Users might select points which are most familiar to them, especially when using the private pictures, they might choose the faces of old friends. Or they select pictures related to their existing memory for other thing. • People remember the picture and the sequence points based on their previous knowledge and interpretation of that picture. We also find that the average length of their paper-based picture password is longer than PixelPin’s, with 5.4 points on average, one participant even uses 12 points as passwords. The recall test results (92% correct) is compared with the digital ones (PixelPin’s passwords), shown by Fig. 5. Experiment 4 As shown by Fig. 6, the cross cursor is most popular one than the rest ones. The reason for users do not like to use the original ‘finger’ lies in that people might be puzzled by which part of ‘finger’ is activate area (Is it the point clicked by the fingertip or wrist?). Surprisingly, one subject likes the normal arrow cursor.

5 Conclusion As we mentioned before, the main issues of Blonder-style graphical passwords (or click-based passwords) are shoulder surfing and its use on the mobile device (because of the difficulty of typing on mobile devices). Our experiment also reaffirms the ‘hot-spot effect’ which was firstly proposed in the Dirik et al.’s research upon the PassPoints [10]. The proposed attacks have been proved in the lab study, like ‘Human-seeded’ attack [37] and purely automated attack [31].

40

Z. Min et al.

Fig. 5 Users’ preference to use

Fig. 6 Users’ preference to different cursors

These issues are receiving increased attention. Chiasson et al. have devised the Cued Click-Points [7] and Persuasive Cued Click-Points [8] to deal with the hot spots effect. Some studies on the resisting the shoulder surfing [21, 41] consider the recognition-based graphical passwords, but not for the cued recall-based graphical passwords scheme. By our knowledge so far, all these issues are still await to be solve. So, we should put our effort on dealing with those issues in our future work. It might be more work on user interface (UI) design (because UI may unintentionally sway user behavior towards less secure) or adopt new technology to change user’s behavior, like persuasive techniques [12], etc.

The Factors Affect User Behaviour

41

Acknowledgments I would like to thank Brendan Ryan and Sarah Atkinson, who supervised this project, for their supervisions and experiment supporting. I would like to thank Brian Taylor from OEM partnership Ltd for his useful feedback. I would like to thank all anonymous reviewers for their valuable comments and suggestions.

References 1. Adams A, Sasse MA (1999) Users are not the enemy. Commun ACM 42(12):41–46 2. Biddle R, Chiasson S, van Oorschot PC (2011) Graphical passwords: learning from the first twelve years. AMC Computing Surveys 3. Biederman I, Glass AL, Stacy EW (1973) Searching for objects in real world scenes. J Exp Psychol 97: pp 22–27 4. Blonder GE (1996) Graphical passwords. United States patent 5559961 5. Brostoff S, Sasse MA (2000) Are passfacesTM more usable than passwords? A field trial investigation. In: Proceedings of human computer interaction, pp 405–424 6. Brown AS, Bracken E, Zoccoli S, Douglas K (2004) Generating and remembering passwords. Appl Cognitive Psychol 18:641–651 7. Chiasson S, van Oorschot PC, Biddle R (2007) Graphical password authentication using Cued Click Points. Eur Symp On Res In Comput Secu (ESORICS), LNCS 4734:359–374 8. Chiasson S, Forget A, Biddle R, van Oorschot PC (2008) Influencing users towards better passwords: Persuasive cued click-points. In Human computer interaction (HCI), British Computer Society 9. Dhamija R, Perrig A (2000) Déjà vu: a user study using images for authentication. In: Proceedings of the 9th USENIX security symposium 10. Dirik A, Menon N, Birget J (2007) Modeling user choice in the passpoints graphical password scheme. In: 3rd ACM symposium on usable privacy and security (SOUPS) 11. Eljetlawi AM, Bt.Ithnin N (2009) Graphical password: usable graphical password prototype. J Intl Commer Law Technol 4, p 298 12. Fogg BJ (2003) Persuasive technologies: Using computers to change what we think and do. Kaufmann Publishers, San Francisco 13. Furnell SM, Zekri L (2005) Replacing passwords: in search of the secret remedy. Netw Secur 2006(1):4–8 14. Hafiz MD, Abdullah AH, Ithnin N, Mammi HK (2008) Towards identifying usability and security features of graphical password in knowledge based authentication technique. In: Second Asia international conference on modelling & simulation, pp 396–403. IEEE 15. Ives B, Walsh KR, Schneider H (2004) The domino effect of password reuse. Commun ACM 47(4):76–78 16. ISO-International organization for standardization. http://www.iso.org. Accessed Jan 2011 17. Jain AK, Ross A, Prabhakar S (2004) An introduction to biometric recognition. IEEE Trans. Circuits Syst. Video Technol, Special Issue Image- and Video-Based Biomet 14(1):4–20 18. Jermyn I, Mayer A, Monrose F, Reiter M, Rubin A (1999) The design and analysis of graphical passwords. In: Proceedings of the 8th USENX security symposium, Washington 19. Klein D (1990) A survey of, and improvement to, password security. In:UNIX security workshop II 20. Li SZ, Jain AK (eds) (2004) Handbook of face recognition. Springer, New York 21. Man S, Hong D, Matthews M (2003) A shoulder-surfing resistant graphical password scheme–WIW. In: Proceeding of international conference on security and management, Las Vegas, pp 105–111 22. Markowitz JA (2000) Voice biometrics. Commun ACM 43(9), pp 66–73

42

Z. Min et al.

23. Monrose F, Reiter M (2005) Graphical passwords. In: Cranor L, Garfinkel S (eds) Security and Usability: Designing secure systems that people can use, Chap. 9, O’Reilly Media, Sebastopol, pp 157–174 24. Nielsen J (1994) Heuristic evaluation. In: Nielsen J, Mack RL (eds) Usability inspection methods. Wiley, New York 25. Pierce JD, Warren MJ, Mackay DR, Wells JG (2004) Graphical Authentication: justifications and objectives. In: Proceedings of the 2nd information security management conference, Fremantle 26. Paivio A, Rogers TB, Smythe PC (1976) Why are pictures easier to recall than words? Psychonomic Sci 11(4):137–138 27. Passfaces Corporation. The science behind Passfaces. White paper. http:// www.passfaces.com/enterprise/resources/white_papers.htm. Accessed 23rd Jan 2011 28. Real User Corporation, Passfaces TM, http//:www.realuser.com, Accessed on Dec 2010 29. Renaud K (2009) Guidelines for designing graphical authentication mechanism interfaces. Int J Info Comput Secu 3(1):60–85 30. Roizenblatt R, Schor P et al (2004) Iris recognition as a biometric method after cataract surgery. Biomed. Eng. Online 3(2) 31. Salehi-Abari A, Thorpe J, van Oorschot P (2008) On purely automated attacks and clickbased graphical passwords. In: Annual computer security applications conference (ACSAC). 32. Sasse MA, Brostoff S, Weirich D (2001) Transforming the ‘weakest link’ –a humancomputer interaction approach to usable and effective security. BT Technical J 19:122–131 33. Shepard RN (1967) Recognition memory for words, sentences, and pictures. J Verbal Learn Verbal Behav 6:156–163 34. Sobrado L, Birget J (2002) Graphical passwords, the rutgers scholar, an electronic bulletin of undergraduate research, Rutgers University, Camden New Jersey, Vol 4 35. Suo X, Zhu Y, Owen GS (2005) Graphical passwords: a survey In: annual computer security applications conference (ACSAC) 36. Takada T, Koike H (2003) Awase-E: Image-based authentication for mobile phones using user’s favorite images. In: Human-computer interaction with mobile devices and services, vol 2795 Springer-Verlag GmbH, pp 347–351 37. Thorpe J, van Oorschot PC (2007) Human-seeded attacks and exploiting hot-spots in graphical passwords. In: 16th USENIX security symposium 38. Varenhorst C (2004) Passdoodles: A lightweight authentication method, Research Science Institute, Cambridge 39. Wiedenbeck S, Waters J, Birget JC, Brodskiy A, Memon N (2005) Authentication using graphical passwords: effects of tolerance and image choice, In: Proceedings of symposium usable privacy and security, ACM Press, pp 1–12 40. Wiedenbeck S, Waters J, Birget JC, Brodskiy A, Memon N (2005) PassPoints: design and longitudinal evaluation of a graphical password system. Int J Hum Comput Stud 63:102–127 41. Wiedenbeck S, Waters J, Sobrado L, Birget JC (2006) Design and evaluation of a shouldersurfing resistant graphical password scheme, In: Proceedings of advanced visual interfaces (AVI2006), Venice, Italy, 23–26 42. Wixted JT (2004) The psychology and neuroscience of forgetting. Annu Rev Psychol 55:235–269

Analysis of Smartphone-Based Location Information Dohyun Kim, Jewan Bang and Sangjin Lee

Abstract Location information is an important factor for figuring out user’s behavior in digital forensic investigations. Smartphone-based map applications keep record user’s data about location information such as timestamp, GPS information for searched locations. In this paper, we will analyze data generated by famous map applications–Google map, Daum map, Naver map and extract necessary information to assist digital forensic investigations. Keywords Digital forensics application




Location information




Smartphone




Map

1 Introduction Recently, Smartphone users steadily keep on growing and its rate is expected to be increased by almost 30% in 2011 in worldwide mobile phone market. Furthermore, its shipments will soon overtake those of PCs shipped (both desktops and notebooks) in 2012. These rapid growth comes from various smartphone applications such as games or useful applications in daily life.

D. Kim (&)
J. Bang
S. Lee Center for Information Security Technologies, Korea University, Seoul, South Korea e-mail: [email protected] J. Bang e-mail: [email protected] S. Lee e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_5,  Springer Science+Business Media B.V. 2012

43

44

D. Kim et al.

Table 1 Application-related data storage path of genaeral, galaxy S smartphone Default application User application Additional file storage path General Android Phone Galaxy S

/data/data/package name

/data/data/package name /dbdata/databases/package /data/data/package name name

/mnt/sdcard/package name /mnt/sdcard/package name

In numerous smartphone-based map applications are most commonly used among users, and it offers users location search, route search, transportation search, destination viewing, bookmark. Whenever a user access this application, user-searched information (starting point, destination, latitude, longitude and so on) and timestamp are automatically recorded in particular files in application. In a view of digital forensic investigations, this information is key evidences to figure out a user’s behavior and intention as well as to find a critical clue to a case. Thus, it is necessary for us to research this famous map applications connected to digital forensic investigations. In this paper, it will be discussed how to extract inside information recorded in particular files through famous map application—Google map, Daum map and Naver map and how to analyze internal data, related to digital forensic investigations. And we suggested digital forensic tool (MapAn) utilizing smartphonebased map application for the investigation. This paper consists of five parts, not including introduction–related work, the analysis of smartphone-based map applications, tool for analyzing smartphonebased map application, conclusion and future work.

2 Related Work In this chapter, it will be discussed about a technique to extract files which contain history using an application. In case of Android phone, these data are mainly recorded to SQLite Database files while iPhone recorded to SQLite Database files or plist files according to an application.

2.1 The File Path in Android Phone Android phone has a different file path to store applications related to Galaxy S and other Android smart phones. In case of general Android phone, default application—SMS/MMS, address book, e-mail and call history as well as user application are installed in the same file path. On the contrary, Galaxy S installs both default and user application in different file path. However, after installation, a file path for both Android phone and Galaxy S become same on account of file download function. These paths presented in Table 1.

Analysis of Smartphone-Based Location Information

45

Table 2 Map application’s function and description Function Description Location Search Route Search Transportation Search View (Street & Air) Bookmark

Finding destination Looking for the way from current location to destination location Finding public transportation to destination Providing view on the street and air Storing searched result

Rooting is necessary for extracting files from a path. General rooting is performed, integrity may get damaged, whereas Temporary rooting does not affect any damage to an internal data. Therefore, by performing Temporary rooting, we use ADB (Android Debugger Bridge), type ‘‘adb pull [The path for extracting a file from Android Phone] [The path for storing an extracted file]’’ instruction, and extract the files asked for [1].

2.2 The File Path in iPhone In case of iPhone, all internal data are encoded; selecting files for extraction is not available. For extracting files without damaging integrity, we use not Jail Breaking but iTunes’s Backup function, the software which is able to sync with iPhone, so as to get all Backup files in iPhone. The path for creating Backup files is ‘‘%APPDATA%/Roaming/Apple Computer/MobileSync/Backup/[Device ID]’’. The backup files substitute for the hash value toward the name of the file and are generated in the path. In the process of extracting a file from the path, the real names and path of files need to be figure out among the files named as hash value. With open tool named ‘‘mbdbump.exe’’, not only the files real name, but also file’s path to extract [2].

3 Analysis of Smartphone-Based Map Applications In this chapter, we are going to discuss about the famous smartphone-based map applications and those functions and then analyze files that recorded users history according to an application.

3.1 The Type and Functional Analysis of Map Applications Google map, Daum map, and Naver map are mainly used as the famous map applications. These map applications offer useful functions such as location search, route search, transportation search, view support and bookmark. The description for each function is presented in Tables 2 and 3.

46

D. Kim et al.

Table 3 Offered functions by each map application Location Route Transportation Search Search Search Google map Daum map Naver map

View (Street & Air)

Bookmark

O

O

O

X

X

O

O

O

O

O

O

O

O

O

O

Table 4 Target file of Google Map in Android Phone File Name Search_history.db Path General Android Phone /data/data/com.google.android.apps.maps Galaxy S /dbdata/com.google.android.apps.maps File Type SQLite Database

3.2 Analysis of User History about Map Applications To analyze the user history of using map applications, the path to files that recorded user history data and its format should be identified. Here are the steps for analyzing the path and the format of files. First of all, due to the fact that third Party applications record data into files in its own format, files should be analyzed individually according to the applications. Secondly, meaningful data selection and extraction from a lot of data should be conducted to help digital forensic investigations. In map applications, the file name, path and format that recorded in the user data are given in Table 4, 5, 6, 7, 8, and 9. Google Map Android Phone. In Android phone, search ‘‘history.db’’ file record the user history data of Google Map. In viewpoint of digital forensic, meaningful data is data 1 field in suggestions table. When location search is performed, the contents are recorded in order, but the timestamp is not recorded in the field. More recent data is located at the bottom of contents. The meaning of each field is given in Table 10. iPhone. Unlike Android phone, files recorded user history in iPhone of Google Maps are ‘‘b60c382887dfa562166f099f24797e55c12a94e4 (History.plist)’’ and ‘‘a30335a2c 0f0316c9610d868a527b2ade1911542 (com.apple.map.plist)’’. Both files are formed as plist file. In a view of digital forensic, meaningful data from ‘‘b60c382887dfa56216 6f099f24797e55c12a94e4 (History.plist)’’ file are keyword for location search, latitude and longitude. This file does not contain time information, located at the top of the more recent data is the data retrieved. ‘‘a30335a2c0f0316c9610d868a527b2ade 1911542 (com.apple.map.plist)’’ file located in the most recently conducted searches and searches where the location information(latitude, longitude) and time to have the information [4]. The meaning of each element is in Table 11.

Analysis of Smartphone-Based Location Information

47

Table 5 Target file of Google Map in iPhone File Name b60c382887dfa562166f099f24797e55c12a94e4(History.plist) Path %APPDATA%/Roaming/Apple Computer/MobileSync/Backup/[Device ID] File Type plist File Name a30335a2c0f0316c9610d868a527b2ade1911542(com.apple.map.plist) Path %APPDATA%/Roaming/Apple Computer/MobileSync/Backup/[Device ID] File Type plist Table 6 Target file of Daum Map in Android Phone File Name history.db Path /data/data/net.daum.android.map/map/data File Type SQLite database File Name favorite.db Path /data/data/net.daum.android.map/map/data/[Daum Account ID] File Type SQLite database Table 7 Target file of Daum Map in iPhone File Name ae6522d1ef6dd52694d53cc015c04749603ac95a(history.db) Path %APPDATA%/Roaming/Apple Computer/MobileSync/Backup/[Device ID] File Type SQLite database File Name 59ea2d577b192bcce21149ce2a8385180d72313e(favorite.db) Path %APPDATA%/Roaming/Apple Computer/MobileSync/Backup/[Device ID] File Type SQLite database Table 8 Target file of Daum Map in iPhone File Name mapHistory.db Path /data/data/com.nhn.android.nmap/databases File Type SQLite database Table 9 Target file of Daum Map in iPhone File Name 6be48053cd29804a3a30e78846c30d828c75eb8a(History.db) Path %APPDATA%/Roaming/Apple Computer/MobileSync/Backup/[Device ID] File Type SQLite database File Name d5c541e19cdfec4cdbb833719ced812564dd43ba(NMap.db) Path %APPDATA%/Roaming/Apple Computer/MobileSync/Backup/[Device ID] File Type SQLite database

Daum Map Android Phone. History files of Daum Map in Android phone are shown as ‘‘history.db’’ and ‘‘favorite.db’’. In ‘‘history.db’’ file, there are ‘‘route_history’’ table which contains ‘‘startPoint, endPoint, hitcount, and updatetime’’ fields as well as ‘‘word_history’’ table which takes ‘‘key, address, hitcount, updatetime’’ fields. On the other hand, in ‘‘favorite.db’’ file, ‘‘name, and attime’’ fields exists. Both files contain very much important data in digital forensic investigations and the meaning of each field is displayed in Table 12.

48 Table 10 Structure of target file for Google Map in Android Phone File Name seach_history.db Table Name Suggestions (Location Search) Field Name _id data1

D. Kim et al.

Contents Number Keyword

Table 11 Structure of target file for Google Map in iPhone File Name b60c382887dfa562166f099f24797e55c12a94e4(History.plist) Contents • Result of Location Search • Location of Performed Location Search (Latitude, Longitude) File Name a30335a2c0f0316c9610d868a527b2ade1911542(com.apple.map.plist) Contents • Last Time of Performed Location Search • Last Location of Performed Location Search (Latitude, Longitude) • Contents of Last Performed Location Search

iPhone. In iPhone, history of Daum map is stored as ‘‘ae6522d1ef6dd52694d53cc015 c04749603ac95a (history.db)’’ and ‘‘59ea2d577b192bcce21149ce2a8385180d72313e (favorite. db)’’ and internal structure is as same as ‘‘history.db and favorite.db’’ in Android phone. Naver Map Android Phone. It is ‘‘mapHistory.db’’ file that is stored as history of Naver map in Android phone. Key data as evidence in digital forensics are inside of ‘‘Bookmark’’ table as ‘‘title, time, tel, and addr’’ fields. ‘‘uid, and time’’ fields in ‘‘RecentPub’’ table, ‘‘start, end, and time’’ fields in ‘‘RecentRoute’’ table as well as ‘‘uid, and time’’ fields in ‘‘RecentWord’’ table are meaningful data for digital forensic investigations. (See below Table 13 for each meaning in field.) iPhone. In iPhone, ‘‘6be48053cd29804a3a30e78846c30d828c75eb8a (History.db)’’ and ‘‘d5c5 41e19cdfec4cdbb833719ced812564dd43ba (NMap.db)’’ are history file of Naver map. In ‘‘6be48053cd29804a3 a30e78846c30d828c75eb8a (History.db)’’ file, not only ‘‘uid, and time’’ fields in ‘‘LocHistory’’ table, ‘‘uid, and time’’ fields in ‘‘PubHistory’’ table, but also ‘‘start, destination, and time’’ fields ‘‘RouteHistory’’ table is essential key in digital forensic perspective. And, in ‘‘d5c541e19cdfec4cdbb833719 ced812564dd43ba (NMap.db)’’ file, in viewpoint of digital forensics, we put emphasis on ‘‘title, time, tel, and addr’’ fields in ‘‘Bookmark’’ table, ‘‘uid, and title’’ fields in ‘‘OfflineMapList’’ table and ‘‘uid, start, and destination’’ fields in ‘‘OfflineRouteList’’. The meaning of each field is shown in Table 14.

4 Tool for Analyzing Smartphone-Based Map Application When using map application, a user puts the destination where they are looking for into the search bar. Then, he searches for the route, transportation from the present location to the destination and follows the direction to reach the place. When

Analysis of Smartphone-Based Location Information

49

Table 12 Structure of target file for Daum Map in Android Phone File Name seach_history.db Table route_history (Route Field Name Contents Name Search) idx Number startPoint Starting Point for Route Search startX Latitude of Starting Point for Route Search startY Longitude of Starting Point for Route Search endpoint Destination for Route Search endX Latitude of Destination for Route Search endY Longitude of Destination for Route Search hitcount Count of performed Search updatetime Time of performed Search (Unix Time) word_history (Location Field Name Contents Search) idx Number key Keyword for Location Search chosungKey Initial Consonant of Keyword address Address of Location for Location Search posX Latitude of Location for Location Search posY Longitude of Location for Location Search hitcount Count of performing search updatetime Time of performed Search (Unix Time) File Name seach_history.db Table favorite (Bookmark) Field Name Contents Name id Number name Contents of Bookmark type Location: 100, Route: 200 cords Lattitude and Longitude, registered in Bookmark attime Time to Register Bookmark mtime Time of Modified Bookmark route_start_name Starting Point for Route Search route_end_name Destination for Route Search

coming closer to the destination, a user tries to search for the destination again so that they could get more precise information from where they are to where they want to be. In this process, all information is recorded inside of application files whenever he accesses the application. Furthermore, the place where a user was looking for might have been visited. Thus, this recorded information is very much useful in digital forensic investigations.

50

D. Kim et al.

Table 13 Structure of target file for Naver Map in Android Phone File Name mapHistory.db Table Bookmark (Bookmark) Field Contents Name Name title Location of Registered Bookmark x Latitude of Location Registered Bookmark y Longitude of Location Registered Bookmark time Time of Registered Bookmark (Unix Time) tel Telephone Number of Location addr Address of Location of Registered Bookmark Bookmark (Transportation Field Contents Search) Name uid Contents for Transportation Search time Time of Transportation Search (Unix Time) File Name favorite.db Table RecentRoute (Route Search) Field Contents Name Name start Starting Point for Route Search startx Latitude of Starting Point for Route Search starty Longitude of Starting Point for Route Search end Destination for Route Search endx Latitude of Destination for Route Search endy Longitude of Destination for Route Search time Time of performed Search (Unix Time) RecentWord (Location Field Contents Search) Name uid Contents of Location Search time Time of Performed Search (Unix Time)

If a criminal intends to commit a crime, normally he sets the place to do and then visits the scene in advance. Instead of going to the place by himself, he is able to check the place through map application. Not only looking around the place by using street view or air view through location search, but also he can take a picture from many angles, capturing every inch of the scene. Those pictures can be sent to e-mail or sent as MMS message. However, the history data from street and air view function are not recorded individually. Accordingly, making a thorough investigation into photo book, e-mail, MMS records is needed in digital forensic investigations. History data gives us the information of the user’s behavior and his route by the hour and is going to be much helpful factor in digital forensic investigations.

File Name Table Name

File Name Table Name

OfflineRouteList (SaveRoute Picture)

OfflineMapList (SaveLocation Picture)

Bookmark (Bookmark)

RouteHistory (Route Search)

PubHistory (Transportation Search)

LocHistory (Location Search)

Table 14 Structure of target file for Naver Map in iPhone 6be48053cd29804a3a30e78846c30d828c75eb8a (History.db) Field Name Contents uid Contents of Location Search time Time of performed Search (Unix Time) Field Name Contents uid Contents of Location Search time Time of Transportation Search (Unix Time) Field Name Contents start Starting Point for Route Search startx Latitude of Starting Point for Route Search starty Longitude of Starting Point for Route Search destination Destination for Route Search endx Latitude of Destination for Route Search endy Longitude of Destination for Route Search time Time of performed Search (Unix Time) d5c541e19cdfec4cdbb833719ced812564dd43ba (NMap.db) Field Name Contents title Location of Registered Bookmark x Latitude of Location Registered Bookmark y Longitude of Location Registered Bookmark time Time of Registered Bookmark (Unix Time) tel Telephone Number of Location addr Address of Location of Registered Bookmark Field Name Contents uid Time of Saved Location Picture (Unix Time) title Location of Saved Picture x Latitude of the place you saved the picture y Longitude of the place you saved the picture radius The radius of where you saved the picture Field Name Contents uid Time of Saved Route Picture (Unix Time) start Starting Point sposx Latitude of Starting Point sposy Longitude of Stating Point destination Destination destposx Latitude of Destination destposy Longitude of Destination xml Driving directorition from Starting Point to Destination

Analysis of Smartphone-Based Location Information 51

52

D. Kim et al.

Fig. 1 Contents of ‘‘MapAn.db’’

Table 15 Structure of ‘‘MapAN.db’’ Field Name Contents time type_id action_id app_id content detail hitcount contact_id deleted_flag

Time (Google Maps does not provide) Type of Application Function of Application Application Name Searched Result (Starting Point for Route Search) Searched Result2 (Destination for Route Search) The Number of Searches Performed (Only provide Daum map) Phone Number for Search Results (Only provide Naver map) Display Deleted Records

Despite of these helpful evidences, in investigation process, too much time is consuming in data opening, analyzing, and selecting for needs. Since all different file formats and paths are generated in Google map, Daum map, and N aver map, data in Android phone do not match with those in iPhone. Thus, we need analysis tool for selecting essential data from extracted file and integrating them into singular format. ‘‘MapAn’’ application is the useful tool for digital forensics by extracting valuable data generated from the files that contain the user data at various angles. Once a user input files that contain user history data, a single database file which called ‘‘MapAn.db’’ comes out and there are information about connect and using hour, the name of application, the searched contents and so forth. By analyzing ‘‘MapAn.db’’ file, we can figure out which map-based application a user use, when he access to the application, what he does with application and where he goes. Coming-out data is appeared in Fig. 1, and Table 15.

5 Conclusion and Future Work Smartphone-based map application such as Google map, Daum map, and Naver map leaves various information in smartphone. Such information like connect hour or searched information is much valuable in a criminal investigation because it

Analysis of Smartphone-Based Location Information

53

becomes such a good evidence to find out a criminal. Even though these application is preferred to be analyzed, the numerous applications are out there and have great many approaches to extract data from user’s files depending on various smartphones. In this paper, we suggest ‘‘MapAn’’ that helps us to analyze a file produced from smartphone-based mapping application in a view of digital forensic investigations. Besides ‘‘MapAn’’ application, more effective analysis would be possible if other applications like ‘‘Call history and SMS/MMS’’ can be integrated as well as recovered data changes the shape, combined with a real image map. These application becomes much useful tools for accurate analysis in the future and additional researches should be carried out for future study.

References 1. Lessard J, Kessler GC (2010) Android forensics: simplifying cell phone examinations. Small Scale Disgital Forensics J 4(1):1–12 Sep 2. iPhone Backup Browser, http://code.google.com/p/iphonebackupbrowser/wiki/MbdbMbdx Format 3. Husain MI et al (2011) A simple cost-effective framework for iPhone forensic analysis. Digital Forensics and Cyber Crime. Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 53. pp 27–37 4. Google Map, http://maps.google.co.kr/maps 5. DaumDNA: openApi mapAPI coordinate system, http://dna.daum.net/apis/maps/coordinate

Dynamic Visualization for Sensing Information Based on Mobile Sensor Network Su-Hyun Yang, Eun-Ha Song and Young-Sik Jeong

Abstract In a mobile sensor network, when sensor data are sent from a sensor node to the sink node, traffic may be congested at one sensor node, in which case overhead occurs at that node, and this may possibly affect the entire network. Also, this causes frequent data loss as there is excessive traffic or as the topology constantly changes. In this paper, VSDR (Visualization Sensing Data Routing) is designed and implemented, which visually shows on a GML-based map with GPS coordinates the packet paths between sensor nodes as well as the amount of packet traffic. VSDR also includes an algorithm that dynamically reroutes paths in the case of sensor data overhead or a fault occurrence at one or more sensor nodes.










Keywords Wireless sensor network GML Network simulator Sensor node Packet traffic Packet routing Fault tolerance Dynamic re-routing













1 Introduction One of the most important technologies in the ubiquitous society is the wireless sensor network [1], which consists of mobile sensor nodes that are characterized by their computation ability, communication ability, sensing ability, and mobility. S.-H. Yang
E.-H. Song
Y.-S. Jeong (&) Department of Computer Engineering, Wonkwang University, 344-2 Shinyong-Dong, Iksan, Jeonbuk, Korea e-mail: [email protected] S.-H. Yang e-mail: [email protected] E.-H. Song e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_6, Ó Springer Science+Business Media B.V. 2012

55

56

S.-H. Yang et al.

In a sensor network (that has low power demand), micro sensors are used to recognize a variety of situations or changes in the environment and the sensor data are sent to the users. Currently, a lot of research is being done on wireless sensor networks [2]. The main components of a wireless sensor network are the sensor node, which measures or senses something using a sensor and sends the data over the wireless network; the sink node that collects all data sensors measured and sends them to the middleware; and the relay node that works between the sensor node and the sink node in sending sensor data. Sensor data measured by a sensor node are sent to the sink node over the network, and they are subsequently sent to the middleware or to the server through the sink node. In contrast with other types of networks, the primary objective of wireless sensor networks is not to serve as a means of communication but rather as a means of automatic and remote collection of data. Wireless sensor networks have a wide range of applications in a variety of fields including science, medicine, military, and business [2]. Given a sensor network composed of sensor nodes, sink nodes, and relay nodes according to their appropriate placement and function, when data transmission takes place from a sensor node to a sink node, the data traffic may be concentrated at a single sensor node on the way to the sink node. In this case, due to overloading of the sensor node, a problem could be caused for the entire sensor network. In addition, as excessive traffic occurs or the topology constantly changes, there would be frequent data loss. In this paper, the packet paths between sensor nodes and the amounts of the packet traffic are visually shown on a GML-based map with GPS coordinates. The parent node sends a signal upon receiving data from the child node in order to ensure correct delivery of data and to minimize data loss. Also, if while sending data between sensor nodes the amount of traffic exceeds a certain limit, a parent node is automatically chosen among neighboring nodes and the path is changed accordingly; alternatively, the user can manually select a specific node from neighboring nodes to change the path. Even when a fault occurs at one or more sensor nodes, the path can be either automatically changed or the user can directly change the path. Based on this, a network simulator called VSDR (Visualization Sensing Data Routing) is designed and implemented, which takes into account the amount of the sensor data traffic as well as faults at sensor nodes in a wireless sensor network that is made up of sensor nodes.

2 Related Works WSN simulators are a hardware device or a software program designed to predict and analyze network operations and the effects. They are particularly useful in simulating a situation that has a low likelihood of happening in real life, or when the testing setup would be expensive to implement using real components, such as when the network consists of a large number of sensor nodes.

Dynamic Visualization for Sensing Information

57

Table 1 Comparison of characteristics between existing WSN simulators and VSDR Simulator Features and characteristics NS-2

This is an open-source simulator developed for research purposes. It uses the discrete event simulation approach and a wide range of network protocols can be simulated. However, it is difficult to use when there are a large number of nodes and for complicated, large systems. There are also many unnecessary interdependencies between modules. Most Unix-based OS platforms are supported, and a script language called TCL is used to construct the simulation environment GloMoSim This simulator was developed to simulate large networks. PARSEC, a C-based language for parallel simulation was used to develop it. Protocols have been modularized according to the OSI layers. Currently only wireless networks are supported. Although network models or transmission scenarios can be verified, coverage cannot be expressed QualNet As the next version of GloMoSim, this was designed to simulate large wireless networks. It follows the IEEE 802.11 MAC and Physical Layer specifications. Although it supports sensor networks, it has limitations in expressing coverage in that only the RF analysis is supported TOSSIM This software emulates TinyOS to analyze application operations and hardware effects. Measuring the rate of packet loss and CRC detection are the main features. However, scalability is low in that only the Mica series of CrossBow can be used with it, and there are difficulties in adding sensors whose type is not LED NetSim This is a simulation software designed for studying how to set up a network. The simulation takes place in real-time as to allow interaction with the user. Not only can transport protocols, MAC protocols, and routing protocols be simulated, but devices such as switches, routers, and APs can be simulated as well J-Sim This is a Java-based open source wireless sensor network simulator. Execution time, the number of events, and the memory usage according to the size of the network can be measured. An event sent from the target node getting sent to the sink node in the form of packets can be simulated. Coverage cannot be expressed SWNANS SWANS, a wireless network simulator, has been placed on top of Jist, a discrete event simulator engine based on PARSEC. It is open source and unlike NS-2 or GloMoSim, can simulate large networks. However, like other simulators, only protocol verification is possible VSDR This is a Java-based wireless sensor network simulator. Sensor nodes can be placed on a GML-based map taking into account connectivity of sensor nodes and locations of obstacles. Paths are set by calculating the number of hops to the sink node. If during data transmission a fault occurs in the traffic between sensor nodes or at sensor nodes the path gets changed in an efficient way

Some of the simulators available for sensor networks include the following: NS-2 [3], GloMoSim [4], QualNet [5], TOSSIM [6], NetSim [7], J-Sim [8], and SWNANS [9]. These simulators have been developed for a variety of platforms according to their purpose, and were developed using C++ or Java, which supports templates. They include extensive network technologies and use nodes and links as the basic units in constructing a complicated environment or a layered network environment (Table 1).

58

S.-H. Yang et al.

Fig. 1 Overall architecture of VSDR

3 Design of VSDR 3.1 Structure of VSDR VSDR is made up of eight main components: User Interface Component, GML Analyzer Component, Map Layer Manager Component, Map Controller Component, Node Manager Component, Target Area Manager Component, Path Manager Component, and Viewer Component. The overall structure of the system is shown in Fig. 1. The User Interface Component provides a user interface for inputting configuration values needed to run VSDR. It is the component using which the map and nodes are set up. The GML Analyzer Component reads a GML document, analyzes it, and then creates map objects and sends them to the Map Layer Manager. The Map Layer Manager Component manages the map objects sent by the GML Analyzer and sets up obstacles. The Map Controller Component is used to control the map—zoom in, zoom out, zoom area, and move—and then to output the results. The Node Manager Component applies the node settings from the User Interface Component to sensor nodes and creates sensor node packets. It also creates sensor nodes that interact with the target area defined in the Target Area Manager, as well as with the obstacles defined in the Map Layer Manager, and operates the sensor nodes. Statistics collected by sensor nodes are also calculated.

Dynamic Visualization for Sensing Information

59

Fig. 2 Packet structure in VSDR

Table 2 Traffic classifications of VSDR with volume of packets Packet volume 2 Mb; 2–4 Mb 4–6 Mb

6–8 Mb

8–10 Mb

Color used to denote

Orange

Red

Blue

Green

Yellow

The Target Area Manager Component sets up and manages areas that need to be observed in the field set up by the GML document. The Path Manager Component checks whether there has been excessive traffic between sensor nodes and whether there has been a fault in a sensor node and, if so, the path is rerouted. The Viewer Component shows on the screen map objects of the Map Layer Manager as well as mobile sensor nodes of the Node Manager.

3.2 Path Visualization According to Change in Sensor Data Traffic VSDR, sensor nodes make up the network while maintaining their connectivity on a map based on GML [10–12]. All of the sensor nodes take into account the number of hops to the sink node in selecting their parent node among neighboring nodes. Each node creates packets every specific period of time. The structure of the packet used in VSDR is shown in Fig. 2. The name of packets that are created at sensor nodes is defined using the sensor node ID and the packet ID. Sensor nodes send created packets to their parent nodes. Sensor nodes with child nodes puts together the packets received from their child nodes and the packets created by themselves and sends it to their parent node. The traffic is determined according to the number of packets sent to the parent node. Supposing the size of a single packet is 8 bytes, the traffic size classifications according to the packet volume are shown in Table 2. If the packet volume exceeds 10 Mb when packets are sent from sensor nodes to their parent node, it is assumed that overhead has occurred and neighboring sensor nodes are probed to find one that is appropriate as a parent node. The one that has lower number of hops than itself as well as low packet volume is chosen as a new parent node and from that point on packets are sent divided between the current parent node and the new parent node. The parent node sends an ACK signal when it has received a packet from a child node in order to confirm that the transmission has been successful. The child node can check that the packet transmission is going well based on whether it has received the ACK signal or not.

60

S.-H. Yang et al.

Fig. 3 Packet transmission process of VSDR

Fig. 4 Dynamic re-routing of single mode

The visualization of sensor nodes’ packet paths is classified into two types: (1) Single Mode, in which only a single sensor node is selected and the path from it to the sink node is shown; and (2) Multiple Mode, in which multiple sensor nodes are selected and the multiple paths to the sink node are shown (Fig. 3).

3.2.1 Single Mode In Single Mode, only a single sensor node is selected and the path from it to the sink node is shown. The user can see detailed information about the selected node, and the path of the selected node can be automatically changed or the user can manually select a parent node to change the path. Figure 4 shows the path from Sensor Node A to Sink Node S, as well as the process of rerouting when overhead occurs at a sensor node; specifically, when overhead occurs at Sensor Node D, it selects Sensor Node E among the neighboring nodes as its new parent node, which has lower number of hops to Sink Node S.

Dynamic Visualization for Sensing Information

61

Fig. 5 Dynamic re-routing of multiple mode

From then on, Sensor Node D will send some packets to the original parent node and the rest to the new Parent Node E.

3.2.2 Multiple Mode In Multiple Mode, multiple sensor nodes are selected and the multiple paths to the sink node are visualized. Stated in a different way, the user gets to see not just the path involving a single sensor node but rather all the paths involving multiple sensor nodes to the sink node. Like in the Single Mode, the user can select a specific sensor node to either automatically change the path or manually change it by selecting the parent node for the sensor node. Figure 5 shows the path from Sensor Node A to Sink Node S, as well as the path from Sensor Node F to Sink Node S. In Multiple Mode, paths of more than one sensor node can be shown. When overhead occurs at Sensor Node D, it selects among neighboring nodes Sensor Node E as its new parent node, which has lower number of hops to Sink Node S. From then on, Sensor Node D will send some packets to the original parent node and the rest to the new Parent Node E.

3.3 Path Visualization with Fault In an implemented sensor network, a fault may occur at sensor nodes, in which case the entire network may be damaged or there may be loss of or damage to data from the sensor node to the sink node. The parent node sends an ACK signal upon receiving a packet from a child node, in order to confirm that it has received it successfully. If an ACK signal has not arrived within a specified time, it can be assumed that a fault has occurred at the parent node. If a fault occurs among sensor nodes, the child nodes that has the faulted node as their parent selects a new suitable parent node from neighboring sensor nodes. The one with a lower number of hops than themselves and which has low packet volume is selected as the new parent node and the path is rerouted.

62

S.-H. Yang et al.

Fig. 6 Visualization of VSDR with a single fault occurs

Fig. 7 Visualization of VSDR with a multiple fault occurs

There are two types of faults that can occur at a sensor node. In the first type, the fault occurs at a single node (see Fig. 6). In the second type, the fault occurs at multiple sensor nodes (see Fig. 7). Given that there is a path between Sensor Node A to Sensor Node S, if a fault occurs at Sensor Node C, Sensor Node B, which has Sensor Node C as its parent, selects among its neighboring nodes the one with lower number of hops to Sink S as its new parent node, namely Sensor Node E. In Fig. 7, it can be seen that, in addition to Sensor Node B, the path of Sensor Node J is changed as well, which has Sensor Node c as its parent node. In the case of multiple faults as well, the path is rerouted in the same way. In addition, in VSDR, a forced fault can be caused at a sensor node. For the sensor nodes that have the forcibly-faulted sensor node as their parent, the path is rerouted to the sink node by going around it.

4 Implementation of VSDR The first screen that is shown when VSDR is started is shown in Fig. 8. The screen consists of the following. At the top, there is the Toolbar for loading a map and zooming in and out, among other things. On the right, there is the Configuration

Dynamic Visualization for Sensing Information

63

Fig. 8 Setup of VSDR and visualization for packet transmit in VSDR

Panel, for setting up sensor nodes as well as the sensor area shown; the Single Panel, for showing the path and information of a single sensor node; and the Multiple Panel, for showing the paths of multiple sensor nodes. On the bottom, there is the Status Panel, which shows information regarding the status of operation of VSDR. Finally, in the center, there is the Viewer, which shows the status of the map, as well as the status of sensor nodes. The Toolbar is made up of the following buttons: the Map Add button, for adding a map described by a GML document in order to make up the sensor network field; the Zoom In button, for zooming in a loaded map; the Zoom Out button, for zooming out; the Full Extension button, for resizing the map to fit the size of the screen; the Select Zoom Area button, for zooming in a specific area on the map; the Select Target Area button, for selecting a target area on the map; the Add Sensor Node button, which helps with adding an additional sensor node as necessary; and the Sensor Node Position button, for showing the locations of the current sensor nodes. The Configuration Panel is made up of the following: the Range Control, which sets the sensing/communication/supersonic ranges of a sensor node; the Frame delay Slide Bar, which adjusts the movement speed of a sensor; the check boxes, which are used to set whether sensing/communication/supersonic ranges of a sensor node are shown or not; the Degree K Taker, for setting the number of neighboring nodes that sensor nodes need to maintain; and the Number of Sensor Nodes Taker, for setting the number of sensor nodes to be placed within the target area. Finally, on the bottom, there is the Initial Deployment Selector, for selecting the initial location for placement of sensor nodes when VSDR starts up; the buttons for starting and configuring VSDR; and buttons that temporarily stop the operation of VSDR and start it again. The Single Panel is made up of the following: the Node List, which has a list of sensor nodes currently placed on the map; the Node Information, which informs the user of the type and ID of the selected sensor node, as well the parent node ID; the radio buttons for setting whether to automatically reroute the path or to

64

S.-H. Yang et al.

manually reroute it when rerouting the path; the Compulsion Fault Select, which causes a forced fault at a sensor node; the Change Node List, for the selecting sensor nodes for which the paths are changed; and the Parent Select List, for selecting among sensor nodes that meet the criteria as a parent node for the selected sensor node. The Multiple Panel is made up the Node list, for repetitively selecting a list of sensor nodes currently placed on the map, and other components that work the same as with the Single Panel: the radio buttons for setting whether to automatically or manually reroute the path when rerouting the path; the Compulsion Fault Select, which causes a forced fault at a sensor node; the Change Node List, for selecting among sensor nodes that meet the criteria as a parent node for the selected sensor node. Finally, the Viewer shows the status of sensor nodes placed on the map, as well as the paths between sensor nodes. In Fig. 8, sensor nodes have been placed on the map described by GML, showing the interconnectedness of the sensor nodes within the communication range. In addition, several sensor nodes have been chosen at random, and their paths to the sink node are shown. When packet transmission begins, the sensor nodes put together packets created by themselves and the packets received from their child nodes and sends them to their parent node. The paths are colored differently according to the amount of packet traffic. In the figure, it can be seen that overhead occurred at a single sensor node. In this case, if the path is rerouted to go in a roundabout way, the traffic between sensor nodes can be reduced.

5 Conclusions In this paper, VSDR was designed and implemented, which visually shows the paths between sensor nodes as well as the amount of packet traffic on a GMLbased map with GPS coordinates. VSDR is a simulator that includes a dynamic path rerouting algorithm when an overhead or fault condition occurs at a sensor node. Data loss is minimized by checking for successful delivery of data by sending a signal upon a parent node’s receiving data from its child nodes. In the case of transmission of data between sensor nodes, if the amount of traffic exceeds a certain limit, the path can be changed by automatically selecting a new parent node among neighboring nodes or, alternatively, the user can manually select among the neighboring nodes the parent node. Even if a fault occurs at a single or multiple sensor nodes, the path can be automatically rerouted or the user can manually change the path. For future work, a technique of path search that uses a variety of routing protocols should be used when overhead occurs in transmission of traffic in a sensor network. Also, the life of sensor nodes should be taken into account in adding a variety of different types of mobile sensor nodes.

Dynamic Visualization for Sensing Information

65

Acknowledgments This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2010-0022589, 2011-0003896)

References 1. Zhao F, Guibas LJ (2004) Wireless sensor networks. Elsevier, San Francisco 2. Arkin R, Ali K (1994) Integration of reactive and telerobotic control in multi-agent robotic systems. In: Third international conference on simulation of adaptive behavior, Aug 1994, pp 473–478 3. The Network Simulator—ns-2. http://isi.edu/nsnam/ns/ 4. Bajaj L, Takai M, Ahuja R, Tang K, Bagrodia R, Gerla M (1999) GloMoSim: a scalable network simulation environment. UCLA computer science department technical report 990027 5. Qualnet. http://qualnetworld.com/ 6. Levis P, Lee N, Welsh M, Culler D (2003) TOSSIM: accurate and scalable simulation of entire TinyOS applications. In: Proceeding of first ACM conference on embedded networked sensor system (Sensys 2003), Apr 2003, pp 126–137 7. NetSim. http://tetcos.com/software.html 8. Sobeih A, Chen W-P, Hou JC, Kung L-C, Li N, Lim H, Tyan H-Y, Zhang H (2006) J-Sim: a simulation and emulation environment for wireless sensor networks. IEEE Wirel Commun 13:104–119, Aug 9. Java in Simulation Time/Scalable Wireless Ad hoc Network Simulator. http://jist.ece. cornell.edu/ 10. OpenGIS Consortium, INC., Geography markup language [GML] implementation specification. http://www.opengeospatial.org/ 11. Shekhar S, Vatsavai RR, Sahay N, Burk TE, Lime S (2001) WMS and GML based interoperable web mapping system. GIS: Geogr Inf Syst 106–111, Nov 12. Guo Z, Zhou S, Xu Z, Zhou A (2003) G2ST: a novel method to transform GML to SVG. In: Proceeding of the 11th ACM international symposium on advances in geographic information systems, November, pp 161–168

Model Driven Real-Time Power System Application Development Using Just-In-Time Compilation Izudin Dzafic, Priyanka Mohapatra and Nenad Lecek

Abstract This paper describes a method of developing flexible power system applications using model driven development (MDD) and Just-In-Time (JIT) compilation for usage in real-time industrial solutions. MDD is used to create and process mathematical input models for power applications and to confine application development to creation of such models only. Thus application specific programming ends with mathematical model creation. Once a component model is designed, implemented and tested, it should be possible to reuse it as many times as necessary for a variety of purposes in power system application domain. In order to obtain code reusability a non-linear equation handler and solver are implemented. Both the application-specific converter and the general purpose equation handler and solver are designed as components. A JIT compiler is used to generate native code for given models. Models are compiled at run-time. Whenever the model shall be reused, generated native code is reused. Such approach provides significantly better performance compared to solution based on model interpreter. Compared to traditional, compiled software implementations, such solver implementation provides native code execution performance without programming. This provides a possibility to field experts to develop high performance software solutions for their algorithms, also without advanced knowledge of a specific programming language.

I. Dzafic (&)  P. Mohapatra  N. Lecek Energy Sector, Power Distribution Division, Siemens AG, Humboldtstr 59, 90459 Nuremberg, Germany e-mail: [email protected] P. Mohapatra e-mail: [email protected] N. Lecek e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_7,  Springer Science+Business Media B.V. 2012

67

68

I. Dzafic et al.

Keywords Model driven development analysis Rapid prototyping





Just-in-time compilation



Symbolic

1 Introduction The demand for efficient and re-usable software solutions for power system algorithms has increased significantly in last years. Development of power system applications using the traditional way of programming demands very good software engineering skills and very good domain know-how. A lack of software engineering skills by domain experts should not be an obstacle in development of high performance power system applications. The developer of a software solution for power systems should be more concentrated on the power system algorithms than on software design and implementation. During solution lifetime, some modifications in code are typically requested and implemented. With time, a software solution becomes fragile, often runs into maintenance problems and cannot be re-used because it is very specific to the particular application. This paper analyzes a solution to the above mentioned problems using MDD [1–8]. MDD aims to increase productivity in software development and manage the complexity of software by utilizing modeling to shift from programming in solution-space to modeling in domain-space. As described in papers [4] and [9] the complexities in software development are of two types: Essential and Accidental. The goal of using MDD is to decrease the accidental complexity in order to be able to better deal with essential complexity. In this paper is extensively discussed how to apply MDD to develop industrial and real-time solutions for power system algorithms. The paper proposes usage of MDD just for power system algorithms. A majority of software development is still done with code-centric approach and does not use separately defined models at all. It relies almost entirely on the code written, and expresses the model of the system directly in a programming language such as Java, C++, or C#. It becomes difficult to manage the evolution of these solutions as they scale and complexity increases. Models provide abstractions of a physical system that allow engineers to reason about that system by ignoring extraneous details while focusing on relevant ones. The previous work done in this field [10–15] identifies necessity for creation of re-usable analytical models. Prior to MDD re-usable software was developed by means of defining a domain architecture [16]. A domain model formulated the network using a system of simultaneous nonlinear equations. An object-oriented framework was derived from such a domain model. The domain architecture provided adaptive usage of the framework. MDD is used to create such an analytical model at run-time directly and this model is re-usable for future calculations, unless any topological changes occur in the actual network. For the example of power flow calculation the model created is a set of equations which is later on passed to the code generator. The main aim of using MDD is to keep the application development constrained to model creation step only.

Model Driven Real-Time Power System Application Development

69

In earlier works [15] machine modeling was done using declarative style of programming. The emphasis on a declarative style of programming was on the formulation of the problem and not on the solution steps of the problem. This kind of programming permits the mathematical description of a problem at the level of abstraction an expert is used to in this discipline. A completely general environment for the manipulation of mathematical models of all kinds is provided by Solver Q [10–15], including not only numeric but also symbolic computational capabilities. The Solver Q was developed as a generalized tool to solve variety of power system problems, in a way closer to human thinking [10–15]. This paper also focuses on such a solution which combines mathematical representation of power system models and symbolic computations using expression trees and developing a solver to deal with equations related to various problems in power systems. The problem of differentiation of power flow equations in order to compute Jacobian matrices is discussed in papers [17–19]. An automatic differentiation scheme is proposed which can be easily incorporated into a conventional functioncomputational code. The CPU time spent is only a function of the number of functional terms used and not the number of variables which is a major problem in case of symbolic computation. In papers [18] and [19] this method is further extended to use efficient auto-generated 3GL Jacobian, mismatch computation code and high-level symbolic equation and variable coding. The automatic solution to large systems of non-linear network equations by means of Newton– Raphson method using automatic code generation is analyzed. Over the last few years experts are analyzing the suitability of applying MDD to power system solutions. Paper [11] describes its actual application to real-time solutions, for example optimization, solving non-linear algebraic equations and time domain analysis. In previous approaches the numeric methods generated to solve the above mentioned types of equations needed a lot of programming and compilation and execution code generated later also lacked efficiency. Thus such approaches are unsuitable for usage in industrial solutions, which demand good performance and stability. In some other works, the optimization problem in power systems were solved using a combination of black-box tools for optimization and automatic code-generation for first order derivates [20]. The so-called black box tools were combination of a general-purpose optimization package (MINOS) and an automatic code differentiation tool (ADIFOR). Symbolic computational tools are basically designed to obtain explicit symbolic solutions to a variety of linear and nonlinear mathematical problems. In the area of power systems, pioneer work was performed in the late 80s at the University of Wisconsin-Madison in the development, application and use of symbolic computational tools [21] as well as symbolic-assisted numeric computational tools. The development and application of generic symbolic and numeric computational tools and graphical user interfaces for obtaining solutions of nonlinear problems and related equations are widely discussed, especially those associated with the simulation of power system transients. The use of these tools for power system modeling and simulation has yielded some interesting

70

I. Dzafic et al.

and unique results. Symbolic and symbolic-assisted numeric computational tools have now become part of the mainstream programs used for the analysis, modeling and simulation of power systems, particularly in research and education. For example, in paper [21] usage of MATLAB based programs, which are numeric computational tools that can be used to perform a variety of power system studies, has been discussed. These tools are a good example of how symbolic computation can be used in the development of commercial-grade software for power system analysis, as Jacobian and Hessians can be readily and reliably obtained symbolically for a variety of system models to then integrate them into any numeric computational tool. Another usage of such tools is cited in paper [22] where large disturbance behavior of power systems is captured in a systematic way by a model that consists of differential, switched algebraic and state-reset (DSAR) equations. The DSAR model is then differentiated symbolically. Yet another usage [23] shows modeling of a system as hybrid automation. Such models include the ‘mixed logical dynamic system’ (MLD) or a modified version the ‘dynamic mixed integer program’ (DMIP). Then a symbolic tool is used to convert the hybrid automaton model of a power system into one of these forms. Also symbolic numeric steady state power flow model can be developed for fast and comprehensive contingency analysis [24]. This paper introduces the novel approach of using JIT compiler [25, 26] instead of code interpreter [12, 13] to solve a set of numeric methods from the analytical model. The JIT compiler translates source code into native binary code. This compilation process is done only once, and a link is created between the source code and the corresponding compiled binary code. This increases the speed at which a program runs. It speeds up execution of mathematical intensive functions by up to 50 times. The main reason for using a JIT compiler is to reach the performance of static compilation, while maintaining the advantages of source code interpretation. This paper presents a new approach to power system software application development with the following goals: • • • • •

Easier involvement of domain experts in power system application development Code reusability Speeding up programming Easier maintenance of large power system models Independence from programming languages

2 Comparison of Traditional Programming and MDD The difference in the model creation is illustrated with an example. A simple radial feeder of a power system network (Fig. 1) is considered and the input model is created both in a traditional way and by using MDD. Traditionally the initial model for power flow calculation is created by performing topological tracing over the database storing the network configuration

Model Driven Real-Time Power System Application Development Fig. 1 IEEE 4 node test feeder

2

1

71 3

Infinite Bus

4 2500 ft.

2000 ft.

[I12]

[I34]

Load

[I23]

/// Class CModel implements the general model used by PF Calculation class CModel { public: /// Map storing system buses std::map m_mCBus; /// Map storing slack buses std::map m_mCSlackBus; /// Map storing tap changers std::map m_mCTapChan; /// Map storing system transformers std::map m_mCXfmr; /// Map storing system lines std::map m_mCLine; /// Map storing system loads std::map m_mCLoad; /// Map storing system shunts std::map m_mCShunt; /// Map storing system generators std::map m_mCGen; /// Map storing system switches relevant for the current flow calculations std::map m_mCSwitch; };

Fig. 2 Power system model using traditional programming methodology

and collecting the various network equipments in C++ Standard Template Library (STL) containers. The model created for the node test feeder shown in Fig. 1 using C++ and STL containers is represented in Fig. 2. This process is very time consuming and requires extensive knowledge of C++, STL and other programming concepts to be able to write code to create such a model. Additionally it increases memory consumption of the program and is cumbersome when it comes to large power system networks. As can be seen the model thus created is only a collection of the network elements and their static properties. This way of model creation needs more programming to define the set of equations to be solved for power flow calculation. This is the traditional approach. Instead, a mathematical model of the power system network can be directly created from the information available in the input database during topological tracing and stored in an intermediate format. In case no topological changes are reported by SCADA (Supervisory Control and Data Acquisition) this mathematical model remains unchanged and can be re-used for future calculations. In the component based approach, described in this paper the model created is nothing but a set of equations directly created during topological tracing from the input database specific to the purpose of the power system application. It is then stored in an intermediate format (Expression Trees) which will be then used by the

72

I. Dzafic et al.

Fig. 3 Mathematical model representation

code generator in the next step to create auto-generated code. The mathematical model created from the network is shown in Fig. 3. It should be noticed that the equations needed to be solved for power flow calculation are created on the fly and as a part of model creation. The model creation is basically the end of the application specific development. The next steps to solve these equations are described in the following sections. These steps are independent of the physical network and are generic modules to solve any set of equations. Thus MDD is applied to auto generate executable binaries as a part of model creation procedure. For the network presented in Fig. 3, there are eight variables and eight equations to be solved for. The mathematical model consists of the following: • Model constants (admittance of the lines y) • Model parameters (Load values Sload obtained from load curves, measured, or estimated) • State variables (voltage at various nodes V)

3 Features of Power System MDD There are many advantages of MDD; as mentioned in the previous section the process of model creation is purely analytical and presentation of the physical network as a set of mathematical equations. In this section some additional features are listed which makes MDD a good choice for application to real-time

Model Driven Real-Time Power System Application Development

2

1 y 12 V1

3 y23

V2

2

1 I12

4 y 34 V4

3

4 I34

2/3 Node Reduction

4

I12 = (V1 − V2 )y12

V3

I23

1

73

V2− V3 = 0, where Z 23 = 0 I 34 = (V3 − V4 )y34

⇒ I 34 = (V2 − V4 ) y34 I12 − I 23 = 0 I 23 − I 34 = 0

Fig. 4 Difference in handling of switches

applications. One feature of auto-generating mathematical models for power flow calculations is illustrated in Fig. 4. In the 1st approach in traditional methods of programming the switches in the model are eliminated, by means of node reduction. Since switches have no impedance the two nodes of the switches are reduced to single node. In order to achieve such a node reduction, special handling has to be done in the code and extra lines of code have to be implemented. The same problem is solved in MDD approach by writing mathematical equations applicable to switches in the network. The voltages at both nodes of the switch are equal. This implies no special handling is required and model is created by simply writing the correct set of equations. The model thus generated can be used for further calculations with only a minor change of parameters, unless the topology of the network changes. A topological update implies a change in the number of network elements in a subsystem and thus the mathematical model for this subsystem has to be re-generated. But as such topological changes do not occur very often, re-using the model for calculations is possible and it improves efficiency and performance of the application. Another very important feature of MDD is the possibility to create a new model using composition of models. This implies the output after solving one model can be used as an input for another one. This provides the flexibility of creating smaller models and enables multiple usages of the same model. A combination of two or more models is also possible during run-time. In case a particular topology in the network can be easily solved by simply combining two or more existing models; the models are just combined with each other with minor changes and thus the new model created is only partially re-generated and the previously existing models are re-used. The analytical model created may consist of three kinds of equations to solve for. • Nonlinear optimization • Nonlinear algebraic equations • Time domain analysis

74

I. Dzafic et al. Model Creation Input Model (Set of Equations) Code Generator Expression Tree Syntax Validation

Simplification of Equations

Symbolic Computation Lagrange,Jacobian

Initial Guesses

Numeric Methods

RPN Converter

Solver Model Parameters C/C++ Code

OR

Interpreter

OR

JIT Compiler

Calculation Parameters

Output

Fig. 5 Workflow MDD for power systems

Depending on the type of problem to be solved, the code generator goes through different steps to create numeric methods to be passed on to the Reverse Polish Notation (RPN) converter. The code generator is the module which converts the mathematical model described in the section above into a set of tokens in RPN to be solved by the solver in the next step. The following steps are then executed in the code generator in order to generate tokens. The complete workflow is represented in Workflow MDD for Power Systems (Fig. 5). In the first step, the syntax of set of equations generated as a part of model creation is validated. This step makes sure that the equations are meaningful and no errors were made during the process of model creation. It is a very important step in order to avoid problems due to obvious errors, in creation of numeric methods and parsing in later steps. The checks are made for syntax errors like opening and closing of parenthesis, proper end of lines. In following steps, expression trees are used as model holders. The further simplification of the equations is done in this step, to reduce the number of steps required to process the equations in next steps. Depending on the type of the mathematical model created and the type of power systems problem to be solved different kinds of symbolic computations are carried out for generation of numeric methods. The goal of optimization analysis is to find optimal solution for the given criterion. The following paragraphs give a brief review of mathematical formulation needed for automatic code generation for solving the optimization problem in power systems. This mathematical model is going to be used in explaining the proposed approach. The general optimization problem is described as follows:

Model Driven Real-Time Power System Application Development

75

Minimize : min f ðx1 ; x2 . . .xn Þ Subject to : Gðx1 ; x2 . . .xn Þ ¼ 0;

ð1Þ

Applying Lagrange Multipliers the Lagrange functions are constructed as shown in Eq. (2). LðX; kÞ ¼ f ðXÞ  kGðXÞ

ð2Þ

where X ¼ ½x1 ; x2 ; . . .xn  k ¼ ½k1 ; k2 ; . . .kk  Applying Karush–Kuhn–Tucker conditions, the equations can be re-written as, rL ¼ 0 i:e; k of X oGm  km ¼ 0; i ¼ 1. . .n oxi oxi m¼1

ð3Þ

and; Gðx1 ; x2 . . .xn Þ ¼ 0 (3) is a set of N nonlinear equations with the same number of unknowns. This is later solved using a Newton–Raphson method. It is important to notice that ‘‘developer’’ writes only equations in the first step (1). A process of automatic code generation transforms mathematical optimization model described with (1) into (2) and (3). An additional step is then required to generate the symbolical Jacobian, needed to solve for Eq. (3). This is, however, part of symbolical Jacobian computation. x1 x2 f1 2 of1 ox

f2

.. . fn

6 of 1 6 2 6 ox1 6 6 6 6 6 . 6 .. 6 6 6 4 ofn ox1

of1 ox2 of2 ox2



x3 of1 ox3

..

 





. ..





. 

   xm 3 of1 oxm of2 oxm

7 7 7 7 7 7 7 .. 7 . 7 7 7 7 5

ð4Þ

ofn oxm

In the case where the symbolic computation is being done only for a set of nonlinear algebraic equations, after model creation the symbolic computation begins directly with generation of Jacobian. The Jacobian is generated for any set of functions f with any set of variables x as represented in Eq. (4).

76 Table 1 Stack contents evaluating expression in RPN Execution step 1 2 3 4 5 Stack Contents 1 1,2 3 3,4 12

I. Dzafic et al.

6 12,5

7 12

8 17,3

9 14

After the creation of the numerical methods, they are passed on to the RPN converter for generation of tokens in RPN in order to be passed on to the solver for final calculation. After the numerical methods are determined and before the first iteration the initial guesses are put into the equations in order to reduce the number of variables and set the boundary conditions. The generation of the numerical methods is an iterative procedure, after each calculation the calculated values are replaced in the Jacobian and the function f in order to determine the next set of equations. This procedure is repeated till the convergence criterion is achieved.

4 RPN Converter A stack machine is a computational model where operands to instructions are always placed in a LIFO data structure, or a stack. The biggest advantage of stack machines is the high density of the code, as operands selection is not encoded in the instructions stream, but they are implicitly gathered from the top of the stack. The taxonomy of these kinds of machines is defined by the number of stacks and the purposes of each of them. The expressions in RPN are evaluated from left to right with the following rules: • Numeric values are pushed on the stack • Operators pop the arguments from the stack and push the result back. The above approach is for binary operators, but the same is extensible to unary functions. Although the features of such architecture were deeply investigated in the late eighties, and several physical platforms were implemented for this model, current mainstream CPUs are register based, with stack support. In RPN calculations occur as soon as an operator is specified. Thus, expressions are not entered wholesale from right to left but calculated one piece at a time, most efficiently from the center outwards. The automatic stack permits the automatic storage of intermediate results for use later: this key feature is what permits RPN interpreters to easily evaluate expressions of arbitrary complexity: they do not have limits on the complexity of expression they can calculate, unlike algebraic interpreters. Table 1 shows the stack contents during evaluation of expression 5-((1 ? 2)4) +3; written in RPN 1; 2; +; 4; -; 5;*; 3; +, [24]. Figure 6 represents the stack status during the execution of the example. The RPN converter is a part of the code generator and it converts the numeric methods from expression trees into RPN to be solved by the solver as the next step.

Model Driven Real-Time Power System Application Development Fig. 6 Stack status during execution of the example

Fig. 7 Automatic code generation

77 push 2

RPN Converter

Fig. 8 Native code generation using JIT compiler

push 1

2

add

1

1

3

Automatic Code Generator

C/C++ Code

RPN Converter

C++ Compiler&Linker

Executable

JIT compiler

Native Code

5 JIT Compiler (Solver) The code generated from the code generator can be passed on to an interpreter, JIT Compiler or converted to C/C++ code to be executed as shown in Fig. 7. The fastest way to solving the equations in case of power system applications is to use the JIT Compiler. C/C++ code is higher level programming language and needs extra compilation and interpretation of the compiled code. This slows down the execution. Interpreted code is translated from a high-level language to a machine code continuously during every execution, whereas statically compiled code is translated into machine code before execution, and only requires this translation once. A common goal of using JIT techniques is to reach the performance of static compilation, while maintaining the advantages of source code interpretation. JIT compilers generally offer far better performance than interpreters. In addition, they can in some cases offer better performance than static compilation, as many optimizations are only feasible at run-time. Automatic code generation is the generation of source code at runtime (Fig. 8). The numeric methods in RPN are converted to C/C++ code at runtime and then compiled to generate an executable. This process though popular is time consuming and slow. The job of the JIT compiler is to convert intermediate code into the machine code or the native code for execution (Fig. 8). With the JIT code the code can be optimized to a particular operating system. The advantage of converting the source code to the intermediate code is that it can run on any operating system provided JIT for that operating system is available. The basic workflow of the solver is shown in Fig. 9. The tokens generated by the RPN converter are passed on to an interface function in the solver as the calculation parameters. Equation (4) represents the Jacobian created during

78

I. Dzafic et al.

Fig. 9 Workflow JIT compiler

∂f1 ∂xn

JIT compiler

f(x)

Table 2 Comparison of performance of power flow calculation Traditional MDD using programming (ms) interpreter (ms) Creation of power system model Power flow calculation Total

Pointer to function f(x)

MDD using JIT compiler (ms)

444

492

511

605 1049

1814 2306

542 1053

Table 3 Comparison of performance of optimal power flow calculation Traditional MDD using MDD using JIT programming (ms) interpreter (ms) compiler (ms) Creation of power system model Optimal Power flow calculation Total

704

784

807

2238

7243

2186

2942

8027

2993

symbolic computation; given the elements Jacobian are not constant but functions as represented in Fig. 9, this function is passed on to the JIT compiler as a set of tokens and after compilation a pointer to the compiled function is returned to the code generator. In contrast to solvers implemented as interpreted code, solvers based on JIT compilation have much smaller memory consumption. This is a big advantage for such solvers to be used in real time applications.

6 Results In order to test the performance of MDD with JIT compiler, tests were carried out on a power systems network with 6108 nodes, 1896 lines, 1212 busbars, 120 transformers and 864 loads. Results are compared with hand made code. The tests were conducted on x86 as well as SPARC processors, on Windows, SUN Solaris, and LINUX platforms. There are two sets of tests conducted for two different power system problems. The performances of power flow calculation using traditional programming methodology and MDD with interpreter and JIT compiler are displayed in Table 2. The second test was carried out for comparing the performances for solving optimization problems. The results obtained are displayed in Table 3.

Model Driven Real-Time Power System Application Development

79

As it can be observed from the test results, the creation of power system model is marginally faster using traditional method of programming in all test cases. The solution based on MDD and JIT compiler provides comparable run-time performance as traditionally developed solution and significantly better performance than MDD using interpreter.

7 Conclusion As explained in this paper, MDD has several advantages over traditional ways of programming. It guarantees stability, efficiency and re-usability. MDD, however, cannot completely replace traditional programming. It provides the possibility to the field experts to develop high performance algorithms and analytical models, without wasting much time on traditional ways of programming. The tests results confirm that the performance of the power system applications developed with MDD and JIT compiler is comparable to those developed using traditional methods of programming. As expected, the test results show that MDD solution using JIT compiler has significantly better performance than solution based on MDD with interpreter. These results show a way to relieve power engineers from time consuming and demanding software development for some power applications. At the same time, the power system applications developed with MDD and JIT compiler are simpler and easier to maintain than traditional software implementations. It can be concluded that MDD in future can be used on large scale in industry applications, in order to avail of its advantages. There is also a lot of scope in future to use MDD in power systems real-time applications.

References 1. MDA resources. Object Management Group. [Online]. Available:http://www.omg.org/mda/ index.htm 2. Selic B (2003) The pragmatics of model-driven development. IEEE Software 20(5):19–25 3. The Middleware Company (2003) Model driven development for J2EE utilizing a model driven architecture (MDA) approach. Productivity Analysis, June 2003 4. Selic B (2006) Model-driven development: its essence and opportunities. In: Proceedings of the ninth IEEE international symposium on object and component-oriented real-time distributed computing 5. Gao J, Li D, Zheng S (2006) Model-driven development of real-time software based on OMG standards. In: Proceedings of the first international conference on innovative computing, information and control 6. Ma Z, He X, Kang L (2009) A model driven development platform for service-oriented applications. World conference on services-II 7. Aho P, Merilinna J, Ovaska E (2009) Model-driven open source software development—the open models approach. In: Fourth international conference on software engineering advances

80

I. Dzafic et al.

8. Heijstek W, Chaudron MRV (2009) The impact of model driven development on the software architecture process. In: 36th EUROMICRO conference on software engineering and advanced applications 9. Brooks F (1986) No silver bullet—essence and accident in software engineering. In: Proceedings of the IFIP tenth world computing conference, pp 1069–1076 10. Dzafic I, Tesnjak S, Glavic M (2002) Automatic object oriented code generation to power system on-line optimization and analysis. 21st IASTED international conference on modeling, identification, and control (MIC 2002), Innsbruck, Austria, February 18–21, 2002 11. Dzafic I, Glavic M, Tesnjak S (2004) A component-based power system model-driven architecture. IEEE Trans Power Syst 19(4):2109–2110 12. Glavic M, Dzafic I, Tesnjak S (2005) A general-purpose symbolically assisted numeric computation environment as a support in power engineering education. IEEE Trans Power Syst 19(1):3–12 13. Dzafic I, Alvarado FL, Glavic M, Tesnjak S (2002) A component based approach to power system applications development. In: 14th power systems computation conference, Sevilla, Spain, June 24–28, 2002 14. Alvarado FL, Liu Y (1988) General purpose symbolic simulation tools for electric networks. IEEE Trans Power Syst 3(2):689–697 15. Alvarado FL, Canizares CA, Keyhani A, Coates B (1991) Instructional use of declarative languages for the study of machine transients. IEEE Trans Power Syst 6(1):407–413 16. Bosshart P, Bacher R (1998) A domain architecture for solving simultaneous nonlinear network equations. IEEE Trans Power Syst 13(3):1006–1012 17. Jerosolimski M, Levacher L (1994) A new method for fast calculation of Jacobian Matrices automatic differentiation for power system simulation. IEEE Trans Power Syst 9(2):700–706 18. Bacher R (1996) Computer aided power flow software engineering and code generation. IEEE Trans Power Syst 11(1):490–496 19. Orfanogianni T, Bacher R (1999) Using automatic code differentiation in power flow algorithms. IEEE Trans Power Syst 14(1):138–144 20. Orfanogianni Yy T, Bacher R (2000) Increased OPF Code development efficiency by integration of general purpose optimization and derivative computation tools. IEEE Trans Power Syst 15(3):987–993 21. Canizares CA (2005) Applications of Symbolic computation to power system analysis and teaching. In: IEEE-PES general meeting, San Francisco, CA, June 2005 22. Hiskens IA, Sokolowski PJ (2001) Systematic modeling and symbolically assisted simulation of power systems. IEEE Trans Power Syst 16(2):229–234 23. Kwatny HG, Mensah EF, Niebur D, Teolis C (2006) Symbolic construction of dynamic mixed integer programs for power system management. IEEE PSCE 24. Liao H, Niebur D (2005) Symbolic/numeric steady-state contingency analysis of electric power systems. IEEE 25. The SPARC architecture manual, version 8, 1991, 1992 SPARC International Inc. 26. Intel 64 and 32 Architectures software developer’s manual, volume 2A, Instruction Set Reference, A-M, 1997–2010 Intel Corporation 27. Intel 64 and 32 Architectures software developer’s manual, volume 2B, Instruction Set Reference, N-Z, 1997–2010 Intel Corporation

An Efficient Distribution Sensitive Privacy for Real-Time Applications Arijit Ukil, Jaydip Sen and Sumit Ghosh

Abstract Traditionally privacy preserving database sanitization utilizes some kind of systemic transformation of the original data set and maps the entries into a different set subject to specific utility and privacy requirements. It is obvious that the distribution of sensitive attributes in original database is domain specific and dependent on the application. Another factor that influences the choice of transformation method is the processing and publishing latency constraints, deterministic for handling the required complexity. In this paper, we propose an efficient way of database privacy protection algorithm which provides an optimal operating point for privacy preserving complexity while maintaining the required utility to minimize the information loss. Our algorithm is sensitive-attribute’s distribution aware and the optimization space is limited by the distribution type. Using the conventional k-anonymity and l-diversity techniques, we analyze and show the effectiveness of our proposed protocol, which outperforms the well-known methods in certain domain of database privacy preservation applications.









Keywords k-anonymity l-diversity Privacy-preservation Utility Leptokurtic Information loss

A. Ukil (&)  J. Sen Innovation Lab, Tata Consultancy services BIPL, Sector-5, SaltLake, Kolkata 700091, India e-mail: [email protected] J. Sen e-mail: [email protected] S. Ghosh Sikkim Manipal Institute of Technology Rangpo, East Sikkim, India e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_8, Ó Springer Science+Business Media B.V. 2012

81

82

A. Ukil et al.

1 Introduction With the proliferation of World Wide Web and Internet, it is widely accepted that databases and data repositories represent an important asset for many applications. The availability of huge numbers of databases recording a large variety of information about individuals makes it possible to discover information about specific individuals by simply correlating a certain number of available databases. There are also some requirements of releasing sanitized database for research and other purposes. Privacy relates to what data can be safely disclosed without leaking sensitive information about the legitimate owner. As the volumes of stored data increases, privacy concerns are also increasing and this leads to the rapid development of privacy preserving data management research. Number of techniques like k-anonymity has been suggested in recent years in order to perform privacypreserving data mining. Fundamental motivation of privacy preserving data management is to sanitize the database to achieve the needed amount of privacy using some transformation or mapping algorithm. These transformation methods and the mapping of original database and published database are governed by the privacy requirement. The existing schemes for database privacy preservation algorithms like k-anonymity and l-diversity are search over space of possible multi-dimensional solutions, so they are NP-hard in nature [12]. The high dimensionality of these techniques also adds computational overhead [3]. So, these types of solutions are not suitable in real-time systems. In order to resolve the issue of real-life privacy preservation in data management where real-time publication of the data is required; two optimization problems need to be addressed: 1. Trade-off between privacy requirement and utility loss: in this paper, we consider privacy preservation as a hard requirement and utility is to be maximized under that constraint. 2. Trade-off between response/publishing time and privacy protecting techniques’ complexity. In this paper, we attempt to solve these two issues from a completely different perspective, by considering the distribution of the sensitive attributes. Solving the two issues simultaneously as stated above is practically infeasible [7]. It is observed that in many real-life databases the distribution of sensitive attributes follow leptokurtic pattern, where the sample space is more concentrated within certain range. Another point to be noted is that computational complexity of privacy preserving algorithms presented in the literature [7] is high. Approximate solution of k-anonymity is O (klogk) [2], which is improved to O(k) [12] and that of l-diversity is O (l2) [11]. In view of that, we propose a sensitive attribute distribution aware protocol which minimizes the computational complexity while preserving same amount of privacy and utility score. Our algorithm is most optimal in leptokurtic distribution, where as in ideal platykurtic distribution our algorithm does not yield a comparative better result.

An Efficient Distribution Sensitive Privacy

83

This paper is organized as follows. In Sect. 2, we present related work. In Sect. 3, the problem is presented formally. In Sect. 4, the considered database, which has some real-life implications, is described. Our experiments are based on this kind of database holding some practical properties. We present our algorithm in Sect. 5. In Sect. 6, experimental results and analysis are shown. Finally, we conclude the paper in Sect. 7.

2 Related Work There exists diverse techniques for studying different transformation methods associated with privacy as randomization [7, 14], k-anonymity [4], and l-diversity [11]. Another important factor influencing the privacy preserving method is the utility loss. It is understood that privacy preservation results in utility loss. An absolute privacy protected database has zero utility. Association rule hiding and query auditing are the traditional methods used for utility based privacy preservation [5, 13]. Another issue that is to be addressed is the complexity of the algorithms found in the literature. For some real-life conditions like online streaming of and sharing of users’ private data, vehicular communications, minimizing the processing and publishing time of the original database is of utmost importance. In [13], privacy preservation in data streams is discussed using noise addition, which minimizes the utility of the data. Agarwal and Yu [1] described condensation based data stream privacy by using the concept of incremental updating. Their approach is useful for driving aggregated statistics, not raw data anonymization. So, this scheme is not suitable where perturbation of sensitive data stream is required. However, our objective is not directly related to provide privacy preservation in data streams, but to minimize the latency of privacy preservation process so that it can be fine tuned for real-time databases and data streams.

3 Problem Statement Let T ¼ ft1 ; t2 ; . . .; tn gbe a table with attributes fA1 ; A2 ; . . .; Am g. We assume that T is a subset of some larger population X where each tuple represents an individual from the population. Let K denote the set of all attributes fA1; A2 ; . . .; Am g and  t ½ Ai  denote the value of attribute Ai for tuple t. Now, if C ¼ C1 ; C2 ; . . .; Cp  K;   then we use the notation t½C to denote the tuple t½C1 ; . . .; t½Cp  , which is the projection of t onto the attributes in C while S denote the set of all sensitive attributes and N denote the set of nonsensitive attributes. k-anonymity [7] is a property that captures the protection of released data against possible re-identification of the respondents to whom the released data refer. A table T satisfies k-anonymity if for every tuple t 2 T, there exist k-1 other tuples

84

A. Ukil et al.

ti1 ; ti2 ; . . .; tii1 2 T such that t½C ¼ ti1 ½C ¼ ti2 ½C ¼    ; tii1 ½C for all C 2 QI. k-anonymity suffers from homogeneity and background knowledge attack. Therefore, the technique of l-diversity was proposed which not only maintains the minimum group size of k, but also focuses on maintaining the diversity of the sensitive attributes. Let a q*-block be a set of tuples such that its non-sensitive values generalize to q*. A q*-block is l-diverse if it contains l ‘‘well represented’’ values for the sensitive attribute S [10]. In order to preserve privacy of a database from attacker, it is absolute necessary that both k-anonymity and l-diversity is to be incorporated. However, it is very impractical in real-life large database where real-time release of data is required, to k-anonymized l-diverse database sanitization. The first problem to be addressed is that how to achieve same kind of privacy measure with lesser complexity. In fact, jointly performing k-anonymity and l-diversity on a database results in computational complexity of O (k* l2) even for approximate computation [11, 12]. The second problem is that this algorithm should not be compromised in terms of sensitive value disclosure probability ðdÞ; which means that release of sanitized database would be such that d\dThreshold . So, the problem is to transform the table T to T*, such that: O ðT ! T*Þ  / (publishing time) dðT ! T*Þ\dThreshold

4 Sensitive Attributes’ Distribution Pattern It is previously discussed that to simultaneously optimize k-anonymity and l-diversity for generic database, large amount of computational complexity is to be handled. In order to circumvent this problem, we need to analyze the distribution of the sensitive attributes of the original dataset. We find that kurtosis is a good indicator of diversity spread. Kurtosis is a measure of the peakedness of a distribution is. As stated earlier, most of the real-life large databases consist of sensitive attributes which are mostly concentrated, i.e. they are leptokurtic in nature. In this paper, we consider only leptokurtic distribution. The sample snapshot from a large medical database is, shown in Table 1. Kurtosis (j) of the original table is shown in Fig. 1 (with unimodal kurtosis j of the above figure is 4

Þ  3: +7.8), where j is defined: j ¼ Eðsl r4

5 Distribution Sensitive Variable Privacy Algorithm The objective is to provide an algorithm that is of low complexity in nature while the privacy and utility measures are not compromised. So, we consider these parameters:

An Efficient Distribution Sensitive Privacy

85

Table 1 Sample dataset Name Age

Sex

HIV

Salary(K)

John Rita Doug Eva Martin Alice Tim

M F M F M F M

-ve -ve -ve -ve +ve -ve -ve

250 9425 21 291 163 151 208

56 45 19 71 22 43 31

Fig. 1 Sensitive attribute distribution (following leptokurtic distribution)

Normalized Certainty Penalty (NCP): It measures the generalization of attribute Ai in terms of interval size. Consider a table T with the set of quasi-identifying attributes ðA1 ; . . .; An Þ: Suppose a tuple t ¼ ðx1 ; . . .; xn Þ is generalized to 0 tuple t ¼ ð½y1 ; z1 ; . . .; ½yn ; zn Þ; such that yi  xi  zi : Then, we define NCP of tuple t on attribute Ai as: NCPAi ðtÞ ¼

zi  yi j Ai j

jAi j ¼ max t:Ai  min t:Ai t2T

t2T

Adversary’s baseline knowledge (Abase): We formally define Abase as the vector of probabilities representing the distribution of sensitive attribute values in the entire table T: Abase ¼ hp ðT; s1 Þ; . . .; pðT; sn Þi p(U, s) is the probability that a randomly chosen member of U has the sensitive attribute value S. Adversary’s posteriori knowledge (Asan): The adversary can uniquely identify the quasi-identifier equivalence class \ t [ containing the sanitized record of t in T0 . Asan is the distribution of sensitive attribute values within this class \ t [: Asan ðhtiÞ ¼ hpðt; s1 Þ; . . .; pðhti; sn Þi We use k-anonymized l-diverse transformation to privacy preserve the original database. The objective is to maximize the utility score (information gain) while d\dThreshold : It is evident that as the value of k in k-anonymity increases, the lesser

86

A. Ukil et al.

will be the knowledge gained by the adversary Aknow ¼ ðAsan ; when Abase \AThrshold base Þ looking at the sanitized database. When k [ Kth where Kth is a threshold, the adversarial knowledge gain Aknow becomes zero. Also, when ldiversity is included along with k-anonymity in the sanitization method, there is a substantial reduction of the adversarial knowledge gain, that is, Alknow  Aknow , where Alknow is the adversarial knowledge gain when the sanitization method includes k-anonymity and l-diversity, as we know Aknow ¼ f ðkÞ and Alknow ¼ f ðk; lÞ: But, when k [[ l, Aknow ! Alknow , provided that the threshold value when l-diversity is included in the sanitization method, KlTh , remains same, i.e., KlTh  KTh . This happens due to the fact that for l-diversity, adversary’s accuracy gain decreases with increasing l and for k-anonymity, adversary’s knowledge gain decreases with increasing k. Increasing k value much larger than l of k-anonymized l-diverse sanitization, decreases the privacy disclosure probability ðdÞ which reduces adversary’s information gain (related to l-diversity). We prove that adversary’s information gain is bounded by d. So, providing d is sufficient when k [[ l. Adversary’s information gain (I. Gain) is the amount of information the adversary gains about the original database analyzing the published database. I. Gain (S, Q) is defined as the difference between the entropy of S and the conditional entropy H(S|Q): I: Gain ðS; QÞ ¼ HðSÞ  HðSjQÞ. Lemma If T satisfies d-disclosure privacy, then I:Gain ðS; QÞ\d. We say that an equivalence class \t [ is d-disclosure private with regard to the sensitive attribute S, if for all s e S,    pðhti; sÞ log \d  pðT; sÞ  A table T is d-disclosure private if for every t  EQ , htiis d-disclosure private [6]. Proof   hti  jTs j s and bt;s ¼ pðhti; sÞ ¼ Let, as ¼ pðT; sÞ ¼ jTj jhtij HðSÞ ¼

X

as log as and HðSjQÞ ¼ sS

I: Gain ðS; QÞ ¼ ¼

X sS

as log as 

X tEQ

X tEQ

X

jhtij X bt;s log bt;s tEQ hTi sS

jhtij X bt;s log bt;s sS hTi

bt;s jhtij X bt;s log sS as hTi   pðhti; sÞ log \d  pðT; sÞ 

An Efficient Distribution Sensitive Privacy

87

Table 2 Symbol and meaning Symbol

Meaning

EQP c cl n m

Set of k-anonymized l-diverse equivalence classes A k-anonymized equivalence class A k-anonymized l-diverse equivalence class Total number of k-anonymized equivalence classes Total number of k-anonymized l-diverse equivalence classes

X

jhtij X b d tEQ T sS t;s X X   d hti  \ s jTj tTQ sS

I: Gain ðS; QÞ\

As,

P P  htis ¼ jTj

tTQ sS

I: Gain ðs; QÞ\d

This shows that if we satisfy d for a certain database, adversary’s information gain can be assumed to be less than the prescribed limit. This proves that satisfying d is very important. This is due to the fact that when the diversity of the sensitive attributes is much more than granularity of inter-group representation, k-anonymity outperforms l-diversity. So, if k  l, adversary’s knowledge gain and accuracy gain is same for k-anonymized and k-anonymized l-divevrse sanitization. As the value of k in k-anonymity increases, there is a greater number of tuples in each equivalence class c 2 EQ , where EQ is set of k-anonymized equivalence class. In case of leptokurtic distribution, the chances of a more diverse set of sensitive attribute values being present in a single equivalence class increases. Therefore, we may predict that the effect of l-diversity decreases in sanitization for a larger value of k. So, in distribution of sensitive attributes for each quasiidentifier has low entropy. This results low value of I:Gain ðS; QÞ. Consequently small value of d is sufficient, for which even when k l, Aknow ¼ Alknow and AACC ¼ AlACC : Based on those arguments, we propose our algorithm, termed distribution aware privacy preservation. This algorithm first detect the j of the sensitive attributes. Then it finds the optimal value of k and l. It settles for k-anonymity only if it finds k [ l. However, different sets of sensitive attributes can have different j values in a single database. In that case, only the sensitive attributes which has j [ 3 are considered. This optimization does not yield better performance gain for tables with sensitive attributes distribution is mesokurtic or platykurtic (Table 2).

88

A. Ukil et al.

Table 3 Parameter values of the experiment data sets No. Test case 1.

User requests for medical database publication

2.

User requests for salary database publication

Evaluated parameters Required privacy provided Measures k only NCP 27.52 0.238 Aknow Aacc 0.14 q 0.52 Required privacy provided Measures k only NCP 32.76 0.152 Aknow Aacc 0.09 q 0.436

k=5 k with l 32.57 0.202 0.11 0.43 k = 10 k with l 34.55 0.149 0.09 0.436

6 Experimental Results and Analysis We have performed experiments to prove the efficacy of our proposed algorithm. These experimental results and consequent observations are shown in this section. We have considered two test cases, with sensitive attributes in medical domain (patient’s disease/treatment information) and portfolio management in finance (customer’s salary information). We have found following outcomes of the parameters based on which we show the effectiveness of the proposed algorithm. This is shown in Table 3. First, we demonstrate the variation of adversary’s knowledge gain when k-anonymized l-diverse algorithm is used and when only k-anonymized algorithm used. We used leptokurtic distribution of kurtosis j ¼ 7:8 for medical database and j ¼ 5:5 for financial database. It is observed in Fig. 2 that initially Alknow  Aknow upto k = l, when k = l = 5, Alknow ffi Aknow . This confirms our previous analysis. From Fig. 3, it is evident that as the value of k in k-anonymity increases, the lesser is adversary’s accuracy gain ðAacc Þ With Aacc , the attacker/adversary may predict the sensitive attribute value looking at the sanitized database. With increasing value of k, Aacc decreases and when l-diversity is included along with k-anonymity in the sanitization method, there is a substantial reduction of the adversarial accuracy gain, that is, Alacc  Aacc , for k  l; where Alacc is the adversarial accuracy gain when the sanitization method includes k-anonymity and l-diversity. However, when k [ l, Alacc  Aacc (at k = 6 onwards). This again confirms our postulate that k-anonymized l-diverse data set becomes independent of l-value, when k [ l for leptokurtic distribution in terms of Aacc and Aknow . We conduct the same experiments to prove our hypothesis in case of d-variation and it is found that we can draw same conclusion. This is shown in Fig. 4. As the value of k in k-anonymity increases, there is a greater number of tuples in each equivalence class c 2 EQ . So, the probability of sensitive attribute values being present

An Efficient Distribution Sensitive Privacy

89

Fig. 2 Adversary’s knowledge gain with and without l-diversity

Fig. 3 Adversary’s accuracy gain with and without l-diversity

Fig. 4 d-variation with and without l-diversity

in a single equivalence class increases, thereby decreasing the sensitive attribute disclosure probability. Thus the upper bound of sensitive attribute disclosure d also decreases. This ensures that adversary’s differential entropy (information gain) is bounded by the prescribed threshold, which minimizes the impact of l-diversity on the sensitive attributes. From these observations, it is clear that if the sensitive attribute distribution is leptokurtic, k-anonymity value when crosses optimum value of l of l-diversity, the resultant k-anonymized l-diverse dataset’s privacy and utility parameters can be sufficiently achieved with k-anonymity alone. This, in effect, reduces the computational complexity to a large extent. The k-anonymized l-diverse dataset has approximate complexity of O(k*l2) [11, 12]. As per our algorithm, when optimum

90

A. Ukil et al.

value of k is larger than optimum value of l, we can achieve same kind of privacy and utility only with k-anonymity, subject to leptokurtic sensitive attribute distribution. This results in substantial decrease of computational overhead: O(k*l2) ? O(k). Though it is to be noted for normal distribution, our hypothesis is not valid. In fact, in platykurtic distribution of sensitive attributes, optimum value of l is very high for significant privacy gain.

7 Conclusion In this paper, we have proposed a privacy preserving algorithm with less publication time which is dependent on the distribution pattern of the sensitive attributes of the database. Leptokurtic distribution of sensitive attributes favours our claim of minimization of computational complexity of the process of privacypreservation of a database. We showed that with leptokurtic distribution of sensitive parameters, k-anonymized l-diverse sanitization is same as only k-anonymized sanitization when k [ l. Through experimental results we have confirmed our hypothesis. Our future work is to extend this algorithm to incorporate t-closeness [9] and to analyze the relationship that t-closeness, k-anonymity and l-diversity holds in presence of different distribution pattern of sensitive attributes.

References 1. Agarwal C, Yu P (2004) A condensation approach to privacy preserving data mining. EDBT 183–199 2. Aggarwal G, Feder T, Kenthapadi K, Motwani R, Panigrahy R, Thomas D, Zhu A (2005) Approximation algorithms for k-Anonymity. J Priv Tech (Nov) 3. Aggarwal CC (2007) On randomization, public information, and the curse of dimensionality. IEEE ICDE conference, pp 136–145 4. Bayardo RJ, Agrawal RJ (2005) Data privacy through optimal k-anonymization. IEEE ICDE conference, pp 217–228 5. Blum A, Dwork C, McSherry F, Nissim K (2005) Practical privacy: the SuLQ framework. ACM PODS conference, pp 128–138 6. Brickell J, Shmatikov V (2008) The cost of privacy: destruction of data-mining utility in anonymized data publishing. ACMKDD 70–78 7. Fung BCM, Wang K, Chen R, Yu PS (2010) Privacy-preserving data publishing: a survey of recent developments. ACM J Comp Surveys (CSUR) 42(4) 8. Li F, Sun J, Papadimitriou S, Mihala G, Stanoi I (2007) Hiding in the crowd: privacy preservation on evolving streams through correlation tracking. IEEE ICDE, pp 686–695 9. Li N, Li T, Venkatasubramanian S (2007) t-Closeness: privacy beyond k-anonymity and l diversity. IEEE ICDE conference, pp 106–115 10. Liu J, Wang K (2010) On optimal anonymization for l-diversity. IEEE ICDE conference, pp 213–224 11. Machanavajjhala A, Kifer A, Gehrke J, Venkitasubramaniam M (2007) L-diversity: privacy beyond k-anonymity. ACM Trans Knowl Disc Data TKDD 1(1)

An Efficient Distribution Sensitive Privacy

91

12. Meyerson A, Williams R (2004) On the complexity of optimal k-anonymity. ACM PODS conference, pp 223–228 13. Nabar S, Marthi S, Kenthapadi K, Mishra N, Motwani R (2006) Towards robustness in query auditing. VLDB conference, pp 151–162 14. Wang H, Liu R (2011) Privacy-preserving publishing microdata with full functional dependencies. Elsevier J Data Knowl Eng 70(3):249–268

DeSPoT: A Method for the Development and Specification of Policies for Trust Negotiation Tormod Håvaldsrud, Birger Møller-Pedersen, Bjørnar Solhaug and Ketil Stølen

Abstract Information systems are ever more connected to the Internet, which gives wide opportunities for interacting with other actors, systems and resources and for exploiting the open and vast marked. This pushes the limits for security mechanisms which in general are too rigorous to fully adapt to such a dynamic and heterogeneous environment. Trust mechanisms can supplement the security mechanisms in this situation to reduce the risk by means of trusted evidences. We propose DeSPoT, a method for the development and specification of policies for trust negotiation. DeSPoT is created to be easy to use for business level experts, yet demonstrated in an industrial study to be useful for those who develop and maintain the system conducting trust negotiation within acceptable risk. Adherence to a DeSPoT policy should ensure that the target fulfills the organizational level requirements to the trust behavior, and that the target is not exposed to unacceptable risk. The paper gives an example-driven presentation of the method. Keywords Trust negotiation control Trust policy





Trust management
Risk management
Access

T. Håvaldsrud (&)
B. Solhaug
K. Stølen SINTEF ICT, Oslo, Norway e-mail: [email protected] B. Solhaug e-mail: [email protected] K. Stølen e-mail: [email protected] T. Håvaldsrud
B. Møller-Pedersen
K. Stølen Department of Informatics, University of Oslo, Oslo, Norway e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_9, Ó Springer Science+Business Media B.V. 2012

93

94

T. Håvaldsrud et al.

1 Introduction Systems at the Internet exploit the potential of the open market and the possibility of interacting with a vast number of other systems for the purpose of realizing opportunities. Trust mechanisms are introduced to mitigate the fact that security mechanisms usually are too rigorous to fully adapt to this dynamic and heterogeneous environment. When a system is exposed in an environment it is subject to risk, which we define as the combination of the likelihood of an incident and its consequence for an asset [1]. Security mechanisms are introduced to reduce the risk, but they are not able to eliminate it entirely. Moreover, increased security tends to be at the cost of interoperability. Security mechanisms should be used to achieve the necessary security level, i.e. keep risk below a certain critical level, and leave it to trust mechanisms to treat the residual risk by means of trust and perceived knowledge. In practice we often need to make assumptions regarding uncertain information, and this forces us to take uncertainty into account when making decisions. Even though information is uncertain it provides important indications of the actual situation. The challenge is to make the uncertainty sufficiently visible so that we are aware of its extent and not just of its existence. In trust mechanisms uncertainty is the focal point of judgment, whereas security mechanisms hide uncertainty by the assumption that it is sufficiently small to be ignored in the clear defined situation. To achieve trust, systems may conduct trust negotiation [2] utilizing trust mechanisms. The systematic use of trust mechanisms may be formulated in a trust policy. The developers need specialized methods to support the development and maintenance of the trust policy in the same way as they need specialized methods for the development and maintenance of security policies. A natural way to describe trust behavior is, as for security behavior, by means of rules. Many security systems do not explicitly define the rules, but rather embed them in the implementation of the system as actions triggered by events. For this reason it is natural to aim for a rule-based policy specification language. The contribution of this paper is a method for the Development and Specification of Policies for Trust negotiation (DeSPoT). Correctly enforcing such a policy should ensure that the trust behavior realizes opportunities while keeping risks at an acceptable level. Focusing on the requirements to and the criteria for risk and trust at an organizational level, the method aims to support decision makers in understanding the potential implications of trust mechanisms without going into the low level details of trust negotiation protocols. This is achieved by systematically linking the high level organizational requirements and criteria to the developed trust policy, which in turn is linked to the low level trust behavior. The rest of the paper is organized as follows. We give an introduction to trust negotiation in Sect. 2. In Sect. 3 we provide an overview of our method. In Sect. 4 through Sect. 8, we present the five steps of our method in an example-driven manner. We conclude in Sect. 9 by characterizing our contribution and discussing

DeSPoT: A Method for the Development and Specification of Policies

95

Fig. 1 Example of trust negotiation

related work. The reader is referred to the full technical report [3] on which this paper is based for further details about the DeSPoT method.

2 Introduction to Trust Negotiation To introduce trust negotiation and our terminology we use an everyday example. Assume a policeman wants to check the validity of David’s driver’s license. The scenario is illustrated by the sequence diagram in Fig. 1. The diagram has three lifelines, David, the Policeman and PoliceITSystem. Everything is observed from policeman’s perspective, so we tag him as the target of our analysis and David and the PoliceITSystem as partners. All events at the policeman’s lifeline are tagged to show what kind of event it is with respect to trust negotiation from the policeman’s perspective. In general, the chosen target of analysis is the system or organization for which the method aims to develop and analyze a policy to govern the trust negotiation. The partners are the trustees in potential interactions with the target. When the policeman approaches David’s car he notices that David is intimidated by his pose and checks that the doors of the car are locked. The policeman knocks on the closed window and makes it clear that he wants verbal contact. David indicates that he wants the policeman to identify himself. These two social messages are revealing parts of their respective trust policies; the policeman reveals some of his intentions and David requests some evidence from the policeman. As a reaction to David’s request the policeman shows his ID card and his badge so that David can see for himself that the policeman really is a police officer and that it is his badge. The badge and the ID card are assets to the policeman, which he exposes to authenticate himself. An asset is something of value for the target and therefore needs protection against risk. David rolls down

96

T. Håvaldsrud et al.

Fig. 2 The five step process

the window and hands over his driver’s license on request from the policeman. David’s driver’s license is a prospect for the policeman; it can tell him whether David is allowed to drive or not. Generally, a prospect is something of value for the target that can be provided by a partner. To verify the validity of the driver’s license, the policeman employs the PoliceITSystem and runs a validity check using the license number. For the policeman in this situation, the license number is an asset and the potential response from the PoliceITSystem is a prospect that may validate David’s driver’s license. After the interaction with the PoliceITsystem the policeman knows that David is allowed to drive a car.

3 Overview of the Method The overview of the five steps of the DeSPoT process is given in Fig. 2. The process provides guidance in correctly capturing the target’s trust policy requirements and the trust policy. The next five sections exemplify each step in turn.

4 Step 1: Characterizing the Target The characterization of the target is conducted in close cooperation with the commissioning party, who is usually the target owner. Our example target is a power grid system where we focus on the balancing of the production and the consumption of electrical power. This task is quite complex and for the purpose of this paper we simplify the system so that we are able to focus on the important features of DeSPoT. The balancing of the production and consumption is conducted in real time, but it also needs planning from day to day. It is this day to day planning on which we focus.

DeSPoT: A Method for the Development and Specification of Policies

97

The selected target is the Power Production Organizer (PPO) which is a central software system collecting and spreading key information about power production and consumption. Several systems communicate with the PPO; some are providing necessary information, some are providing business crucial information and others are just providing supplementary information. A power flow sensor measures the flow of power through a line. It informs the PPO how much power is transported through the power line during the last 24 h. A power station provides the PPO with information on produced power during the last 24 h, as well as the expected production capacity. Sometimes the capacity is lower because of maintenance, while in other situations the power stations may have higher capacity than usual, for example due to heavy rain. In addition to this, the power station needs to know how much power it is expected to feed into the grid the next 24 h to adjust its production. When the PPO has collected all the information, the PPO finds the best suited production profile for the next 24 h. It then assigns production quotas for the next 24 h to all the power stations. Now we continue by defining the scales for prospect value, asset value and trust level from the perspective of the target. These scales may be both quantitative and qualitative, depending on the desired granularity of the analysis or what is otherwise suitable for the target in question. The prospect value scale is used to measure the value of objects and information given to the target by partners. The value should reflect the direct value, the sensitivity and how easy it is to fake. For example, a credential that is very easy to fake should be given a relatively low value, whereas an unforgable message about the delivery of a crate with high quality goods can be given a high value. The prospect value scale chosen for the PPO is of five values from 1 to 5. Each of these values must be defined in terms of a precise interval or a qualitative description. Next we define the scale used to measure the value of assets. As already indicated, an asset is something the target already possesses and that should be protected from harmful incidents. An asset may for instance be sensitive information. Breach of confidentiality, integrity or availability, for example, could be exploited to damage the reputation or revenue of the target. In this situation the potential total loss should correspond to the assigned asset value. The asset value scale chosen for the PPO is of four values from 1 to 4, each of which must be precisely defined. The last scale needed to be defined is the trust level scale. We use this scale to measure the target’s trust in a partner’s capability and intention to protect the target’s assets. The trust level scale chosen for PPO is No, Low, Medium and High, each representing an interval of subjective probabilities between 0 and 1.

5 Step 2: Capturing Requirements for the Trust Policy The trust formation policy requirements specify how the target is allowed to use incoming prospects as evidence to form trust. In general, the trust formation policy requirements restrict how the target is allowed to perceive the world with respect

98 Table 1 The trust formation policy requirements

T. Håvaldsrud et al. Prospect value

Maximal trust level

1 2 3 4 5

Low Medium Medium High High

to trust. The trust formation policy requirements for the PPO are defined in Table 1 and regulate how prospects are allowed to influence the trust level. The interpretation of the second row, for example, is that a prospect with prospect value 2 can at most support a trust level Medium. The asset exposure policy requirements specify the trust level the target must have in order to expose assets with a specific value. The trust level is an indirect measure of the likelihood of something going wrong, and the level of this risk can be deduced from the trust level and asset value. The asset exposure policy requirements hence specify the acceptable risk level in this setting. The asset exposure policy requirements for the PPO are defined in Table 2, where each row forms an asset exposure rule requirement. The interpretation of row three, for example, is that when the target has trust level Medium it is allowed to expose assets with value 3 or lower, while the last row allows the target to expose all assets (because 4 is the highest asset value) to partners in which its trust is High.

6 Step 3: Modeling the Trust Policy The trust formation policy specifies how the target should form trust based on prospects and their properties. The first task is to specify the prospects and their values, as illustrated in Table 3. The third row means that a prospect matching the prospect description Consumer authentication has the prospect value 3. A trust formation rule defines what may form evidence and how this evidence should influence the target’s trust level with respect to a partner. The evidence consists of a prospect, a prospect property and an evidence type. The prospect is received from a partner and can be anything that may give insight into this partner’s properties, such as intention or capabilities to take care of the target’s assets. The prospect property may be as simple as the confirmed existence of the prospect itself, but also complex properties such as authenticity and validity of a chained signed electronic certificate. There are two different sorts of evidence, namely supporting and exposing. The supporting evidence may build trust, whereas exposing evidence on the other hand may reduce trust. The exposing prospect rules are overruling the supporting, such that the trust is governed by the worst exposing evidence and otherwise by the best supporting evidence.

DeSPoT: A Method for the Development and Specification of Policies

99

Table 2 Asset exposure policy requirements

Trust level

Maximal asset value

No Low Medium High

1 2 3 4

Table 3 Prospect values

Prospect description

Prospect value

Partner’s access policy Full postal address Consumer authentication Power certificate signature Master certificate signature Certificate validation

1 2 3 4 5 5

Table 4 Trust formation rules Prospect description

Property

Evidence type

Trust level

Power certificate signature Full postal address Consumer authentication Power certificate signature Master certificate signature Certificate validation

Invalid Existing Valid Valid & Correct Correct Trusted

Exposing Supporting Supporting Supporting Supporting Supporting

No Low Medium High High High

The first row in Table 4 should be understood as follows: If a partner provides an invalid power certificate signature, it is perceived as an exposing evidence and results in a trust level no higher than No. Sometimes a prospect property must be verified by another prospect. This is typically the case in chains of certificates. In order to take into account and keep track of such relations, we document these in a designated table as exemplified in Table 5. The first row should be understood as follows. A Certificate validation prospect with the property Trusted & CertValid verifies the Valid property of a Power certificate signature prospect. Moreover, the Trusted property of the former can in turn be verified by the Correct property of Master certificate signature in the third row. The policy reception rules specify how to handle requests from the partner. These are referred to as such, because in trust negotiation the partners expose their policy when requesting assets from the target. The policy reception rules for the PPO are documented in Table 6. The first row should be understood as follows. If the partner requests that the target system provides a Power certificate signature the target may raise the trust level up to Low based on this evidence.

100

T. Håvaldsrud et al.

Table 5 Prospect property verification Prospect description Required property Certificate validation

Trusted & CertValid

Certificate validation

Trusted & CertInvalid

Master certificate signature

Correct

Prospect description

Verified property

Power certificate signature Power certificate signature Certificate validation

Valid Invalid Trusted

Table 6 Policy reception rules Requested asset

Evidence type

Trust level

Power certificate signature Report power flow Get assigned power quota Report power consumption

Supporting Supporting Supporting Supporting

Low Low Low Low

Table 7 Defining asset values Asset description

Asset value

Get total power consumption Target’s access policy Report consumer consumption Report power production Get assigned power quota Report power production capacity Report power flow

1 2 3 3 3 4 4

The assets identified for PPO are listed in Table 7 together with their respective values. The asset Get total power consumption, for example, is assigned the asset value 1. This is a service provided by the target system and exposes an asset. The asset exposure policy specifies how the target may expose assets based on the trust level. The asset exposure rule for PPO is modeled in Table 8. The first column specifies the minimum trust level for exposing the associated asset. Hence, the asset Power certificate signature can be exposed when the trust level is Low or higher. Or, given the trust level Medium, all the assets of the first five rows can be exposed. PPO also needs to expose parts of its own trust policy. These exposures may be sensitive and for that reason we explicitly model how the target is allowed to expose policies through policy exposure rules. These are presented in Table 9. The first row should be understood as follows: The target must have at least trust level No in the particular partner to be allowed to request the partner for the asset Power certificate signature.

DeSPoT: A Method for the Development and Specification of Policies

101

Table 8 Asset exposure rules

Needed trust

Requested asset

Low Low Low Medium Medium High High

Power certificate signature Report consumer consumption Get total power consumption Report power production Get assigned power quota Report power flow Report power production capacity

Table 9 Policy exposure rules

Needed trust

Asset description token

No No No

Power certificate signature Consumer authentication Postal address

7 Step 4: Analyzing the Current Trust Policy with Respect to its Requirements At this point we have both the trust policy and its requirements. In this step we look for possible gaps between them. Every trust formation rule forms trust based on a prospect with a specific value. It can therefore be easily checked against the corresponding trust formation rule requirement which specifies the highest acceptable trust to be formed for a prospect of this value. Consider, for example, the fourth trust formation rule in Table 4. The prospect description is Power certificate signature. According to Table 3 this prospect has value 5. Further on, evidence formed by the rule supports trust level High. To sum up, the rule supports trust level High based on a prospect of value 5. To check whether this trust formation rule adheres to the trust formation requirements we look into Table 1. This table states that evidences based on prospects with prospect value 4 and 5 can support trust level High. This means that the fourth prospect rule in Table 4 meets the trust formation requirements. The second asset exposure rule in Table 8 assigns access to the service Report consumer consumption and requires at least trust level Low. The service Report consumer consumption has the asset value 3 as shown in Table 7. This rule is then exposing an asset with asset value 3 based on a trust level Low. According to the PPO’s asset exposure policy requirement shown in Table 2, it is not allowed to give access to assets of value above 2 when the trust level is at Low. Hence, this is an example of an asset exposure rule that does not adhere to the asset exposure policy requirements. This is the only breach of adherence in the case of our example.

102

T. Håvaldsrud et al.

8 Step 5: Updating the Trust Policy to Reflect its Requirements The trust policy formulated above allows the consumers to report their power consumption just by giving their postal address, which is not hard to fake. If the asset exposure rule had required trust level Medium instead of Low the customer would be required to log on with their Customer authentication which is quite normal for this kind of service. In that case, adherence with respect to the asset exposure policy requirement would be ensured. This change is implemented by inserting Medium instead of Low in the second row of Table 8.

9 Conclusion We have presented DeSPoT, a method for the development and specification of policies for trust negotiation. The trust policy is linked to risk assessment through the requirements for the trust policy. The trust policy must adhere to the trust policy requirements to delimit the trust behavior within acceptable risk. The method supports negative (exposing) as well as positive (supporting) evidences, sensitive assets (credentials), separation between the trust formation and the asset exposure, static adherence check of the policy with respect to the trust policy requirements, and prospects that verify the properties of other prospects which are the general mechanism behind delegation of trust (recommendation). The method is built around a five step process. Our rule-based approach enables the development of a trust policy the enforcement of which ensures trust negotiations within the limits of acceptable risk. The method is independent from specific trust negotiation protocols, and does not assume such protocols to be predefined. Our focus has been to create an easy-to-understand language for trust policies with few details, nevertheless containing the most important trust mechanisms. The language is made to be understandable for people knowing the target (e.g. a company) at the business level, while being useful and understandable for those that develop and maintain the business application. In this way we are able to assemble a trust policy that contains both the risk and asset knowledge from the business level and the technical knowledge about different security technologies in one trust policy. Such a combination may reveal inconsistencies in the perception of the trust domain internally in the company. The approach to system authentication presented in [4] implements trust negotiation as described in [5] to build trust. Approaches like Trust-X [5], [6], [7], TrustBuilder [8], [9] and Protune [10], [11], [12] are examples of policy based access control systems for automated trust negotiation. While these approaches make use of mechanisms for building trust, the involved trust level is only implicit and not extracted as an explicit element of value in itself. Yao et al. presents in

DeSPoT: A Method for the Development and Specification of Policies

103

[13] a value and privacy scoring for credentials and find an optimal exposure with minimal privacy and sufficient value to achieve access. The automated trust negotiation proposed in [14] emphasizes that negative evidences cannot be supported because an agent only controls what it sends and not what it receives and therefore opens for Denial of Service (DoS) attacks. For this reason there is very few that support negative evidence. We believe, however, that negative trust evidence is important in trust management. Revocation of certificates, banning of accounts and blocking of credit cards are examples of activities based on new information resulting in lower trust. Hence, not all digital trust functions are non-decreasing. In our approach every use of evidence is based on the trust in the partner providing it. To conduct a DoS attack through negative evidence, one must exploit misplaced trust and be able to pose as a trusted communication partner. If this posing is possible, then the trust in the evidence is overrated. In a trust policy as well as in a security policy the risk may be underrated as this is a possibility in all risk analysis. The vital thing is to be aware of this fact, and try to avoid it. Acknowledgments The research on which this paper reports has partly been funded by the Research Council of Norway through the DIGIT (180052/S10) project, and partly by the European Commission through the NESSoS network of excellence.

References 1. International Organization for Standardization (2009) ISO 31000 Risk management— principles and guidelines 2. Winslett M (2003) An introduction to trust negotiation. In: iTrust 2003, LNCS vol 2692. Springer pp 275–283 3. Håvaldsrud T, Møller-Pedersen B, Solhaug B, Stølen K (2011) DeSPoT: A method for the development and specification of policies for trust management. Technical report A20174, SINTEF 4. Seigneur J-M, Farrell S, Jensen CD, Gray E, Yong Chen Y (2004) End-to-end trust starts with recognition. In: Secur Pervasive Comput LNCS 2802:130–142 5. Bertino E, Ferrari E, Squicciarini A (2004) Trust negotiations: concepts, systems, and languages. Comput Sci Eng 6:27–34 6. Bertino E, Ferrari E, Squicciarini A (2004) Trust-X: a peer-to-peer framework for trust establishment. IEEE Trans Knowl Data Eng 16(7):827–842 7. Squicciarini A, Bertino E, Ferrari E, Paci F, Thuraisingham B (2007) PP-trust-X: A system for privacy preserving trust negotiations. ACM Trans. Inf Syst Secur 10 8. Lee A, Winslett M, Perano K (2009) TrustBuilder2: A reconfigurable framework for trust negotiation. In: Trust management III, IFIP, Advances in information and communication technology, vol 300. Springer, pp 176–195 9. Winslett M, Yu T, Seamons KE, Hess A, Jacobson J, Jarvis R, Smith B, Yu L (2002) Negotiating trust in the web. Internet Comput IEEE 6(6):30–37 10. Bonatti P, De Coi JL, Olmedilla D, Sauro L (2010) A rule-based trust negotiation system. IEEE Trans Knowl Data Eng 22:1507–1520 11. Bonatti P, Olmedilla D (2005) Driving and monitoring provisional trust negotiation with metapolicies. In: Proceedings of the sixth IEEE international workshop on policies for distributed systems and networks (POLICY’05), IEEE Computer Society, pp 14–23

104

T. Håvaldsrud et al.

12. De Coi JL, Olmedilla D, Zerr S, Bonatti P, Sauro L (2008) A trust management package for policy-driven protection and personalization of web content. In: Proceedings of the 2008 IEEE workshop on policies for distributed systems and networks (POLICY’08), IEEE Computer Society, Washington, DC, pp 228–230 13. Yao D, Frikken KB, Atallah MJ, Tamassia R (2008) Private information: To reveal or not to reveal. ACM Trans Inform Syst Secur TISSEC 12(1):61–627 14. Winsborough WH, Seamons KE, Jones VE (2000) Automated trust negotiation. DARPA Inform Surviv Conf Expo 1:88–102

Extended NIST Prime Family for Efficient Modular Reduction Young In Cho, Nam Su Chang, Chang Han Kim and Seokhie Hong

Abstract The FIPS 186-2 standard recommends five prime fields with a modulus so-called NIST primes for elliptic curve cryptosystems. Primes of the special form such as NIST primes have a property yields modular reduction algorithms that are significantly fast. However the number of NIST primes are not large enough. In this paper, we further extend the idea of NIST primes. Then we find more primes can provide fast modular reduction computation that NIST prime family does not support. Our method provides more efficient modular arithmetic than Montgomery algorithm in prime fields that NIST primes does not support. Keywords Modular arithmetic

 NIST prime  Finite field arithmetic

Y. I. Cho  S. Hong (&) Graduate School of Information Management and Security, Korea University, Seoul, South Korea e-mail: [email protected] Y. I. Cho e-mail: [email protected] N. S. Chang Department of Information Security Systems, Sejong Cyber University, Seoul, South Korea e-mail: [email protected] C. H. Kim Department of Information and Security, Semyung University, Jecheon, South Korea e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_10,  Springer Science+Business Media B.V. 2012

105

106

Y. I. Cho et al.

1 Introduction Efficient implementation of modular arithmetic is very important for the high performance of many public-key cryptosystems such as RSA, elliptic curve cryptosystems and pairing based cryptosystems based on prime fields. There are many well-known techniques for efficient implementation of modular arithmetic. For general primes, the most efficient exponentiation algorithm is Montgomery exponentiation algorithm [3]. Montgomery exponentiation algorithm uses a special representation to perform efficient arithmetic, the division and remaindering essentially being performed by bit shifting. If a modulus p has a special form, modular arithmetic can be accelerated significantly. The Mersenne numbers of the form 2m  1 are well-known examples [1]. However, unfortunately, only a few Mersenne primes are useful for cryptography. Solinas introduced the family of generalized Mersenne numbers [4]. They are expressed as p ¼ f ðtÞ, where t is a power of 2. This representation permits fast modular reduction without division operation. NIST prime family is the special case of generalized Mersenne numbers. NIST prime family has the property that can be written as the sum or difference of a small number of powers of 2. Furthermore, the exponent appearing in these expressions are all multiples of 32. This property yields modular reduction algorithms that are especially fast on machines with wordsize 32. The FIPS 186-2 standard recommends five primes so-called NIST primes : p192 ¼ 2192  264  1; p224 ¼ 2224  296 þ 1; p256 ¼ 2256  2224 þ 2192 þ 296  1; p384 ¼ 2384  2128  296 þ 232  1; p521 ¼ 2521  1: The powers appearing in NIST primes are all multiples of 32 except for p521. In this paper, we find more primes can support fast modular reduction by further extending the idea of NIST prime family. Instead of considering modulus with a special form, we try to discover the integer p belongs to Extended NIST Primes (ENP) that provide almost same performance with NIST primes. We show that how to perform modular arithmetic operation with a modulus ENP. Then we present the running time of modular multiplication and exponentiation with a modulus ENP comparing with Montgomery algorithm. ENP support more prime fields that permit very fast modular reduction than NIST prime family does.

Extended NIST Prime Family

107

2 Extended NIST Primes In this section, we introduce ENP and discuss its computational efficiency. We denote the number of powers of 2 appearing in p as wt(p). For example, when p ¼ 2a þ 2b þ 1 ð0\b\aÞ; wtðpÞ ¼ 3: Definition 1 (ENP) A prime p is called Extended NIST Prime (ENP) if there exist a generalized Mersenne number q ¼ p  h for some positive odd integer h. In this case, exponents of 2 appearing in q should be all multiples of wordsize except for wt(q) = 2. In other words, we try to find positive integers of the form NIST prime family which have a large prime factor. Then the large prime factor would be ENP. We devote subsection 2.1 to the description of modular arithmetic with a modulus ENP.

2.1 Modular Arithmetic Modular Reduction : NIST primes have a property that modular reduction algorithms are very fast on machines with wordsize 32. Note that we can apply the same modular reduction rule to modular reduction operation with a modulus q of Definition 1 since q is a form of NIST prime family. In order to compute c modulo p for some integer c, we reduce the result of c modulus q. For example, consider ENP p = p285 of bitsize 285 when q = 2288 + 296 - 1 and P 32 32i h = 11. Let c be an integer with 0  c  p2285 . Let c ¼ 17 i¼0 ci 2 be the base-2 32 representation of c, where each ci 2 ½0; 2  1. We reduce c according to the following congruences : 2288 2352 2416 2480 2544

 296 þ 1 ðmod qÞ;  2160 þ 264 ðmod qÞ;  2224 þ 2128 ðmod qÞ;  2192 þ 296  1 ðmod qÞ;  2256 þ 2160  264 ðmod qÞ:

2320 2384 2448 2512

 2128 þ 232 ðmod qÞ;  2192 þ 296 ðmod qÞ;  2256 þ 2160 ðmod qÞ;  2224 þ 2128  232 ðmod qÞ;

Thus we obtain c



c17 2256

þc17 2160 c16 2224

c17 264 þc16 2128

c15 2192 c14 2256

c16 232 þc15 296

c15

þc14 2160 c13 2224

þc13 2128 c12 2192

þc12 296 c11 2160

þc11 264 c10 2128

c8 2256

þc7 2224

þc6 2192

þc5 2160

þc4 2128

þc10 232 c9 296 þc3 296

þc2 264

þc1 232

þc9 þc0

ðmod qÞ:

108

Y. I. Cho et al.

Hence, c modulo p can be obtained by adding ten 288-bit integers, c172256+c172160 -c17264, c162224+c162128-c16232, c152192+c15296-c15, -c142256+c142160, c132224+c132128, -c122192+c12296, -c112160+c11264, -c102128+c10232, -c9296+c9 and c82256+c72224+c62192+c52160+c42128+c3296+c2264+c1232+c0 and subtracting p repeatedly until the result is less than p. Algorithm 1 is the modular reduction algorithm with a modulus ENP p285. Algorithm 1 Fast reduction modulo P285 Input: q = 2288 + 296 - 1, an integer c = (c17,…,co) in base 232 with 0  c \p2285 and a modulus ENP p285. Output: c mod P285 1: Define 288-bit integers : S1 = (C8,C7,C6,C5,C4,C3,C2,C1,C0) S2 = (-Cl4, -C13, - C12, - C11, - C10, - C9, 0, 0, 0), S3 = (0,0,0,C14,C13,C12,C11,C10,C9), S4 = (Cl7, C16, C15, C17, C16, C15, - C17 -Cl6 -C15). 2: Return (s1 + s2 + S3 + s4 mod p285). Algorithm 2 Modular multiplication with a modulus ENP Input: two positive integers A = (an-1,  a0)b, B = (bn-1, b0)b and a modulus ENP where b = 232. Output: S = (Sn-1,, S0)b such that S = AB mod p 1: Compute A B 2: Compute U = A B mod q 3: Compute S = U mod p (with rights shift and subtractions) Modular Multiplication : Modular multiplication with a modulus ENP for some integer A and B is performed in three steps. We know that modular reduction in step 2 of Algorithm 2 is very simple like the reduction with a modulus NIST prime family. Comparing modular multiplication with a modulus NIST prime, the extra operation is only step 3. Since we assume that the modular multiplication is implemented in software, we analyze the computational efficiency in terms of word length. Let tm and td respectively denote the time required for a microprocessor to compute word level multiplication and division. According to [2], the running times required to compute long integer multiplication, squaring and division are given as follows: MðnÞ ¼ n2 tm ; SðnÞ ¼ ðn2 þ nÞ=2tm ; Dðn1 ; n2 Þ ¼ ðn1  n2 Þðn2 þ 3Þtm þ ðn1  n2 Þtd ;

Extended NIST Prime Family Table 1 Comparison of modular multiplication running time

109 Modular multiplication

Computational efficiency

Classical Montgomery ENP

ð2n2 þ 3nÞtm þ ntd ð4n2 þ 4nÞtm þ ntd n 2 tm

where M(n) and S(n) are the running times for a long integer multiplication and a long integer squaring respectively, when multiplicands are both n words long. D(n1, n2) is the running time for a long integer division when the dividend is n1 words long and the divisor is n2 words long. For simplicity, we do not consider the time required for additions and subtractions in our analysis. We compare the computational efficiency of the modular multiplication in Table 1. Modular Exponentiation: ENP is especially useful for modular exponentiation since mod p extra operation spends a small part of a whole operation. We use left-to-right binary exponentiation algorithm. In algorithm 3, step 4 and 7 are computed with word additions. Now we restrict the range of q to gain better performance. If q occupies more word blocks than p, then the results of mod q operations in step 4 and 7 may also occupy more word blocks than p does. Algorithm 3 Modular Exponentiation Input: g = (gngn-1.g1g0)b and a positive integer e = (etet-1…e1e0)2. Output: ge mod p. 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11:

A/1 for i = t to 0 do A/AA A / A mod q if ei -1 then A/Ag A / A mod q end if end for A / A mod p Return (A)

So the computational complexity of multiplication in step 3 and 6 may be higher than the complexity of multiplication using the results of mod p operations in iteration. Moreover if h is too big, then modular reduction in step 10 may be expensive. The following remark presents our restriction of the range of q and h for better performance. Remark 1 In order to perform modular arithmetic, we consider q occupying the same number of word blocks with p. And h occupies half of the wordsize.

110

Y. I. Cho et al.

Table 2 Comparison of modular exponentiation running time Modular exponentiation Computational efficiency ð3=2n2 þ 1120tÞtm þ ð32tÞtd 3nðn þ 1Þðt þ 1Þtm ð3=2n2 tÞtm

Classical Montgomery ENP

Extra computation – Precomputation Right shift and Subtraction

Table 3 Comparison of the average running time of modular arithmetic operation Modular multiplication (ls) Modular exponentiation (ls) Classical Montgomery ENP

wt(q) = 2

wt(q) = 5

wt(q) = 2

wt(q) = 5

6 11 2

19 33 6

3992 1097 1042

37510 12612 10338

* The bitsizes of p are 700 where wt(q) = 2 and 1649 where wt(q) = 5

If squaring is approximately as costly as an arbitrary multiplication then the expected amount of work is represented in Table 2.

2.2 Experimental Comparisons In this subsection, we compare practical performance of modular arithmetic with a modulus ENP and NIST prime family through experimental result. We find 32*2048 bit primes belong to ENP family with wt(q) = 2*5 and restrict the range of q and h (recall remark 1). We select two ENPs with respectively wt(q) = 2 and wt(q) = 5. Two ENPs are p700 where q = 2702- 1, h = 3 and p1649 where q = 21664 ? 21184 ? 2896-2288 - 1, h = 65531. We generate 700 and 1649 bit random prime numbers for classical and Montgomery algorithm. We compare the running time of modular arithmetic operation with a modulus each bit prime for two random numbers in Table 3. We performed modular arithmetic operation a million times and obtained the average running time for each algorithm. We used OpenSSL crypto library which has been used for implementing of a wide range of cryptographic algorithms. The target architecture was assumed to have a 32-bit word length, the actual timings being implemented on Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz processor. We know that ENP family can provide quite fast modular arithmetic operation in spite of extra operation from Table 3.

3 The Number of ENP Our contribution is the provision of primes satisfying two properties below. • Primes of bitsize that NIST prime family can not support • Primes have the merit of modular reduction as NIST prime family

Extended NIST Prime Family

111

Table 4 Comparison of the number of primes Primes\The number of powers of 2 2

3

4

5

Prime fieldsà

NIST prime family ENP family 

45 91

472 911

23119 88799

130 1006

11 84

  : The number of ENP according to corresponding wt(q) à : The number of fields can be applicable

Thus the number of ENP with different bitsize is crucial to our work. As previously stated, we restricted the range of q and h. Table 4 presents the number of 32*2048 bit ENP with wt(q) = 2*5. Comparing NIST prime family, there exist relatively many ENP and they can be used for prime fields with various size.

4 Conclusion In this paper we defined ENP family which is a new family of moduli suitable for cryptography by further extending the idea of NIST prime family. ENP can provide fast modular reduction in prime fields that NIST prime family can not support. We explained how to perform modular arithmetic with a modulus ENP, and we explicitly provided experimental result. We can expect that ENP are very useful for elliptic curve cryptosystems based on prime field.

References 1. Crandall R (1992) Method and apparatus for public key exchange in a cryptographic system. US Patent, 5,159,632 2. Menezes AJ, Oorschot PC, Vanstone SA (1997) Handbook of applied cryptography. CRC Press 3. Montgomery PL (1985) Modular multiplication without trial division. Math Comput 44(170):519–521 4. Solinas J (1999) Generalized mersenne numbers. Technical report CORR 99-39, center for applied cryptographic research, University of Waterloo, Waterloo

High Attack Cost: Hash Based RFID Tag Mutual Authentication Protocol Jung-Sik Cho and James J. (Jong Hyuk) Park

Abstract Radio Frequency Identification (RFID) systems are expected to replace barcode systems in object identification fields in the future, but various security problems are obstructing their diffusion. This paper proposes hash-based RFID tag mutual authentication protocol to solve these security problems of RFID systems. The proposed protocol is designed to demand high cost of acquiring the tag information from attackers. Keywords Hash

 Mutual authentication  RFID

1 Introduction Radio Frequency Identification (RFID) systems are automatic identification systems using radio frequencies, and consist of RFID tags, RFID readers, and a back-end server [1]. Tags store the unique identification information of objects and are attached to the objects to be identified. Reader requests unique identification information from a tag using a radio frequency and the tag returns the stored identification information using the radio frequency sent from the reader. The reader then sends the identification information received from the tag to the back-end server. The back-end server manages in its database (DB) the information of objects with attached tags and

J.-S. Cho Chung-Ang University, Seoul, Korea e-mail: [email protected] J. J. Park (&) Seoul National University of Science and Technology, Seoul, Korea e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_11,  Springer Science+Business Media B.V. 2012

113

114

J.-S. Cho and J. J. (Jong Hyuk) Park

the unique identification information of the objects, and supplies the information of the objects when a reader requests it [1, 2]. Typical security problems that RFID systems must solve are privacy infringement and forgery. These security problems originate from the objects (tags, readers, back-end server) of the RFID system and information leakage during the communication process [3]. Many studies are being conducted to address these security problems of the RFID systems, which are largely divided into studies on the blocking of tag identification itself [3] and studies on the use of tag authentication protocols [3–6]. Blocking tag identification itself presents the potential loss of usefulness and management efficiency and may be abused by attackers. On the other hand, the tag authentication protocols allow the components of the RFID system to authenticate tags by processing and exchanging information in accordance with the defined procedure through cryptographic primitives. As only the valid components can generate and confirm information, the participation of invalid components is restricted and information leakage is prevented. This technique is most actively studied in diverse directions because it enables many variations. This paper reports on a study about a tag authentication technique using hash functions. There have been many difficulties in applying hash functions to lowpriced tags, but as with many other studies on hash-based techniques, this paper also assumes that hash functions can be applied to tags in the near future.

2 RFID System Security and Related Studies The primary reason that RFID systems cause security problems is the limited resources of tags. Tags do not have their own power source, and consist of a small microchip and an antenna. When a reader requests it in accordance with a predefined procedure, the tag returns the stored unique identification information to the reader. The tag generates power using the radio frequency delivered through the reader’s request and performs limited operations before returning the unique identification information stored in the microchip to the reader. In this process, the tag information is leaked by various attacks by attackers [4]. Types of attacks by these attackers include attacks for privacy infringement such as wiretapping, analysis of communication content, and location tracking, and attacks for tag forgery such as replay attack, spoofing attack, and physical attack [6]. Many techniques for security of RFID systems are being proposed, and the security level of these techniques is evaluated by measuring the satisfaction of security requirements in general. The security requirements mentioned here are defined as confidentiality, indistinguishability, forward security, and mutual authentication [3, 4]. Among the RFID security techniques, hash-based techniques offer a high level of security. However, they are regarded as impractical because as the security level becomes higher, the implementation and performance of tags get more unrealistic and the efficiency of back-end server drops. As a typical example, the technique

High Attack Cost: Hash Based RFID

115

proposed by Ohkubo [7] has high a level of security, but the cost of searching tags by the back-end server is very inefficient. On the other hand, the hash lock technique [8] is sufficiently practical in the possibility of tag implementation and the efficiency of back-end server, but it has many weaknesses in security. This technique is particularly vulnerable to indistinguishability. Many modifications of these two techniques have been proposed later. However, they share common problems in their basic forms. When attackers make specific formats of requests or meaningless requests to tags, the tags make the same response or their information is easily leaked [8]. Synchronization is another problem that needs to be solved, but techniques addressing this problem also show security vulnerability [9, 10]. It seems that this problem originates from the usage of random numbers which are used to satisfy the security requirements of RFID systems [6]. Therefore, this paper proposes a technique to solve this problem.

3 Proposed Technique Most RFID tag authentication techniques use random numbers for authentication by readers and protection of tag information. These random numbers, however, are easily exposed because they are transmitted through radio frequencies without any processing. Thus, attackers can obtain them with simple means such as wiretapping and use them to obtain information that they want. To address this problem, this paper proposes a technique to prevent the exposure of tag information by processing the random numbers of readers and tags.

3.1 Notations and Arithmetic Operations The notations used in the proposed technique are defined as shown in Table 1 below. The size of each value is assumed to be 96 bits. To process random numbers, the proposed technique presents the following Eq. 1: Fis ðRÞ ¼ ðR  R mod s þ 1Þð0 : 47ÞjjðR þ s  R mod sÞð48 : 95Þ ¼ RIDi

ð1Þ

The argument in Eq. 1 is a random number. The random numbers are grouped on the basis of the secret value S, and the MSB 48 bit of the minimum value and the LSB 48 bit of the maximum value of each group are combined. The role of the secret value s is to define the range of each group when random numbers, which have been entered as argument, are grouped. The secret value s can be equal for every tag. That is, it is not unique for each tag. Tags have two secret values: St and Sr . The secret values used in Eq. 1 are determined by the following Eqs. 2 and 3 depending on the random numbers that are entered as argument.

116

J.-S. Cho and J. J. (Jong Hyuk) Park

Table 1 Notations Notations R IDk , DATAk S h(), a Fyx ðzÞ, RID

Descriptions This means a random number. Rr is a random number generated by reader, and Rt is the random number generated by tag k ID is the unique identification information (ID) stored in tag k. DATAk is the information of the object attached with the tag k This is the secret value shared by back-end server and tags. Each tag has two secret values: Ss ; St h() is the hash function and a is the hashed value Fyx ðzÞis a function that has z as an argument and performs operation based on x, y is the communication session information. RID is the result value

r

Fis ðRr Þ ¼ RIDri t

Fis ðRt Þ ¼ RIDti

ð2Þ ð3Þ

Secret values do not have ‘0’ and ‘1’. This is to consider the grouping of random numbers.

3.2 Proposed Protocol The proposed technique proposes the protocol as shown in Fig. 1 below. Each step of this protocol is described below. Phase 0: Information sharing between back-end server and tags – The back-end server stores and manages the following tag information in DB: – – – –

DATA: Information of the tagged object. ID : Unique identification information of the tag Sti ; Sri : Secret values currently shared by tag and back-end server Sti1 ; Sri1 : Secret values of the previous session shared by tag and back-end server

Phase 1: Reader’s request – The reader generates a random number (Rr ) in the session and requests tag information using this number. Phase 2: Generation of response message – The tag generates a random number (Rt ) and also generates RIDti using Eq. 3. – The response message is created using the hash function as shown in the following Eq. 4:

High Attack Cost: Hash Based RFID

117

Fig. 1 Proposed protocol

a ¼ hðIDk  Rt  Rr  RIDti Þ

ð4Þ

Phase 3 and 4: Transmission of response – The tag sends the response message (a) generated from Eq. 4 and the random number (Rt ) to the reader. – The reader sends the response message received from tag and the random number generated in the current session to the back-end server. Phase 5: Tag authentication and secret values update – Back-end server searches the tag in the DB based on the information received from reader. – When the tag is found, the back-end server updates the secret values of the tag. The secret values of this session are stored in Sti1 and Sri1 , and the newly updated secret values are stored in Sti and Sri . – If the back-end server fails to find the tag, it searches again using the secret values (Sti1 ,Sri1 ) of the previous session. If no tag is found, it is judged as abnormal authentication message and the session is terminated. – When the tag searching, authentication, and update of secret values are completed, a message is generated with the following Eq. 5:

118

J.-S. Cho and J. J. (Jong Hyuk) Park

DATAk jjhðRIDri  RIDti ÞjjRIDri  Stiþ1 jjRIDti  Sriþ1

ð5Þ

– The components of Eq. 5 are as follows: – Information of the tagged object (DATAk ) to be sent to the reader. – A message to provide authentication of the back-end server to the tag (hðRIDri  RIDti Þ) – A message to provide authentication of the back-end server to the tag (hðRIDri  RIDti Þ) – A message to safely deliver the newly updated secret values to the tag (RIDri  Stiþ1 jjRIDti  Sriþ1 ) Phase 6 and 7: Delivery of message generated by back-end server – The back-end server delivers the message generated through Eq. 5 to the reader. – The reader acquires the information of the tagged object from the message received from the back-end server and sends the remaining message to the tag. Phase 8: Back-end server authentication and secret values update – The tag authenticates the back-end server based on the message received from the reader. – The tag generates RIDri by calculating Eq. 2 with the random number (Rr ) received from the reader. – The tag performs hash using RIDti and RIDri that have been generated. – The tag authenticates the back-end server by confirming that the hashed value is identical to hðRIDri  RIDti Þ that it received from the reader. – When the back-end server authentication is completed, it expels the updated secret values from RIDri  Stiþ1 jjRIDti  Sriþ1 and updates the secret values. – If back-end server authentication fails, the secret values are not updated.

4 Analysis This section analyzes the security level of the proposed technique. Attackers can perform wiretapping to the proposed technique. Information leakage through wiretapping occurs when reader and tag exchange messages through a radio frequency in phases 1, 3, and 7. However, attackers cannot get any tag information just by wiretapping. Even with wiretapping in multiple sessions, attackers cannot get any information of tags and cannot even identify them, because tag information is processed through hash operation, random numbers, and Eq. 1.

High Attack Cost: Hash Based RFID

119

Now we will calculate the cost of attack when the attacker performs an analysis attack of the communication content to acquire tag information from the message obtained through wiretapping. If the attacker obtained a; Rr ; Rt by wiretapping in phase 1 and 3, he will try to estimate the ID and secret values of the tag. At this time, the attacker will perform a brute-force attack and the cost has the complexity of 2192 . If the attacker additionally performed wiretapping in phase 7, the cost of finding the two secret values also has the complexity of 2192 . This is a very high cost compared to the highest cost 296 when the attacker performs a brute-force attack to the techniques of the existing studies. By requiring a high cost of acquiring tag information from attackers, the proposed technique satisfies all the security requirements for RFID systems as described below. • Confidentiality: The proposed technique protects the ID of tags based on hash operation, and the attacker must find the secret values to get it but cannot find them due to high complexity. • Indistinguishability: The proposed technique uses random numbers and grouping of them to make it impossible to predict the message generated by tags in each session. • Forward security: To ensure forward security, the proposed technique is designed in such a way that when the tag information is updated, the back-end server will generate it randomly. • Mutual authentication: The proposed technique provides authentication between back-end server and tags based on secret values, and provides reader authentication based on random numbers generated by reader. The proposed technique solves the user privacy infringement and forgery problems by satisfying all the security requirements as shown above.

5 Conclusion The RFID system is a low-priced non-contact automatic identification technology. Due to the most basic characteristic of using radio frequency with small, chip tags, however, RFID systems has user privacy infringement and forgery issues. To solve these problems, the present paper proposed a hash-based RFID tag mutual authentication protocol. The basic feature of the proposed protocol is grouping of random numbers to address the problem of the exposure of random numbers which are used to protect tag information and provide anonymity, resulting in acquisition of the random numbers by attackers through wiretapping or other means to access tag information. As a result, the proposed technique demands a high cost of brute-force attack from attackers.

120

J.-S. Cho and J. J. (Jong Hyuk) Park

The proposed technique solves the user privacy infringement and forgery problems by satisfying all the security requirements for RFID systems. Future studies will need to find ways to provide higher efficiency of RFID systems. Acknowledgments This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology(2011-0024052).

References 1. Finkenzeller K (2002) Fundamentals and applications in contactless smart cards and identification RFID handbook, 2nd edn. Wiley, New York 2. EPC global Inc (2008) EPC Radio-frequency identity protocols class-1 generation-2 UHF RFID protocol for communications at 860 MHz–960 MHz Version 1.2.0. 3. Juels A (2006) RFID security and privacy: a research survey. Sel Areas Commun 24(2):381– 394 4. Yeo S, Kim S (2005) Scalable and flexible privacy protection scheme for RFID systems. In: European workshop on security and privacy in Ad hoc and Sensor Networks—ESAS’05, Lecture Notes in Computer Science, vol 3813. Springer, Heidelberg, pp 153–163 5. Dimitriou T (2005) A lightweight RFID protocol to protect against traceability and cloning attack. In: Proceedings of SECURECOMM’05 6. Cho J-S, Kim S-C, Yeo S-S (2011) RFID system security analysis, response strategies and research directions. In: Ninth IEEE international symposium on parallel and distributed processing with applications workshops, IEEE Computer Society, pp 371–376 7. Ohkubo M, Suzuki K, Kinoshita S (2003) Cryptographic approach to privacy-friendly tag. In: P RFID Privacy Workshop, MIT, Cambridge 8. Weis SA, Sarma SE, Rivest RL, Engels DW (2003) Security and privacy aspects of low-cost radio frequency identification systems. Security in prevasive computing. LNCS 2802:201– 212 9. Liang B (2010) Security and performance analysis for rfid protocols. dissertations and theses collection, paper 52, http://ink.library.smu.edu.sg/etd_coll/52 10. Vahedi E, Ward R, Blake I (2011) Security analysis and complexity comparison of some recent lightweigth RFID protocols. Computational intelligence in security for information systems

Energy-Efficient Multi-Granularity Resource Reservations for Multimedia Services Joohyung Sun and Hyeonjoong Cho

Abstract This work presents the scheduling algorithms to reduce the energy consumption of hard real-time tasks with fixed priorities assigned by a rate monotonic policy and to guarantee that all the I-frames of each video stream meet their deadlines. The proposed scheduling algorithms are called low-power work/ limit demand analysis with multi-granularity (lpWDA-MG/lpLDA-MG) and it is an attempt to reduce energy consumption by incorporating the DVS technique while simultaneously supporting multimedia such as MPEG video stream services. We present results to show several features of the proposed algorithms and show the experimental results by comparing with the existing methods that support MPEG4 streams. The experimental results show that the proposed algorithms tremendously reduce energy consumption by 2–50%.







Keywords Dynamic voltage scaling Low power Multi-granularity Resource reservation Real-time scheduling



1 Introduction Operating System (OS) resource management for real-time and multimedia systems has been an active research area. Resource reservations based on worst-case requirements over-estimates the desired resource requirements of soft real-time

J. Sun (&)  H. Cho Computer and Information, Korea University, Jochiwon-eup, Yeongi-gun, Chungnam 339-700, South Korea e-mail: [email protected] H. Cho e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_12, Ó Springer Science+Business Media B.V. 2012

121

122

J. Sun and H. Cho

multimedia applications, which generally have the following characteristics: (a) highly varying resource consumption rate, (b) large peak-to-average ratio of resource demand, (c) long burst of large requests, (d) tolerance to occasional deadline misses. As a response, Saewong [1] proposed a Multi-Granularity reservation (MG) model. The scheme extends a dimension of the traditional resource reservation such that more detailed Quality of Service (QoS) requirements can be deterministically specified, analyzed and guaranteed. The proposed admission control allocates only the resource budget sufficient to satisfy the given QoS specification, efficiently manages resources across frames to handle fluctuating demands and still maintains the QoS isolation from other reserves and non-real-time tasks. In the meantime, real-time scheduling plays a key role in the low-power design of real-time embedded systems, not only because timing issues are critical, but also because achieving a low-power design is essentially a resource-usage optimization problem. Many scheduling methods have been published (e.g., [2] by Pillai, [3] by Kim, [3], and [4] by Chen). The core idea of these methods is to employ scheduling techniques that can exploit the modern dynamic configuration capabilities of embedded processors, according to the current or expected workload, to achieve energy efficiency. One such capability is dynamic voltage scaling (DVS). To guarantee the timing requirements of real-time tasks even in the worst-case, DVS can utilize only slack times (or idle times) when adjusting voltage levels. The energy efficiency of a real-time DVS algorithm largely depends on how accurately the slack time of each task is estimated. Normally DVS only gets slack from a processor’s idle time or tasks’ early completion. However the MG model can obtain the additional slack times created when the real-time task’s reserve is depleted. Specifically, the amount of depleted task’s workload on each period can be used as the additional slack, which improves the efficiency of the energy savings. In this paper, we propose the real-time DVS scheduling algorithms called low-power work/limit demand analysis with Multi-Granularity reservation model (lpWDA/lpLDA-MG), which are designed to reduce energy consumption by using computed slack while considering the given MG model for MPEG4 streams. The experimental results indicate that the rate monotonic (RM) DVS algorithms combined with the proposed method can reduce energy consumption by 2–50% compared with those of existing algorithms such as multi-granularity reservation (MULTI-RSV). Next, Sect. 2 summarizes related works. Section 2 explains several models for scheduling and presents a motivational example. Section 3 describes the proposed algorithm. Section 4 provides theorems to show the schedulability of the algorithm. Section 5 presents the experimental results. Section 6 concludes the paper.

Energy-Efficient Multi-Granularity Resource

123

2 Models and Objective 2.1 System and Energy Model The multi-granular reserve specification is given by fC; T; Dg; fC x ; ex T g; . . .; fC y ; ey T g; where ex \. . .\ey and 8i; ei 2 Zþ; each tuples specified in [1]. At each granular timer interval (e.g. ex TÞ; a corresponding reserve budget (e.g. Cx ) will be refilled. If the task executes more than the granted amount, then the task’s priority is downgraded to the same level as (or below) that of other nonreal-tasks. This paper focuses on a preemptive hard real-time system in which periodic real-time tasks are scheduled under the RM scheduling policy, where the shorter period task has the higher priority. We suppose that the target processor can continuously scale its driving voltage and clock speed within its operational ranges, ½Vmin ; Vmax  and ½fmin ; fmax ; respectively. A set of n periodic tasks is denoted by T ¼ fs1 ; s2 ; . . .; sn g; where tasks are assumed to be mutually independent. si has a shorter period (i.e., a higher priority) than si if i \ j. Each task si has its own priority prioi , period pi ; worst-case execution time (WCET) wci , and average-case execution time (ACET) aci . The relative deadline di of si is assumed to be equal to its period pi : A task instance is denoted by a single subscript such as sa when no confusion arises. During run-time, we refer to the earliest job of each task that has not completed execution as the current job for that task, and we index that job with cur, for example, Jicur is the current job for task si : We consider Martin’s system-level energy consumption model to derive the energy consumption at a given CPU speed [5]. In this model, when operating at a frequency f, the amount of power that a component consumes is called a component’s dynamic power consumption and denoted by Pd : Pd of CPU is given as S3  f 3 ; where S3 is a constant. In this paper, we only use the traditional energy model. Summing the power consumption of all components, the normalized CPU speed at frequency fi is Si ¼ fm =f : Thus, the energy consumption at a given CPU speed Si is: EðSi Þ ¼ ci  ðC3  S2i þ C2  Si þ C1 þ C0 =Si Þ where ci denotes si ‘s expected execution time at the maximum frequency fm ; C3 ¼ S3  fm3 C2 ¼ S2  fm2 ; C1 ¼ S1  fm ; and C0 ¼ S0 are system-dependent parameters. It is observed that EðSi Þ is a convex function. By Descartes’ Rule of Signs, there is only one value, denoted Slow that minimizes EðSi Þ:

2.2 Objective/Motivational Example There is a special situation in which the real-time scheduling with MG model yields more slack times that can be used to save more energy than the traditional real-time DVS algorithm. In the MG model, when the real-time task’s reserve is

124

J. Sun and H. Cho

Table 1 Task sets for motivational example s Reserve

ACET

Act. ETs

1 2

1 0.5

2 1

{2, 4, 4},{8, 20} {1, 5, 5},{4, 20}

depleted, the depleted task’s priority is downgraded to the same level as (or below) that of other non-real-time tasks [1]. Then, the depleted task’s (i.e., the non-real-time task) workload can be added to slack. For the simple explanation of our motivational example, we assume that the slack times only get from the depleted task excluding those from the tasks’ early completion, processor idle time. The next examples show a special situation, which yield additional slack times and two different approaches on how to use the additional slack times are suggested. The example uses the task set in Table 1. Figure 1a illustrates the worst-case RM scheduling without MG model (Reserve) and DVS, neither. Figure 1b illustrates the worst-case RM scheduling on a frequency-scaled processor with MG model (Reserve). In Fig. 1b and c, s1 is depleted at time t = 14, s1 ’s priority is downgraded. Now we determine the amount of slack generated from the depleted task as all of workload: slack2new ¼ wc1 ; and a part of workload: slack2new ¼ wc1  ac1 : In all of workload, s2 ’s slack time includes all workload of the depleted task s1 ; and thus the speed of s2 is scaled down to 0.3333ð¼ wc2 =ðwc2 þ slack2 Þ ¼ 1=ð1 þ 2ÞÞ as in Fig. 1b. Although this method significantly improves the energy efficiency of real-time tasks, it causes negligence of non-real-time tasks, i.e., delayed completions of non-real-time tasks. On the other hand, a part of workload leaves a part of workload of depleted task s1 to itself and the remaining workload of s1 is added to the slack of s2 : Then, the speed of s2 is scaled down to 0.5ð¼ wc2 =ðwc2 þ slack2 Þ ¼ 1=ð1 þ 1ÞÞ as in Fig. 1c. This method advances the completion of non-real-time tasks by sacrificing energy efficiency compared to Fig. 1c. These two aggressive and conservative approaches show the tradeoff between energy efficiency and capability of processing non-real-time tasks.

3 The Algorithm We first propose a simple mechanism for providing voltage scaling while maintaining schedulability of a given real-time task set based on the multi-granularity reserve model. In this mechanism, we select the lowest possible operating frequency that will allow the RM scheduler to meet all deadlines of a given task set. This frequency is set statistically and will not be changed unless the task set is changed. The worst-case response time of a task is the summation of its own computation time and possible preemption from all its higher-priority tasks. This leads to the proof of Theorem 1. The maximum preemption time by si with speed factor in ðt ;t þtÞ time interval ðt0 ; t0 þ tÞ; denoted as Pi 0 0 is given by [1] with modified Ci =Sl :

Energy-Efficient Multi-Granularity Resource

125

Fig. 1 Examples of DVS only considering the MG model

Theorem 1 For a multi-granularity resource reservation system, the worst-case response time for a task si is the smallest solution to the following equation: P ð0;wk Þ ð0;wk Þ where Pj is the preemption time from higher wkþ1 ¼ Ci =Sl þ j\i Pj priority task si in the interval ð0; wk Þ: While Sl is selected in Sl 2 f S1 ; . . .; Snf jS1 \. . .\Snf g where nf is the number of speed factor, the formula in Theorem 1 finds the smallest speed factor, which can be solved recursively starting with w0 ¼ Ci =Sl and terminating when wkþ1 ¼ wk on success or when wkþ1 [ Di on failure. Fig. 2a illustrates the multi-granularity resource reservation non-DVS algorithm (MULTI-RSV). In this figure, s0 and s2 is the real-time tasks and s2 is the non-real-time task. The example of Fig. 2 uses the task set in Table 2. Fig. 2b illustrates these mechanisms, showing sample execution traces under statisticallyscaled RM scheduling. By Theorem 1, the static DVS-MG algorithm is scheduled with the frequency setting as 0.9. Algorithm 1. lpWDA-MG (lpLDA-MG with mm) Compute the available execution time and set the voltage/clock speed for sa 1. Initially, put all tasks into ready Q and, for each task sa ; set uda :¼ dicur :¼ pa ; wrem :¼ lowest granularity execution timeðaÞ; a ðtÞ :¼ wca ; reserve l cur ma l cur m P P da da 2. Initially, compute Ha ðtÞ :¼ a1 ð  wci Þ; NNAa ðtÞ :¼ a1  aci Þ; i¼1 i¼1 ð pi pi 3. When a task sa is replenished, set reservea :¼ lower granularity execution timeðaÞ; set a initial priority to sa ; 4. When a task sa is depleted, set uda :¼ next replenish timeðaÞ; set a priority to sa lower than real-time task; (continued)

126

J. Sun and H. Cho

(continued) Algorithm 1. lpWDA-MG (lpLDA-MG with mm) rem 5. When a task sa is activated, l mset lwacur ðtÞm:¼ wca P d p uda Ha ðtÞ :¼ ðwrem  a p a Þ  wci Þ; i ðtÞ þ ð p i

si 2TH a ðtÞ

i

m ðaci þ ð p  pi Þ  aci Þ; i where TaH ðtÞ ¼ f s1 ; . . .; sa1 jprio1 \. . .\prioa1 g 6. When a task sa is completed or preempted, UpdateLoadInfoðTaH ðtÞ; aÞ; 7. When a task sa is scheduled for execution if sa (is real-time task) then slacka ðtÞ :¼ CalcSlackTimeðÞ; mmAa ðtÞ :¼

l

P

si 2TH a ðtÞ

uda

m

l

dcur a pa

else if sa (is non-real-time task) then slacka ðtÞ :¼ 0; wrem ðtÞ

8. set the clock frequency as fclk :¼ slacka ðatÞþwrem ðtÞ  fmax ;   a a wdone mm flimit :¼ max Ai þac ji ¼ 1; . . .; n ; fclk :¼ maxðfclk ; flimit Þ; cur d t i

Algorithm 2. UpdateLoadInfo (C, a) Input: Tasks C and the preempted/completed task index a and the amount of work wdone done for sa in the previous schedule Output: Workloads and reserve are updated to reflect current execution information 10. if (COMPLETION) then 11. if sa (is real-time task) then 12. reservea :¼ reservea  wdone ; uda :¼ uda þ pa ; dacur :¼ dacur þ pa ; end if 13. if sa (is non-real-time task) then dacur :¼ dacur þ pa ; end if l m l cur m d p ð udpia  a pi a Þ  wci ; l m l cur m P d p 15. mm Aa ðtÞ :¼ Aa ðtÞ þ si 2C ð udpia  a pi a Þ  aci ;

14. Ha ðtÞ :¼ Ha ðtÞ þ

16. 17. 18. 19. 20. 21. 22. 23. 24. 25.

P

si 2C

loop from k :¼ a þ 1 until k ¼ n by increasing k Hk ðtÞ :¼ Hk ðtÞ  wrem a ðtÞ; mm Ak ðtÞ :¼ Ak ðtÞ  maxð0; aca  wdone Þ; wrem a ðtÞ :¼ wca ; else if (PREEMPTION) temp :¼ wca  wdone ; loop from k :¼ a þ 1 until k ¼ n by increasing k Hk ðtÞ :¼ Hk ðtÞ  wrem a ðtÞ þ temp; mm Ak ðtÞ :¼ Ak ðtÞ  wrem a ðtÞ þ temp; wrem a ðtÞ :¼ temp;

Energy-Efficient Multi-Granularity Resource

127

Fig. 2 The voltage scheduling examples with MG model in time [60, 66];a MULTI-RSV;b static DVS-MG;c lpWDAMG;d lpLDA-MG

Table 2 Task sets for scheduling example

s

Reserve

ACET

Act. ETs

1 2 3

{1, 3, 3},{4, 30} {1, 4, 4},{4, 40} {1, 6, 6},{5, 60}

0.4 0.4 0.5

0.8 0.8 0.8

3.1 Low-Power Work/Limit Demand Analysis with Multi-Granularity We propose low-power work demand analysis with multi-granularity (lpWDAMG) and low-power limit demand analysis with multi-granularity (lpLDA-MG) as shown in Algorithm 1 with several additional lines marked with mm. Both lpWDA and lpLDA algorithms are based on RM scheduling and the same as MULTI-RSV using the MG model in [1]. That is why lpWDA set the base. In Algorithm 1, the system is first initialized by setting the initial upcoming deadlines (ud), remaining worst-case execution ðwrem Þ; and Ha ðtÞ of each task in lines 1, 2. Ha ðtÞ denotes the estimation of the workload in which higher-priority tasks than sa that must be executed before uda : Whenever a task sa is replenished at time t, tasks’ reserve, priority are updated to be its lower granular execution time and an initial priority assigned by RM rule. When a task sa is depleted at time t, tasks’ uda and priority are updated to be the next replenish time and a lower priority than the other real-time tasks, respectively, in line 4. Whenever a task sa is completed or preempted at time t, wrem a ðtÞ; uda ; and Ha ðtÞ are updated in line 6. In

128

J. Sun and H. Cho

line 7, when a task sa is scheduled for execution at time t, CalcSlackTime() computes the available slack for sa according to Hb ðtÞ and Lb ðtÞ; the work required by the lower-priority tasks that were activated before t or will be activated during ½t; da ; where udb is the earliest upcoming deadline with respect to sa : Formally, to describe the slack analysis method using lpWDA-MG, the following notations are defined: TaACT ðtÞ : the subset containing the active tasks at time t is TaACT ðtÞ :¼ f sk jk\a; sk 2 readyQðtÞ; sk is the real-time tasks}, loada ðtÞ : the amount of work required to be processed in interval ½t; da Þ considering only real ðtÞ þ H ðtÞ þ L ð t Þ slack time tasks loada ðtÞ ¼ wrem a a a ðtÞ : the available slack for a sa scheduled at time t. The equation of above terms are decided same on [3, 4]. Consequently, these modifications can get more slack times than that by the existing scheme. Further, this algorithm provides the energy savings. Now let’s show some examples. Figure 2c and d illustrate the real-time DVS algorithm of lpWDA-MG, lpLDAMG respectively. Fig. 2 shows simple and clear examples of the characteristic that gets the slack time from MG. Further, these figures meet all real-time tasks’ deadline. However, lpWDA-MG uses the non-real-time task s1 ’s work in time interval [60, 64] (presented by a diagonal line) to slack for energy savings. Although the task s1 misses its deadline at time t = 64, the real-time task s2 can be scheduled at the low frequency (0.3333) in Fig. 2c. In the same case, lpLDA-MG schedules the real-time task at a frequency that is higher than the frequency of lpWDA-MG, the non-real-time task meets its deadline. The aforementioned algorithms summarize the proposed slack estimation procedure and scheduling considering the MG model. During run time, these algorithms can be executed at every scheduling point such as activation, resumption, completion, and depletion and replenishment of task instances. In estimating the available execution time for the scheduled task, only one instance is examined per periodic task. Therefore, the proposed algorithm does not increase the computational complexity more than the existing online DVS algorithms. The schedule produced by lpWDA-MG and lpLDA-MG guarantees all system deadlines and has a computational complexity of O(n) per scheduling point, where n is the number of tasks in the system.

4 Algorithm Properties/Schedulability Analysis Our schedulability analysis follows an approach similar to that of Saewong [1]. The proposed schedulability test is followed by worst-case response time analysis. In lpWDA [3], they prove the schedulability of lpWDA by using the worst-case response time (WR or WCRT) analysis. We modify the analysis of lpWDA which proves the schedulability of lpWDA-MG. The following theorem proves the schedulability of lpWDA-MG by using the worst-case response time analysis. Note that lpLDA-MG did not affect the schedulability of lpWDA-MG.

Energy-Efficient Multi-Granularity Resource

129

Theorem 2 Given that a set T of tasks is feasible in the RM schedule, the maximum response time of each task in T under lpWDA-MG is less than or equal to its deadline. We almost follow the same Theorem and proof as that in lpWDA [3] except considering the non-real-time tasks. The detailed proof of Theorem 3 is described in lpWDA [3]. Proof Assuming task sa has the highest priority and task sb is a lower priority task with depleted reserve in readyQ. By loada ðtÞ in [3, 4], we get WRa ¼ loada at time t = 0. When sa runs out slacka ; all subsequent jobs of sa have to separately postpone their response times at most slacka unit of times compared to those in their WCET RM schedule. Assuming sc has lower priority than sa and sb is the depleted task, we prove that the length of new WCRT of sc including slacka and the amount of all depleted task’s WCET is less than dc . According to Theorem 2, we have WRWDAMG þ slackaWDAMG  dc c

ð1Þ

The priority of sb is changed temporally to be lower than sc until its next replenish times. Then, WCRT should be newly calculated. The WCRT of sc under lpWDAMG is changed to WRWDAMG ¼ WRnew  wcb c c

ð2Þ

where WRnew is used in the correctness proof of lpWDA [3]. Additionally the slack c times of sa considering the workload of depleted task sb is denoted as slackaWDAMG ðtÞ ¼ slacka ðtÞ þ wcb

ð3Þ

 wcb þ slacka ðtÞ þ wcb  dc By Eq. (1)–(3), WRnew c WRnew þ slacka ðtÞ  dc c

ð4Þ

Later, the same proofs will proceed the lpWDA proof in [3].

5 Experimental Results For simulation-based experiments, we use OMNeT++ [6], which provides a discrete event simulation environment. We use the traditional energy model whose coefficients are configured as S3 ¼ 1:0; S2 ¼ S1 ¼ S0 ¼ 0: In the experiments, the voltage scaling overhead is assumed negligible. The experimental parameters and scenarios of the lpWDA and MG models are enumerated in [1, 3], respectively. In the lpWDA, the reserves of each task can be represented as {{WCET, Period, Period}, {WCET, Period}}, i.e., the hard real-time

130

J. Sun and H. Cho

Fig. 3 Normalized energy consumption in a Videophone, b Avionics

tasks can be represented in the MG model. A simple multi-granularity reservefor  MPEG4 video decoder is given by R ¼ ffdtmax ; T; T g; dtavg  gop; gop  T } where gop, T, dtmax and dtavg denote the size of GOP, the frame period, the maximum and average decoding time, respectively. For this case, (m,k)-firm deadline constraint for the stream can be written as mmpeg4 ¼ ðdtavg  gopÞ=dtmax ; gop: As suggested in [7], we assume that the video decoding time of each frame is linearly increasing along with its corresponding frame size. The two following metrics utilized for evaluating the performance of scheduling algorithms include MULTI-RSV, static DVS-MG, lpWDA-MG, and lpLDA-MG: Miss I-frame: the ratio of the number of deadline-missing I-frames to the total number of frames (of all types), Dyn: the ratio of dynamic errors (defined in [8] as the failure of a system to satisfy the timing constraints of at least m frames out of any k consecutive frames).

Energy-Efficient Multi-Granularity Resource

131

Fig. 4 Simulation results for MPEG41

Figure 3 illustrates the experimental results for real-world application task sets in [3]. It shows that lpWDA-MG and lpLDA-MG achieve more energy savings than the MULTI-RSV. The energy efficiency of the proposed algorithms increases as the ratio of BCET to WCET decreases. The proposed algorithms have the same energy efficiency as lpWDA and lpLDA when they schedule real-time tasks as shown in [3]. Additionally, we simultaneously run three streams, Jurassic, News and Lecture with both low workload (RT-U = 0.2, U = 0.45) and high workload (RT-U = 0.35, U = 1.25) as shown in Fig. 4a and b, respectively. RT-U and U denotes the hard real-time task utilization and the total system utilization excluding the video stream utilization. Figure 4 shows the normalized energy 1

In the experiments, Miss I-frame and Dyn do not occur at all methods.

132

J. Sun and H. Cho

consumption of each scheduling algorithm when they run multiple video streams. Comparing the energy consumption of MULTI-RSV, it shows that lpWDA-MG and lpLDA-MG reduce energy consumption while Miss I-frame and Dyn do not occur at all. Thus, it confirms that lpWDA-MG and lpLDA-MG reduce energy consumption and at the same time they guarantee the quality of multimedia services.

6 Conclusions There is a special situation in which real-time scheduling with the MG model yields more slack times that can be used to save more energy than that by the traditional real-time DVS algorithm. We propose the real-time DVS scheduling algorithms called lpWDA-MG/lpLDA-MG to reduce energy consumption using computed slack by analysis considering the given MG model. Then, we prove the schedulability of lpWDA-MG using the worst-case response time analysis even if slack is increased by the MG model. The experimental results show that the proposed algorithms reduce the energy consumption by 2–50%. Acknowledgments This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 2011-0011534).

References 1. Saowanee S, Ragunathan R (1999) Multi-granularity resource reservations. In: Proceedings of the 26th IEEE international real-time systems symposium (RTSS7 ‘05). IEEE Computer Society, Washington, DC, USA, pp 143–153 2. Padmanabhan P, Kang Shin G (2001) Real-time dynamic voltage scaling for low-power embedded operating systems. SIGOPS Oper Syst Rev 35(5):89–102 3. Woonseok K, Jihong K, Sang LM (2003) Dynamic voltage scaling algorithm for fixedpriority real-time systems using work-demand analysis. In: Proceedings of the 2003 international symposium on low power electronics and design (ISLPED ‘03). ACM, New York, USA, pp 396–401 4. Da-Ren C (2010) Slack computation for DVS algorithms in fixed-priority real-time systems using fluid slack analysis. J Syst Archit, in press, corrected proof, Available online 15 October 2010, ISSN 1383–7621 5. Martin TL, Siewiorek DP (2001) Nonideal battery and main memory effects on CPU speedsetting for low power. IEEE Trans Very Large Scale Integration (VLSI) Syst 9(1):29–34 6. VARGA A (2005) OMNeT ++ Discrete event simulation system. http://www.omnetpp.org/ Accessed 22 June 2005 7. Bavier A, Montz B, Peterson L (1998) Predicting MPEG execution times. n SIGMETRICS/ PERFORMANCE’98, international conference on measurement and modeling of computer systems 8. Hamdaoui M, Ramanathan P (1995) A dynamic priority assignment technique for streams with (m, k)-firm deadlines. IEEE Trans Comput 44(12):1156–1174

Energy-Efficient Multi-Granularity Resource

133

9. Bren M, Xiaobo SH, Gang Q (2007) Transition-overhead-aware voltage scheduling for fixedpriority real-time systems. ACM Trans Des Autom Electron Syst 7:249–264 10. Bernat G, Burns A, Liamosi A (2001) Weakly hard real-time systems. IEEE Trans Comput 50(4):308–321

Single Camera Vehicle Detection Using Edges and Bag-of-Features Syed Jahanzeb Hussain Pirzada, Ehsan Ul Haq and Hyunchul Shin

Abstract Vehicle detection is becoming a necessary part of Automatic Cruise Control (ACC) and Advanced Driver Assistance Systems (ADAS). Our main focus in this paper is on improving the performance of single camera based vehicle detection systems. Edges are one of the main characteristics of an object, which carries most of the information about an object in an image. In this paper, it was observed that horizontal edges are strong feature for vehicle detection. Therefore, we generated initial candidate using Horizontal Edge Filtering (HEF) on canny edge map. These initial candidates are further verified using Bag-of-Features (BoF) with K nearest neighbor algorithm. A threshold is used on differences of histograms of training and test images for matching the vehicles. In this paper, the combination of edges (initial candidate) and bag-of-features (final verification) has improved detection rate significantly as compared to other well known methods. Our method has 96% detection rate on roads inside a city and 98% detection on highways.




Keywords Bag-of-features Canny edge detection K nearest neighbour Vehicle detection system





Horizontal edge filtering


S. J. H. Pirzada (&)
E. U. Haq
H. Shin Department of Electronics and Computer Engineering, Hanyang University, Ansan, South Korea e-mail: [email protected] E. U. Haq e-mail: [email protected] H. Shin e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_13, Ó Springer Science+Business Media B.V. 2012

135

136

S. J. H. Pirzada et al.

1 Introduction Nowadays travelling on vehicles has become a necessity instead of luxury. However at the same time, number of deaths due to the vehicular accidents is also increasing each year. Worldwide vehicles accidents effects every year between 20 to 50 million people [1]. Therefore, developing an advanced driver assistance system (ADAS) is becoming more and more important. A lot of research is going on developing a vision based vehicle detection system [2, 3]. The main goal of vision based vehicle detection systems is to accurately and efficiently identify dangerous conditions [4]. Usually accidents can be avoided by detecting lanes [5], pedestrian [6] and nearby vehicles [7]. In this paper, we have presented a new vehicle detection system for detection of nearby vehicles. Edge is one of the main characteristics of an object, which carries most of the information of an object in an image. That’s why edge information is very important in image processing, particularly, in the area of object detection and recognition algorithms [8]. There are many edge detection methods used for vehicle detection like Canny [9], Sobel [10] and SUSAN [11] edge detectors. Many papers has also compared [12, 13] different edge based methods for different applications. In this paper, based on comparisons [12, 13], we have found canny edge detector as best technique for finding vehicle edges. It was observed that horizontal edges are strong feature for vehicle detection [15]. So we have applied Horizontal Edge Filtering (HEF) to filter the long horizontal connected edges to detect only vehicle from all the edges in the image. BoF is becoming as an important object recognition and classification technique. BoF was derived from ‘bag-of-words’ (BoW) modeling for text retrieval like Google search. BoW technique is a text analysis algorithm discriminating documents by occurrence of particular words that each document contains [14]. In this paper, initial candidates generated using edges are further verified using Bagof-Features (BoF) with K Nearest Neighbour (KNN). BoF is used for verification because this technique verifies the vehicle and non vehicle candidates effectively. In past decade, there has been a lot of research on real-time detection of vehicles. In [15] authors have reviewed most of the techniques. In [15–17] authors emphasize that vertical and horizontal edges are very powerful heuristic for initial candidate detection. Matthews et al. [16] has used edge detection to find vertical edges for detection of left and right position of vehicle. But vertical edges are not very strong heuristic for on road vehicles as compared to horizontal edges. Moreover, some obstacle on road may also have same vertical and horizontal edge profile as of a car therefore we need an additional verification step. Some authors [18, 19] also used shadow of car and combination of horizontal and vertical edges for vehicle detection. Handmann et al. [17] proposed a template based on the observation that the rear/frontal view of a vehicle has a ‘‘U’’ shape (i.e., one horizontal edge, two vertical edges, and two corners connecting the horizontal and vertical edges). Hence, edges prove to be effective candidate to be used for initial candidate detection. Therefore we selected the HEF for initial candidate selection.

Single Camera Vehicle Detection

137

Fig. 1 Edge map generated by canny edge detection for rear view of vehicles

Although using only edges do not provide good detection rate so we combine edge based initial candidate detection with verification using BoF algorithm. The remaining paper is organized as follow. Section 2 introduces to Canny Edge Detection. Section 3 explains the BoF algorithm. Section 4 explains methodology of proposed technique. Section 5 provides experimental results. Section 6 concludes this paper.

2 Canny Edge Detection Canny edge detector performs smoothing with a linear combination of exponential functions and then detects edges by higher order derivative operators. Canny tried to optimize a set of criterion to maximize the performance of already available edge detection methods. First requirement was low error rate, i.e. high response to edges while minimum response to surfaces or planes. Second criterion was that edges should be well localized on the boundaries. Third criterion was to have only a single response to a single edge. First Canny edge detector smoothes the image to remove noise. Next it calculates gradient of the image. Then next step is to perform the non maximum suppression. In this step all those pixels which are not at the maximum are suppressed. The edge map is further improved by hysteresis, which tracks along the non suppressed pixels. Canny edge detector next uses two thresholds. If a pixel’s value is above higher threshold then it is selected as an edge. However, if a pixel’s magnitude is below the second threshold then it is set to zero. In case if magnitude is between the two thresholds then it is set to zero unless it is connected to an edge pixel. In this paper, canny edge detector is used on images with rear view of vehicles as shown in Fig. 1.

3 Bag-of-Feature Algorithm BoF modelling is carried out by constructing a vocabulary of visual words from a set of Scale Invariant Feature Transform (SIFT) descriptors. That is, it should go through several steps; keypoint detection, extraction of SIFT descriptor, and vector

138

S. J. H. Pirzada et al.

Fig. 2 Shows the flow chart of main methodology

quantization. Keypoint-based regions are located at salient points such as corners and blobs. Thus meaningful keypoints are selected. Next step is the extraction of SIFT descriptor. SIFT descriptors are extracted from a set of detected keypoints. Then vector quantization is carried out to generate a visual vocabulary using a set of SIFT descriptors extracted from various images. Clustering algorithms such as k-means and k-median are used for vector quantization. After k clusters are generated by clustering, the centre of each cluster becomes a visual word. Therefore, the database of SIFT descriptors should be sufficiently large and various to generate visual words. A feature vector of each image is represented by the occurrence of the visual words; it can take typical histogram of visual words. Histogram expresses the frequency of the visual word in any given image.

4 Methodology In this paper, main methodology involves two main steps, initial candidate generation using edge based method and verification using BoF algorithm as shown in Fig. 2.

Single Camera Vehicle Detection

139

Fig. 3 a Shows region of interest (RoI) generation, b shows canny edge map of RoI image, c Shows results of horizontal edge filtering on an image

4.1 Initial Candidate Generation Initial candidate generation is very important for vehicle detection as whole detection accuracy is very much dependent on it. In this paper, for initial candidate generation, we used HEF of canny edge map. For reducing the computation time, we have selected region where there are cars and roads in the input images. For region of interest (RoI) selection we cropped image from top and bottom only. Canny edge detector is applied on RoI of input image to generate edge map as shown in Fig. 3b. In images with rear view of vehicles it was noticed that long connected horizontal edges are usually belongs to vehicles. Depending on this observation HEF is applied on canny edge response for detection of vehicles. In HEF, each row of whole edge map is searched for at least ten consecutive 1’s. Whenever a number of consecutive ones are found, they are preserved as a long connected horizontal edge. Following this criteria whole image is scanned and only those long connected edges are assigned value 1, while small unconnected edges is assigned value 0. As a result small unconnected edges are neglected and only long connected edges are used. Vehicles rear view generally contains long horizontal edges due to the shape of bumper and back windscreen and roof of vehicle etc. These long edges are located close to each other and also vertically aligned. This information can be used to obtain initial candidate. We have observed that if there are three long edges close to each other and vertically aligned then it can be considered as an initial vehicle candidate and labelling rectangle is generated around this region as shown in Fig. 3c.

140

S. J. H. Pirzada et al.

4.2 Verification Using Bag-of-Feature Verification of initial candidate is an important step in our approach, as it reduces number of false matches and improves accuracy of the detection. In this paper, we used BoF algorithm for verification. BoF is one of the most used and successful techniques for object detection. In BoF algorithm, there are three main steps; visual vocabulary generation, training class generation and testing. In the Visual Vocabulary Generation, keypoints and descriptors are generated by using Harris corner detector and Scale Invariant Feature Transform (SIFT) Descriptors [20] respectively. Each descriptor represents a single patch. After patches are formed from all images k-mean clustering [21] is used to cluster similar patches together. After k (k = 500) clusters are generated by clustering, the centre of each cluster becomes a visual word. Visual vocabulary for vehicle and non vehicle class are generated separately. We use 500 images of vehicles from Caltech database and 500 images from random Google links for non vehicle database for vocabulary generation. In the second step training for class generation is performed to generate training histogram of vehicle class and non vehicle class. Same algorithm is followed in training images as of vocabulary generation except the last part of vocabulary generation is replaced by generation of training histograms. In the training cycle 500 images from Caltech vehicle dataset [22] is used for training of vehicle class and many random images (about 500 images) from different Google links is used for non vehicle class. In the third step testing is performed on the labelled images extracted from edge response during initial candidate detection. For test images, histograms are generated using same training class generation algorithm. Then K nearest neighbour algorithm is used for matching the training and test images histograms. If the difference in neighbour distance between histograms of training and test images is less than a threshold then it’s acceptable as a vehicle. Otherwise, the image is considered as a non-vehicle class. A threshold value of 1.50 is used for comparing neighbour distances of histogram; this value is selected by hit and trial method. Then the non vehicle class histogram is matched with histogram of test image and if difference is less than the threshold, then it’s confirmed as a non vehicle. Verification of vehicles double checks the results from edge response and hence improves detection by removing false initial detection.

5 Experimental Results Experiment is performed using MATLAB version R2008b on a PC with Intel Core2 Duo processor with 2 GB random access memory (RAM). In our experiment we extracted images from real life video. The video is captured from camera which is mounted inside a vehicle at a fix place to shoot the road.

Single Camera Vehicle Detection Table 1 Results of vehicle detection S.No 1 2 3 4 5 6

Total vehicle candidates Initial candidates detection Non-vehicles detection Verified candidates Missed vehicles Detection rate

141

Image sequence Inside city

Highways

240 234 3 231 9 96%

150 169 22 147 3 98%

(a)

(b)

(d)

(d)

(e)

(f)

Fig. 4 a shows detection of vehicles in highway, b shows detection of vehicles inside city roads, c shows Wrong detection using inside city roads images, d shows Wrong detection using highway image, e shows undetected vehicles in circles inside city roads. f shows a missed vehicle in a circle which is far from the camera

Table 1 show the results of vehicle detection by our proposed method. The experiment is performed on two different cases, roads inside the city and on highways. Total vehicle candidates represent the amount of total vehicles present

142

S. J. H. Pirzada et al.

in all the frames of video. Initial Candidates detection represents total candidates detected. Non-vehicle detection represents the total non vehicle detection, which is removed by verification process. Verified candidates represent the total vehicles detected after verification by BoF method. Missed Candidates represents number of vehicles that are not detected. The missed vehicles are due to overlapping vehicles and large distance of vehicles from target vehicle. Detection rate represent the rate of detection of vehicles. Fig. 4a shows the detection of vehicles in highways and Fig. 4b shows detection of vehicles in city roads. Fig. 4c shows non vehicle and wrong vehicle detection in highways and Fig. 4d shows it in city roads after initial candidate detection. Fig. 4e shows missed overlapping vehicles. Fig. 4f shows a missed vehicle which is far from the camera. There were many overlapping vehicles on roads inside city as compared with on highways. So the detection rate is less on roads inside city.

6 Conclusion In this paper, edge based initial candidate detection is used with bag-of-feature based verification. The combination of two methods has helped to detect all vehicles using edge based method and remove false vehicle detection by verification through BoF algorithm. So, combination of two well known techniques is used for improving the detection rate of vehicles. Our results show around 98% detection of vehicles on highways where there is low traffic and 96% detection on roads inside city where there is high traffic. In this work we have emphasized on day time sunny and cloudy weather conditions. In future, this work will be enhanced for different weather conditions. Acknowledgments This work was supported by the Ministry of Knowledge Economy (MKE) through IDEC Platform center (IPC) at Hanyang University. Moreover, Jahanzeb and Ehsan were supported by ‘Higher Education Commission (HEC) from the Govt. Of Pakistan’ under the scholarship program titled: MS level Training in Korean Universities/Industry.

References 1. World Health Org. World Report on Road Traffic Injury Prevention, [Online]. http:// www.who.int/violence_injury_prevention/publications/road_traffic/world_report/factsheets/ en/index.html 2. Sun Z, Bebis G, Miller R (2005) On-road vehicle detection using evolutionary gabor filter optimization. IEEE Trans Intell Transp Syst 6:125–137 3. Sun Z, Bebis G, Miller R (2006) On-road vehicle detection: a review. IEEE Trans Pattern Anal Mach Intell 28:694–711 4. Trivedi MM, Gandhi T, McCall J (2007) Looking-in and looking-out of a vehicle: Computervision-based enhanced vehicle safety. IEEE Trans Intell Transp Syst 8(1):108–120

Single Camera Vehicle Detection

143

5. McCall J, Trivedi MM (2006) Video-based lane estimation and tracking for driver assistance: Survey, system, and evaluation. IEEE Trans Intell Transp Syst 7(1):20–37 6. Gandhi T, Trivedi MM (2008) Computer vision and machine learning for enhancing pedestrian safety in Computational Intelligence in Automotive Applications. Springer, Berlin, pp 59–77 7. Gandhi T, Trivedi MM (2006) Vehicle surround capture: Survey of techniques and a novel Omni video based approach for dynamic panoramic surround maps. IEEE Trans Intell Transp Syst 7(3):293–308 8. Hongna Z, Qi W (2003) Image measuring technique and its applications. J Electr Meas Instrum 9. Canny JF (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8:679–714 10. Russ JC (2002) The image processing handbook. CRC Press, Boca Raton 11. Smith SM, Brady JM (1997) SUSAN- A new approach to low level image processing. Int J Comput Vis 23(1):45–78 12. Musoromy Z, Ramalingam S, Bekooy N (2010) Edge detection comparison for license plate detection. IEEE Trans Control Autom Robot Vis (ICARCV), pp 1133–1138 13. Fan C, Ren Y (2010) Study on the edge detection algorithms of road image. In: Third international symposium on information processing, pp 217–220 14. Brin S, Page L (1998) The anatomy of a large-scale hypertextual web search engine. In: Proceedings international WWW conference computer networks and ISDN systems, pp 107–117 15. Sun Z, Bebis G, Miller R (2004) On-road vehicle detection using optical sensors a review. In: IEEE international conference on intelligent transportation systems, Washington, DC, 3-6 October 2004, pp 585–590 16. Matthews N, An P, Charnley D, Harris C (1996) Vehicle detection and recognition in greyscale imagery. Control Eng Pract 4:473–479 17. Handmann U, Kalinke T, Tzomakas C, Werner M, Seelen W (2000) An image processing system for driver assistance. Image Vis Comput 18(5):367–376 18. Tzomakas C, Seelen W (1998) Vehicle detection in traffic scenes using shadows. Technical Report 98-06. Institutfur Neuroinfor-matik, Ruht-Universitat, Bochum, Germany 19. Srinivasa N (2002) A vision-based vehicle detection and tracking method for forward collision warning In: Proceedings of the IEEE intelligent vehicle symposium, pp 626–631 20. Lowe DG (2004) Distinctive image features from scale invariant keypoints. Int J Comput Vision 60:91–110 21. MacKay DJC (2003) Information theory, inference, and learning algorithms 22. Caltech Cars Dataset. http://www.robots.ox.ac.uk/*vgg/data3.html

A New BiGaussian Edge Filter Ehsan Ul Haq, Syed Jahanzeb Hussain Pirzada and Hyunchu Shin

Abstract Edge detection has been the foremost step in image processing and computer vision, because an edge representation drastically reduces the amount of data to be processed. Although classical methods of edge detection like Sobel, Canny, etc. are simple to use but has a dilemma between noise removal and edge localization. If noise is to be removed by using a low pass filter then edges are blurred. However, if edges have to be preserved then noise severly corrupts the edge map. In this paper, we have proposed a new method of edge detection, BiGaussian edge Filter, which simultaneously removes noise from real life images, while generating well localized edges. We have compared our method using images form Berkely’s segmentation data set. Experimental results show the robustness of our method to noise in real life images.







Keywords Edge detection Gaussian filter Noise removal Bilateral filter Canny edge detector Sobel edge detector BiGaussian edge filter







E. U. Haq (&)  S. J. H. Pirzada  H. Shin Department of Electronics and Computer Engineering, Hanyang University, Ansan, South Korea e-mail: [email protected] S. J. H. Pirzada e-mail: [email protected] H. Shin e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_14, Ó Springer Science+Business Media B.V. 2012

145

146

E. U. Haq et al.

1 Introduction Recently, understanding contents of images has become a significant objective of image processing and computer vision researchers. An image contains a lot of information to be classified and interpreted by a machine. However, depending upon the application at hand much of this information could be redundant. Therefore, edge detection has been a primary component of a wide range of application such as image enhancement, compression, recognition, retrieval, morphing, etc. A major reason for this being the foremost step of object recognition is also that edge representation drastically reduces the amount of data to be processed. The edges correspond to major geometrical and photometrical changes in image objects and hence provide important visual information [1–3]. Edge detection is a process of identifying and locating these sharp discontinuities in an image. In the ideal case, the result of applying an edge detector to an image should return a set of curves that correspond to discontinuities in surface orientation. Edge detection has been extensively researched for decades in computer vision, but it has still not been matured to an optimal level [1–6]. Although the classical methods of edge detection like Sobel, Prewitt, Robert’s Cross, etc. [7, 8] are very simple to use and detect edges seamlessly in case of synthesized images. However, these methods are highly sensitive to noise in real life images, therefore, failing to achieve any satisfactory results [1], [2], [8]. Moreover, the problem with differential edge detectors is that one has to search for either local extremas (first derivative) or zero crossings (second derivative). In this paper, we have presented a new simple and non iterative method of edge detection, BiGaussian edge detector, that aims to overcome these above mentioned cons of earlier methods. The underlying idea of BiGaussian edge detector is to reduce noise in spatial domain of an image while simultaneously finding edges in intensity range of the image. As a consequence, this filter acts as a standard domain low-pass filter, which averages away small and weakly correlated differences between pixels caused by noise. Moreover, it also simultaneously acts a standard range high-pass filter to highlight differences among neighbouring pixels. Rest of the paper is arranged as follows. Section 2 briefly overviews the theory of edge detection. Then Sect. 3 explains some drawbacks in these earlier techniques. Next, in Sect. 4 some related work is elaborated. Further we have explained our new method of edge detection in Sect. 5. After that Sect. 6 discusses the comparison results and finally Sect. 7 concludes the paper.

2 Theory of Edge Detection First and second order derivatives of an image are the most popular methods of edge detection [2, 3, 8]. An edge can be positive or negative maxima of the absolute value of first order derivative. Alternatively, it is also the zero crossings of

A New BiGaussian Edge Filter

147

the second order derivative of the image. Gradient vector magnitude’s maximum value would be at the location of edge, whereas, the angle of gradient vector would be orthogonal to the edge. Edge detection based on second order derivatives are usually performed using the Laplacian, which is actually an approximation of the second order derivative of the image f, along the gradient direction. A simple approximation of the first order derivatives fx and fy is done by calculating the differences along the respective axes as follows fs ðs; tÞ ¼ f ðs; tÞ  f ðs þ 1; tÞ; ft ðs; tÞ ¼ f ðs; tÞ  f ðs; t þ 1Þ

ð1Þ

Where fs(s,t) and ft(s,t) are the approximations of gx and gy around the pixel (s,t). These can also be represented by masks, as follows:     f ðs; tÞ 1 ; ft ðs; tÞ ¼ ½f ðs; tÞf ðs; t þ 1Þ ð2Þ fs ðs; tÞ ¼ ½1  1 f ðs þ 1; tÞ 1   1 Hs ¼ ½1  1; Ht ¼ ð3Þ 1 To avoid the problem of asymmetry usually an odd number of pixels are used, as shown by equation below: 2 3 f ðs  1; tÞ ð4Þ fs ðs; tÞ ¼ ½ þ1 0  1 4 f ðs; tÞ 5 f ðs þ 1; tÞ "

þ1 ft ðs; tÞ ¼ ½f ðs; t  1Þf ðs; tÞf ðs; t þ 1Þ 0 1 " # þ1 Hs ¼ ½ þ1 0  1 ; Ht ¼ 0 1

# ð5Þ

ð6Þ

Several other similar approximations to the first order derivative along different orientations and with various scales have been proposed. To use these operators, a product is performed with the image using following equation, fx ðs; tÞ ¼

X X i

j

f ðs þ i; t þ jÞ : ðHx Þij

ð7Þ

Sobel: 2 1 0 14 Hs ¼ 2 0 4 1 0

3 2 þ1 þ1 1 þ2 5; Ht ¼ 4 0 4 þ1 1

þ2 0 2

3 þ1 0 5 1

ð8Þ

148

E. U. Haq et al.

Prewitt: 2

1 0 1 Hs ¼ 4 1 0 3 1 0

3 2 þ1 þ1 1 þ1 5; Ht ¼ 4 0 3 þ1 1

þ1 0 1

  þ1 þ1 ; H135 ¼ 0 0

0 1

3 þ1 0 5 1

ð9Þ

Roberts:  H45 ¼

0 1

 ð10Þ

These kernels [7, 8] are designed to generated vertical and horizontal edges, with one kernel for each perpendicular direction. All of these are only suitable for well-contrasted noiseless images. Marr and Hildreth [2] argued that an edge detecting operator should be a scalable differential operator, which can compute first or second derivative at different scales. Being scalable would allow large operators to detect blurry edges while small operators to detect fine edges. They achieved these goals using Laplacian of Gaussian (LoG) operator. It was also observed by Marr and Hildreth that LoG can be approximated by Difference of Gaussians (DoG). Although Canny edge detector [1] is better than all the detectors presented so far and is also being used most widely. However, it is also more complex and computationally expensive. Canny presented edge detection as an optimization problem with constraints. His optimization objectives were high signal to noise ratio, well localization of edge points and single edge response. He formulated a mathematical expression for these objectives and then tried to find an optimal solution for that expression. However, due to the fact that it is very difficult to find a closed-form solution satisfying all criterions, he showed that using first derivative of a Gaussian approximation successfully achieves 80% optimality.

3 Drawbacks of Earlier Techniques The classical methods of edge detection are highly sensitive to noise in real life images, therefore, fails to achieve any satisfactory results [5, 6, 9]. To differentiate actual edge gradients from noise gradients, concept of thresholding was introduced. To reduce the effect of this noise in real images, Canny and others first applied a low pass Gaussian filter on the images before calculating the gradient. Although this technique improved edge detection significantly, however it introduced other problems. First, if strength of low pass filter is low then noise is not removed, whereas if strength is increased then actual location of the edges is lost. To find the actual location of edges these algorithms further use some use edge localization techniques, making these sophisticated algorithms computationally very expensive (Fig. 1).

A New BiGaussian Edge Filter

(a)

149

(b)

(c)

Fig. 1 Canny Edge maps with different Gaussian strengths. a. Input image. b. More noise with narrow Gaussian low pass filter. c. Original location of edges lost with wider Gaussian low pass filter

Moreover, the problem with differential edge detectors is that one has to search for either local extremas (first derivative) or zero crossings (second derivative). However, techniques for finding either of them are not optimized, hence resulting in a lot false extremas and zero crossings (in case of canny edge detector).

4 Related Work In 1998, Tomasi and Manduchi [10] all introduced a smoothing filter that can also preserve edges by using a combination of domain-range filters. The bilateral filter adaptively adjusts its kernel weights depending upon the geometrical closeness (domain term) and photometric similarity (range term) of the neighboring pixels. If surrounding pixels of the filtered pixel are from the same object having similar intensity, then they are given larger weight. However at the edges where intensity difference is higher, surrounding pixels are weighted lower so that they have less effect on the filtered pixel. A standard bilateral filter is expressed as follows: Jp ¼

  1 X Iq f ðjjp  qjjÞg jjIp  Iq jj q2X Kp

ð11Þ

where p is the pixel being filtered with intensity Ip and q is the surrounding pixel with intensity Iq. Jp is the resulting pixel in the filtered image, Kp is normalization factor and X is the set of surrounding pixels in the window. In the above expression, function f(||p - q||) is called domain term, while g(||Ip – Iq||) is called range term. Generally, both f(||p - q||) and g(||Ip – Iq||) functions are taken to be Gaussian functions. In this paper, we have used the similar concept of bilateral filter to overcome the above mentioned drawbacks of earlier techniques and have introduced a new method of edge detection namely, BiGaussian Edge Filter.

150

E. U. Haq et al.

5 BiGaussian Edge Filter BiGaussian Edge Filter detects edges, while preserving their original location and reducing noise simultaneously. It achieves this by means of a nonlineaer combination of nearby image values. Unlike other methods, it is noniterative and simple. Furthermore, our strategy detects edges based on photometric similarity and geometric closeness. The underlying idea of BiGaussian edge detector is to reduce noise in domain of an image while simultaneously finding edges in range of the image. Two spatially close pixels can be compared perceptually. Spatial closeness means that they are located in same neighbourhood (close in domain). On the other hand, perceptual similarity means they are similar in intensity (close in range) that is a smooth surface, whereas, perceptual difference indicates an edge. In our method, we have exploited this information to device a new edge detector. To reduce noise, a Gaussian function of the Euclidean distance between its arguments can be applied to image in spatial domain, as follows:  2 gðp  qÞ ¼ e

12

jjpqjj dd

ð12Þ

whereas, simultaneously, to detect edges an Inverse Gaussian function of the Euclidean distance between its arguments can be applied in the intensity range as follows,  2   1 jjIp Iq jj ð13Þ h Ip  Iq ¼ 1  e2 dr When combined the final equation would becomes as follows, Jp ¼

  1 X I gðp  qÞh Ip  Iq q2X q Kp

ð14Þ

Where, p is the pixel under consideration and q are the surrounding pixels in neighbourhood. Ip and Iq is the intensity of pth and qth pixels. This method of noise removal with one Gaussian on domain and edge detection with another Inverse Gaussian on range will be denoted as BiGaussian edge filtering. It updates the value at p with an average of close and similar pixels. In a small neighbourhood, one Gaussian is applied on a same small neighbourhood in spatial domain to reduce noise. In same region, in case of smooth neighbourhood, pixel values are similar; therefore, Inverse Gaussian working on similarity assigns approximately zero weight to such pixels. On the other hand, consider now the case of sharp boundary in the small neighbourhood between a dark and bright region. When BiGaussian edge filter is centred on a pixel on bright side of the boundary, the similarity Gaussian function will assign higher weights to pixels on dark side and approximately zero weights to pixels on bright side. Conversely, when filter is centred on a dark pixel then bright pixels are weighted more instead.

A New BiGaussian Edge Filter

(a)

151

(b)

(c)

Fig. 2 Results of applying different edge detectors on synthesized image without noise. a. Input image without noise. b. Canny edge detector. c. BiGaussian edge filter

As a consequence, this filter acts as a standard domain low-pass filter, which averages away small and weakly correlated differences between pixels caused by noise. Moreover, it also simultaneously acts a standard range high-pass filter to highlight differences among neighbouring pixels. The width of Gaussian function in the domain is selected based on desired amount of low pass filtering. Similarly, the spread of Gaussian function on the range is set to achieve the desired amount of thickness of edges, or, the desired amount of combination of pixel values in the neighbourhood.

6 Results Evaluation of edge detecting methods is a very challenging task [5], [6], [11]. For our purpose, we have evaluated our method using both computer generated and real life images. Since Canny edge detector gives most promising results [1], [9], therefore in this paper we have shown explicit comparison with Canny edge detector only. To verify our method, first we tested it on a simple computer generated image. In case of no noise, both Canny and BiGaussian Edge detectors generated the precise edge map, as shown in Fig. 2 below. Next, we corrupted the image with Gaussian noise. In this case, when canny edge detector was applied with narrower low pass Gaussian filter then it failed to remove noise, as shown in Fig. 3b. On the other hand, increasing the strength of low pass Gaussian filter destroyed the edge’s location, as shown in Fig. 3c, where corners of the box are lost and are rounded off. However, Fig. 3d shows the result of our BiGaussian Edge filter, which contains noiseless and well localized edge map. To verify our scheme on real life data, we have used images from the Berkeley segmentation data set [11]. The table below shows the result of applying Canny and BiGaussian edge detector to images from Berkeley segmentation data set. Once again even with real life image to obtain a noiseless edge map Canny edge detector has to smooth image, which in turn destroyed the fine details. As shown in Fig. 4b, tops of both minarets and tomb are rounded off. Moreover, edges at boundaries of minarets have also moved outward. In case of BiGaussian edge

152

E. U. Haq et al.

(a)

(b)

(c)

(d)

Fig. 3 Results of applying different edge detectors on synthesized image with noise. a. Input image with noise. b. Canny edge map with with light noise removal. c. Canny edge map with high noise removal. d. BiGaussian edge filter

(b)

(a)

(c) Fig. 4 Comparison with real life images. a. Input image. b. Canny edge detector. c. BiGaussian edge detector

filter, these artifacts have been removed. Note that the result of canny edge detector is after Non maximal suppression and Hysteresis thresholding. However, we have not applied these techniques to our method to show the explicit comparison. To obtain well connected edges, an edge linking technique can be applied.

A New BiGaussian Edge Filter

153

7 Conclusions Earlier techniques of edges detection have contradiction between noise removal and edges’ localization. In order to obtain desirable results, one has to fine tune numerous parameters for a specific applications. If strong noise removal filter is used then original location of edges is lost. Otherwise, if original location has to be preserved then noise corrupts the edge map. In this paper, we have presented a BiGaussian Edge filter that can overcome this dilemma while maintaining the detection performance. Experimental results, both on synthesized images and on famous data sets have shown that our method outperforms Canny edge detector. BiGaussian edge detector successfully generates noiseless edge map, with edges at their exact position. Moreover, this method is computationally less expensive as it is non iterative and doesn’t search for extremas or for zero crossings. Acknowledgments This work was supported by Ministry of Knowledge Economy (MKE) through IDEC Platform center (IPC) at Hanyang University. Moreover, Ehsan and Jahanzeb were supported by ‘Higher Education Commission (HEC)’ from the Government of Pakistan under the scholarship program titled: MS level Training in Korean Universities/Industry.

References 1. Canny J (1986) A computational approach to edge detection, transactions on pattern analysis and machine intelligence 679–698 2. Marr D, Hildreth E (1980) Theory of edge detection London. Proc Royal Soc Ser B 207(1167):187–217 Feb 29 3. Basu M (2002) Gaussian-based edge-detection methods—a survey. IEEE Trans Syst Man Cybern 4. Torre V, Poggio T (1986) On edge detection. IEEE Trans Pattern Anal mach intell arch 8(2). March 5. Bowyer KW, Kranenburg C, Dougherty S (1999) Edge detector evaluation using empirical ROC curves, computer vision pattern recognition (CVPR ’99). Fort Collins, Colorado. June 6. Heath M, Sarkar S, Sanocki T, Bowyer KW (1997) A robust visual method for assessing the relative performance of edge detection algorithms. IEEE Trans Pattern Anal Mach Intell 19(12):1338–1359 7. Matthews J (2002) An introduction to edge detection: the sobel edge detector. http:// www.generation5.org/content/2002/im01.asp 8. Gonzalez RC, Woods RE (2010) Digital image processing, 3rd edn. Prentice Hall, Ohio 9. Juneja M, Sandhu PS (2009) Performance evaluation of edge detection techniques for images in spatial domain. Int J Comput Theor Eng 1(5). Dec 10. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proceedings of the sixth international conference on computer vision, pp 839–846 11. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the eighth IEEE international conference on computer vision 12. Tagare HD, deFigueiredo RJP (1990) On the localization performance measure and optimal edge detection. IEEE Trans Pattern Anal Mach Intell 12(12):1186–1190 Dec

154

E. U. Haq et al.

13. Huertas A, Medioni G (1986) Detection of intensity changes with sub-pixel accuracy using Laplacian-Gaussian masks. IEEE Trans Pattern Anal Mach Intell PAMI- 8(5):651–664 14. Argyle E (1971) Techniques for edge detection. Proc IEEE 59:285–286 15. Shin M, Goldgof D, Bowyer K, Nikiforou S (2001) Comparison of edge detection algorithms using a structure from motion task, IEEE Trans syst man cybernetics—part B: cybern 31(4). Aug

Self-Healing in Smart Distribution Networks Using Data Exchange Model and Reliable Multicast Protocol I. Dzafic, N. Lecek and T. Donlagic

Abstract Manual interventions are most common way of handling today’s power networks. Replacing manual interventions with a sophisticated algorithm which would be able to recognize the problem and solve it in an optimized time, is in the very near future, and will allow the network to be more flexible, and prevent network operators from making a mistake. The approach described in this paper shows that, by enabling data exchange between different substations, and by exploiting substations unused resources, it is possible to obtain a self-healing distribution network in which risks of manual interventions are reduced to minimum.



Keywords Self-healing Data exchange Distribution networks Control centers



 Substations  Network

topology



1 Introduction Distribution network has experienced a bloom in recent years thanks to the Smart Grid idea. Smart networks are the future core of the distribution power system, and therefore many investors and researchers are involved in its establishment. One I. Dzafic (&)  N. Lecek  T. Donlagic Siemens AG, Humboldtstrasse 59, 90459 Nuernberg, Germany e-mail: [email protected] N. Lecek e-mail: [email protected] T. Donlagic e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_15, Ó Springer Science+Business Media B.V. 2012

155

156

I. Dzafic et al.

important goal of Smart Grid is to introduce a network capable of healing itself without operator’s involvement, i.e. ‘‘self-healing network’’. The concept of selfhealing brings the network reliability and thus, high quality service to consumer. Some of the self-healing network approaches [1, 2] have already been tested and applied in real systems. Others are described through published papers [3, 4]. Reference [5] gives historical background and explains necessity of self-healing network. Reference [1] offers a very good review and examples of self-healing technologies present today. However, these approaches are currently developed only for specific smaller independent electric systems. Most existing approaches [6] describe the concept of agents based self-healing network in which intelligent agents, distributed over network, communicate and share information mutually reconfiguring the local system according to its needs. The approach is still theoretical and is under research at present. Solution described in this paper introduces model of self-healing in distribution networks that could be used as a general solution in future smart power systems. Substations local control centers (LCC) perform local network application calculations, configuration analysis and control and distribute results to neighbor LCCs. Data exchange model based on Pragmatic General Multicast Protocol (PGM) is applied as a communication channel between substations in case of a network configuration change. Detailed algorithm of the solution is described. The algorithm is being implemented at the moment, and first results are expected in the near future.

2 Distribution Network Topology and Control Distribution network has a major role in the power system. It supplies consumer directly and therefore its healthiness is of great importance. To keep the network running properly, one has to be aware of its configuration. Since distribution network is prominent to configuration changes, it is hard for the operator to keep track of all occurred events and handle the network correctly. As a consequence, very often, attempted manual interventions lead to a mistake, and the price in those cases is paid much more than losing expensive equipment or a profitable customer, it is paid with human lives. Allowing network reconfiguration operations to be executed automatically, can release network operators from most of their work. In such system they may need to take control only in very exceptional cases.

2.1 Present Distribution Network Control Today’s distribution networks are controlled by an operator who monitors and controls the network from the Control Center (CC). CC collects all data about the

Self-Healing in Smart Distribution Networks

157

current network state from the substations’ local control centers. SCADA (supervisory control and data acquisition), on the other hand, collects all field data, and serves LCCs. Based on the disposable data, operator, with the results of the distribution network applications (DNA) contributed by CC, makes an analysis, and when necessary, manually sends requests for configuration change in order to correspond to the new network state. This kind of network control obliges full responsibility to the operator in order to answer network problems. Sometimes, this leads to a time consuming tasks, and sometimes an error. Every circuit breaker operation is a risk of its kind, and should be thoroughly analyzed before deciding upon it.

2.2 Local Control Center as a DNA Solver Each network configuration change is followed by data exchange, first between field devices and LCCs, then between LCCs and CC. Having the network data, CC analyzes the complete network configuration. In this way, great amount of data is being processed and results in time consuming operations. Substation’s LCC has only one purpose—to enable data exchange between SCADA and CC. If LCC’s resources would be used for local DNA calculations as well, LCC would not have to exchange network configuration with CC, and thus CC would not have to handle the analysis of the complete network. Having LCCs to analyze only its network configuration would allow application of parallel computing on a large distribution power networks. However, to make this solution implemented in practice, another way of communication, between substations’ LCCs, has to be established. When a network configuration change occurs, affected substations have information about the change per se, but only for the area each is primarily supplying. In order to be able to understand new network configuration, affected substation should be notified about changes in neighboring substations as well. This requires careful design, implementation and deployment of communication software. Each LCC needs dedicated communication software to send data and requests to other LCCs and to receive data and requests from other LCCs.

3 Communication Patterns and Protocols In Smart Grid, LCCs are not monitored and controlled by central, global Control Center. From software architecture point of view, Smart Grid is decentralized, distributed system in which every node (LCC) communicates with neighboring LCCs. The distance between neighbor LCCs results in low network bandwidth and higher network latencies [2]. In case LCCs are inter-connected and communicate using Wide Area Networks (WANs) typical network bandwidth is in range 0.010–600 Mbps and network latency is 100–500 ms.

158

I. Dzafic et al.

The communication software should take into account the nature of such system. As there is no central server, client–server communication system is not an option. LCCs are communicating in similar way as peer-to-peer applications [7]. The key issue is how to figure out that other LCCs are available in neighborhood. Thus, appropriate communication protocol, that supports publish-subscribe communication pattern, is needed. OSI model [8] defines seven protocol layers. Applications use Application Layer (Layer 7) protocol. As name suggests, such protocols are used by application processes and are application specific. The protocols are layered. Higher layer uses only services of layer below. For example, in case TCP is chosen as Layer 4 (transport) protocol, it has impact on application level protocol as well. TCP is connection-oriented, reliable, ordered and bidirectional, unicast, transport protocol. User Datagram Protocol (UDP) [9] is an example of connectionless and unreliable transport protocol. UDP supports broadcast and multicast data delivery. For communication in Smart Grid solutions, transport protocol which combines following properties of TCP and UDP is needed: connectionless, reliable data delivery, ordered data delivery, send multicast, broadcast and unicast, unicast response. The issue considered in this paper is how to reliably and with high-performance deliver data from n data sources (LCCs) to m data consumers (LCCs) in case one application (one data source) sends data to n listeners (data consumers) dispersed within WAN. In cases more than one listener is involved, unicast solutions, based on reliable protocol such as TCP/IP protocol, are not practical. TCP/IP is connection-oriented, bidirectional, reliable protocol and transferring data from source to destination requires connection management. Furthermore, TCP/IP v4 does not support multicast and in case TCP/IP is used to deliver data to n listeners, the same data should be send over network n-times. This increases network traffic and leads to network problems such as network congestion and jitter. The net result of such naïve approach is degraded quality of service. Listeners process data with different performances, and in case of TCP/IP based data delivery, data provider has to handle these delays. This is possible to handle in case number of listeners is small and data amount transferred in data notifications are small. However, the ideal solution for data delivery to n listeners would be to use reliable multicast transport protocol that is connectionless, i.e. does not require connection management. User datagram protocol (UDP) is an example of connectionless protocol that support efficient multicast of data packets. With single data transmission, packet reaches members of multicast group. Considering only user data to be transferred, the data transfer of m bytes to n receivers over TCP/IP results in network transfer of (n 9 m) bytes, compared to m bytes over UDP. However, since UDP is unreliable protocol, packets could be lost, delivered out of order, or duplicated. The problem with direct usage of UDP in applications is that power applications normally require ordered and reliable data delivery. In case UDP is used as transport protocol and that applications must use reliable data delivery, these

Self-Healing in Smart Distribution Networks

159

Fig. 1 Communication between LCCs

Wide Area Network (WAN)/ Virtual Private Network (VPN) PGM

PGM

PGM

PGM

PGM

LCC 1

LCC 2

LCC i

...

LCC n

applications have to implement data handling that ensure reliable data delivery over unreliable transport protocol. This could be solved on application level and would lead to more demanding and error-prone application design. Alternative approach is to leave a task of reliable data transport to the transport protocol which supports reliable multicast. A number of transport protocols with multicast support have been specified and implemented in past two decades [10]. In absence of general solution for reliable multicast, proposed protocols are tailored to needs of specific application types, such as multimedia streaming applications, conferencing applications, file transfer, network gaming, and data distribution service (DDS) [11]. Although reliable unicast is a subset of reliable multicast, it should be noted that reliable multicast communication cannot properly replace unicast, point-to-point communication based on TCP/IP. For example, in cases of issuing control command execution requests, a bidirectional, point-to-point communication over TCP/ IP is safer and simpler to use. In case acknowledged delivery is not needed, reliable multicast protocol such as Pragmatic General Multicast Protocol could be used instead. Pragmatic General Multicast (PGM) is a reliable multicast transport protocol for applications that require ordered or unordered, duplicate-free, multicast data delivery from multiple sources to multiple receivers. PGM guarantees that a receiver in the group either receives all data packets from transmissions and repairs, or is able to detect unrecoverable data packet loss [12]. Features PGM provides satisfy requirements of transport protocol for selfhealing applications (Fig. 1).

4 Data Exchange Model for Smart Operations Algorithm, as in Fig. 2, makes an overview of the data exchange model for distribution networks. Detailed explanation of algorithm is described through following steps: (1) In planning mode, initial default network configuration plans are distributed over all substations’ LCCs either by CC either by distribution center. Each substation receives only configuration data of the network part it is

160

I. Dzafic et al. No

Begin No

Set init data for LLCs

Update LCCs with field measurements

LCCs init configuration changed?

Yes

Analyze new network configuration

New network elements?

Yes

Exchange data with LCCs previously in charged for new network elements

Apply new network state

New configuration change?

No

End

Yes

Fig. 2 Sample algorithm of data exchange model for power distribution networks

(2)

(3) (4)

(5) (6)

supplying. Normal-open points are used as a distinction between different suppliers.1 Assuming every network device’s initial status measured, SCADA sends the data to each substation’s LCC, and thus, initial network configuration is confirmed. In case of initial network configuration change, substation’s LCC is notified again through SCADA. New network configuration analysis is started. Switches statuses are being examined, and if their changed statuses caused network to be distributed differently then in its default state, substations’ LCCs have to be updated accordingly. Due to the fact that substations have only access to default network configuration, communication between different substations has to be established in order to start the information sharing process. After acquiring necessary data, new network state is recognized and set in each LCC. Each additional change in network state is followed by new data exchange between substations.

Example of distribution network, as in Fig. 3a, can be used for practical demonstration of the proposed data exchange approach. Four substations are supplying four independent configurations (C1, C2, C3, and C4 on Fig. 3a). Margin between configurations is presented with the dashed line. Substations work in parallel and serve as a backup one to another. Reconfiguring the default network such that CB2 changes its state, followed by state changes of DIS1 and DIS2, substations start analysis of the new configuration. LCC2 analysis shows that C2 now includes network elements that have previously belonged to C3. The data about these elements are not known to the LCC2, and therefore it has to request data exchange with LCC3. After the data exchange process, LCC2 can apply new network data in establishing stable configuration. If, for some reason, DIS4 needs to be closed, LCC1 has to first determine the status of DIS5 by communicating with LCC2. Only if DIS5’s status is set to open, LCC1 can close DIS4. Similar, when closing DIS3, the LCC that whishes to close it (LCC3 or LCC4) has to first exchange information about statuses of the disconnector DIS6 or CB4. 1

Each boundary normal-open point is included in all substations’ network configurations which are directly connected to it, so that both LCCs have insight in its state.

Self-Healing in Smart Distribution Networks

(a)

Substation S2

Substation S3

LCC 2

LCC 3

C1

C2

161

C4

C3

Substation S2

(b)

Substation S3

LCC 2

C1

COMM

C2

LCC 3

C4

C3

COMM COMM

CB 1NC

CB 2NC DIS1NO

Substation S1

S S2reserves < SA 1 CB 1NC

DIS2NC

LCC 1

CB 2NC DIS1 NO

Substation S4

DIS7NC

LCC 4

Substation S1

DIS2NC

Substation S4

DIS7 NC

LCC 1

LCC 4

A1

SS 4reserves > SA 2 DIS6NC

DIS6NC

CB4NC

CB4NC

DIS3NO

DIS3NO

SS1reserves > S A3 DIS4NO

DIS5NO

DIS4NO

DIS8NO

DIS5NO

A2

A3

Fig. 3 Part of distribution network suitable for data exchange model application. a Configuration is in its default state. b Solution after self-healing algorithm execution

5 Self-Healing in Smart Networks Data exchange model previously described requires network state pre-analysis by the operator. Operator is the one that, after network configuration change, based on the DNA applications results, can conclude about the possible new network state and apply switching procedure which leads to it. The switching procedure, which leads to data exchange algorithm execution, is a missing part that could make selfhealing networks possible. The point of self-healing process is to optimize the energy supply process in case of a network configuration change. Network adaptation after the configuration change is a redundancy dependent action. If the network has good linkage to other network configurations it is very likely that the network can withstand many different ways of supplying. Other attribute that makes network capable of selfhealing is power reserve. If there are enough power reserves in the system, network can support additional loads when required. Determination of power system reserves is an important process in preserving a stable network condition. System stays in a stable condition when power generation equals power consumption. Having an overproduction or underproduction, can make a system unstable and lead to a total system breakdown. Basically, system reserves are calculated as a difference between actual power consumption and maximum power generation in system:

162

I. Dzafic et al.

Fig. 4 Self-healing in distribution networks

Substation(s) which, due to network configuration change, need additional power from other substations

Substation(s) which could provide their power reserves to those in need

Start

Set init data

Set init data

Network configuration change

Network configuration change request

Analyze configuration changes

Analyze switching request

Unsupplied loads? Yes Determine switching procedures for alternate supplying

New network elements?

Send switching requests to competent substations

Yes Request data for new elements

Exchange data for new elements

Run DNA and calculate reserves No

Send information about available reserves Single substation can support unsupplied loads

Yes

No Repeat switching procedure request

Apply switching procedure

Determine switching procedure which involves reserves from multiple substations

No

Apply new switching procedure End

SgenerationMAX ðtÞ  Sconsumption ðtÞ ¼ Sreserves ðtÞ

ð1Þ

Sreserves(t) represent available system reserve at time t which can be injected if consumption demands grow. If Sgeneration(t) \ Sconsumption(t), there is an underproduction in the system. If Sgeneration(t) [ Sconsumption(t), there is an overproduction in system. Allowing substations’ LCCs control over the configurations they are supplying, together with the described data exchange model, one could achieve self-healing distribution network where LCCs take the main role, and CC and operator are no longer needed. One such algorithm is presented on Fig. 4. When a network change occurs, LCC, which supplies network part where the change occurred (main LCC), analyzes the change and checks if there are any unsupplied loads. If such loads exist, main LCC has to determine alternate routes for supplying these loads. Because of distribution network’s cohesion, alternate supplying routes are often found. Switching procedures which will allow new

Self-Healing in Smart Distribution Networks

163

routes to open is recognized by the main LCC. In order to establish new routes, some network elements, which are not under control of the main LCC, but by the LCCs of other substations, connected to main LCC, have to be manipulated. Therefore, switching procedures are sent to these substations. Data is sent to all LCCs in the network (PGM), but only those LCCs that are related to the message, accept the data. Upon acceptance each substation’s LCC analyzes the request, and recognizes if there are new network elements beyond its configuration. Requested LCC exchanges data with main LCC (using, for example, TCP/IP protocol) in order to have an insight in the complete network which its substation would supply in case of the execution of the requested switching procedure. Afterwards, state estimation is made and reserves are calculated. Requested LCC sends reserves information to main LCC. Main LCC identifies does any of the requested LCCs have a direct NO point to main LCC and have enough reserves to cover all unsupplied loads. If found, main LCC will confirm the switching request to the identified LCC, and the loads will be supplied successfully. If none of substations could fully support those loads, main LCC will still request the switching procedure to be made. In this case, the requested LCC will have to create its own switching procedure which would detach some of its regularly supplied loads, send a request to other substations for the newly created switching procedure and apply the switching procedure that was requested before. When applied, the substation that detached some of the loads now becomes main substation which is requesting others for their reserves. This is a sort of chain reaction which finally draws all network reserves in order to supply as many unsupplied loads as possible. Of course, this process has to be stopped if the solution converges too slowly or diverges. For example, if the satisfactory solution cannot be found after involving three additional substations, solution finding process is stopped. Considering the example from Fig. 3a, in case that CB2 trips, self-healing algorithm would apply following steps: 1. CB2 trips 2. LCC3 (main)—start analysis. 3. LCC3—set DIS2 and DIS6 to open. 4. LCC3—send requests: to LCC2—close DIS1; to LCC4—close DIS3. 5. LCC2 and LCC4—analyze switching request 6. LCC2 and LCC4—send request to LCC3 for missing network data 7. LCC3—exchange required data 8. LCC2 and LCC4—calculate reserves and respond to LCC3 9. LCC2 response: not enough reserves to support new elements 10. LCC4 response: enough reserves to support new elements

11. LCC3—repeat switching request 12. LCC4—close DIS3 13. LCC2—open DIS8 and close DIS5 14. LCC2—send request to LCC1—close DIS4 15. LCC1—analyze switching request 16. LCC1—send request to LCC2 for missing network data 17. LCC2—exchange required data 18. LCC1—calculate reserves and respond to LCC2 19. LCC1 response: enough reserves to support new elements 20. LCC2—confirm switching request 21. LCC1—close DIS4 22. LCC2—close DIS1

164

I. Dzafic et al.

After algorithm execution, network elements are supplied with reconfigured network as in Fig. 3b. Loads between DIS2 and CB2 are left unsupplied. Network is kept radial.

6 Conclusion Keeping the consumer satisfied with the constant power supplies is a major success for every network operator. With all the problems that distribution network is facing today, but yet with the opportunities that Smart Grid is offering, this task became a great challenge. Algorithm which would provide ability of the network to heal itself without the operator support is a progress which is still waiting to be implemented in large distributive networks. Background of one such algorithm was presented in this paper. It uses available distribution resources to locally control the system. Each substation makes DNA execution for the network it is supplying. In case of the configuration change it communicates and exchanges data with other substations in the network in order to reconfigure network so that it satisfies new network condition. Exploiting most of the distribution network resources would make network applications more efficient in their execution. Local control principle avoids unnecessary data exchanges, leaving room for other data which might need to be exchanged in future. Improvements gained with this approach would significantly change today’s solutions and the upcoming requests in distribution networks.

References 1. Li T, Xu B (2010) The self-healing technologies of smart distribution grid. In: 2010 China international conference on electricity distribution (CICED), 13–16 Sept 2010, pp 1–6 2. Rasmussen KS (2009) A real case of self healing distribution network. In: 20th international conference and exhibition on electricity distribution—part 1, 2009. CIRED 2009, 8–11 June 2009, pp 1–4 3. Samarakoon K, Ekanayake J, Wu J (2010) Smart metering and self-healing of distribution networks. 2010 IEEE international conference on sustainable energy technologies (ICSET), 6–9 Dec 2010, pp 1–5 4. Xiang G, Xin A (2011) The application of self-healing technology in smart grid. In: Power and energy engineering conference (APPEEC), 2011 Asia–Pacific, 25–28 March 2011, pp 1–4 5. Amin M (2001) Toward self-healing energy infrastructure systems. IEEE Comput Appl Power 14(1):20–28 6. Zhou HJ, Cao JZ, Guo CX, Qin J (2010) The architecture of intelligent distribution network based on MAS-SOA. In: Power and energy society general meeting, 2010 IEEE, 25–29 July 2010, pp 1–6 7. Colouris G, Dollimore J, Kindberg T (2005) Distributed systems concepts and design, 4th edn. Pearson Education Limited, New Delhi, pp 70–73. ISBN 0-321-26354-5

Self-Healing in Smart Distribution Networks

165

8. Zimmermann H (1980) OSI reference model—the ISO model of architecture for open systems interconnection. IEEE Trans Commun 28(4):425–432 Apr 9. Postel J (1980) ‘‘User datagram protocol’’, STD 6, RFC 768, USC/Information Sciences Institute, August 1980. Internet engineering task force (IETF): RFC768 user datagram protocol 10. Birman KP, Chen J, Hopkinson EM, Thomas RJ, Thorp JS, Van Renesse R, Vogels W (2005) dcOvercoming communications challenges in software for monitoring and controlling power systems. Proc IEEE 93(5):1028–1041, May 11. OMG available specification: data distribution service for real-time systems, version 1.2, formal/2007-01-01 12. Internet engineering task force (IETF): RFC 3208: PGM reliable transport protocol specification

Applying Event-Based Framework to Support Power System Integration Nenad Lecek and Izudin Dzafic

Abstract This paper describes a real life integration scenario of two power control systems. For that purpose we designed event-based framework and use it to create soft-real time gateway application service that facilitate integration of two complex, event-driven, distributed systems. The event-based framework uses publish-subscribe event delivery and supports asynchronous messaging, including asynchronous transactions. Extensible event-driven applications can be based on that framework. Event listeners and event type subscriptions could be added or removed at runtime. The framework simplifies application design and implementation because it solves concurrency related issues and access to shared resources by design—application developer does not need to deal with multithreading at all and still can implement flexible, multithreaded, thread-safe, event-driven applications.




Inter-process communication Keywords Asynchronous messaging Multithreading Publish-subscribe Message oriented middleware










1 Introduction The need to integrate distributed, event-driven, in-house developed power control system with external systems lead to considerations to connect systems by a Broker architecture [1]. The system had many Application Programming Interfaces (APIs) N. Lecek (&)
I. Dzafic Power Distribution Division, Siemens AG, Energy Sector, Humboldtstr 59, 90459 Nuremberg, Germany e-mail: [email protected] I. Dzafic e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_16,  Springer Science+Business Media B.V. 2012

167

168

N. Lecek and I. Dzafic

that provide data and control access to internal data providers. These interfaces were designed with specific application or (sub)system functionality in mind. Analysis showed that these APIs are too low-level for system integration purposes. Existing client APIs provide relatively narrow functionality and therefore lacked right level of abstraction required for integration usage. Interfaces on higher level of abstraction were needed. The problem is to orchestrate data received from these data providers, their interaction and provide consistent system views for such complex, event-driven and distributed system. The system needs to provide Application Programming Interfaces (APIs) that support system integration. These APIs external clients could use to communicate with the system. The external system could be considered as an external client. The Broker architecture has two main parts: Server-Side and Client-Side Broker. The Server-Side Broker is added to internal, event-driven, distributed system and is responsible to provide abstraction layer for external clients and to hide complexity of internal system. The Client-Side Broker library communicates with Server-Side Broker and provides consistent, minimal and event-driven interface to external clients. External systems use ClientSide Broker library to communicate with the Server-Side Broker. Many event-based frameworks already exist [2, 3] and are used in CORBA Event Service, JMS, WS-Eventing, WS-Notification and others. Make or buy decision depends on many factors. The key issue to address is how internal data providers and event-based framework will collaborate and support major functionality of ServerSide Broker: consistent data presentation to external clients. These issues are application and system specific and cannot be addressed with Component of the Shelf (COTS) packages. This leads to issues: for each data provider client is required to add bridge component that convert objects and methods calls from event framework to internal data provider client APIs. In order to cope with existing system complexity approach to concurrency is crucial. Events can and should be handled in parallel if their processing does not require access to the same shared resource. Concurrency and synchronization shall be addressed by design [4]. The challenge is to choose design that ensures solutions free of deadlocks, livelocks, race conditions, priority inversions and relieves developers from necessity to explicitly deal with multithreading. Active objects, described in [5, 6] are robust and relatively simple alternative to explicit thread programming. Event-based framework should ensure loose coupling between event listeners and efficiently dispatch events to registered event listeners. Thus, event-based framework implements Observer design pattern [7]. Furthermore, asynchronous communication between client and server-side Broker is needed to achieve service responsiveness [8]. This paper describes a simple and flexible event-based framework that serves as backbone for Server-Side Broker service and simplifies system integration. The paper is organized as follows: Sect. 2 (Motivation) describes problem to be solved and goals solution should achieve, Sect. 3 (Event-Based Framework) describes the key elements and design of created framework, Sect. 4 (Integration with External Systems) shows how event-based framework and Client-Side Broker were applied to address integration of two complex distributed systems and Sect. 5 (Conclusion) presents concluding remarks.

Applying Event-Based Framework

169

2 Motivation The desired solution for integration of internal distributed system with external systems had to be simple to use, user-friendly, reliable and with high performance. A gateway application service is needed. A concrete integration problem requires solving the following issues: (1) Internal distributed system is event-driven. A gateway application should also be an event driven service. The service should handle events as they arrived and should be able to react properly to events even if they come out of order, are late, duplicated or are lost. Components within a system work in parallel and share some resources, leading to possible race conditions. (2) Gateway service responsiveness. Internal distributed system is soft-real time system and gateway performance is very important. Latency should be minimized. (3) Service maintainability. Gateway application service must handle a number of special, complex message sequence scenarios with many similarities. New use cases and defect fixes can be expected during product maintenance phase. The challenge is to provide proper and flexible initial design that allows maintenance without side-effects. (4) Service extensibility. A maintenance or service extension should not require a complete system shutdown. Testability is important. Service modifications shall not require repeating complete system regression testing. Thus, important is to use testable design, e.g. use stable interfaces, interface versioning and cleanly isolate external dependencies. In such case automated unit tests are helpful, if kept up to date, and relatively small effort is required to execute updated regression tests. (5) Service usability. In gateway applications, the challenge is to ensure correct and consistent mapping between internal system context, internal data views and provide public, simple to use, symmetrical interface and data views that suffice clients needs. The internal distributed system is complicated. Data providers are distributed to different computers. To provide replies for relatively simple requests, communication with several data providers is typically required. E.g. simple data request to provide current values for couple of realtime measurement values require data from several data providers: security service check user authorization, name service provides textual descriptions for values and client accessing Shared Memory Mapped File (SMMF) from Supervisory Control and Data Acquisition (SCADA) provides real-time values. It is expected that internal system complexity will continue to increase in following years. Beside that, dispersed development teams are working on various subsystems for longer than decade. As a result of technological changes over these years, a number of implementation technologies and tools are in use. This,

170

N. Lecek and I. Dzafic

RTExplorer

GUIApp1

GuiApp2

Communication FrontEnd

SMMF Client API (C++)

COM server

Applicatio n Services

Network stack (TCP/IP, UDP)

SMMF

Database

DDS

Fig. 1 Simplified system component diagram

accidental difficulty had to be addressed too. Figure 1 shows simplified component diagram before integration with external system. Even simplified system view shows that direct integration is not practical solution. A client side component that directly accesses internal data providers simply needs to create far too many connections, handle many interfaces and maintain internal data caches and differences in implementation technologies. Such monolithic component would be too costly and too difficult to develop and maintain. A similar problem as integration with external system emerges in development of internal, Graphical User Interface (GUI) applications. Internal UI applications could directly contact data providers dispersed within complex distributed system. In such case, development and maintenance of GUI applications become unnecessary difficult and error-prone. Internal UI applications also suffer from system complexity. UI applications need to provide consistent data views and are in some cases forced to implement complex application logic to achieve that. From client’s point of view, integration solution has similar needs as internal UI applications. UI applications and external systems are considered as similar type of entities, as external clients. However, some differences exist, security for example. Authentication and authorization is different for UI application within the system compared to client application from external system. Except security considerations, similar set of interfaces are used for both purposes. These concerns justify the need to move the complexity of data retrieval and data forwarding into a separate layer. The missing layer is a Broker [1] and it finds place between internal data providers (internal services) and external clients, as depicted in Fig. 2.

Applying Event-Based Framework

RTExplorer PDS ClientLib

171

GUIApp1

GUIApp1

External System1

PDS ClientLib

PDS ClientLib

PDS ClientLib

External System2 ... PDS ClientLib

Presentation Data Service (PDS)

Communication FrontEnd

SMMF Client API (C++)

COM server

Applicatio n Services

Network stack (TCP/IP, UDP)

SMMF

Database

DDS

Fig. 2 Simplified system component diagram with Presentation Data Service (PDS) and external clients

A Presentation Data Service is a Server-Side Broker. It is event-driven and serves as Message Gateway, described in [9]. A Messaging Bridge [9] is needed to provide consistent data views and responsiveness between integrated systems. Presentation Data Service serves as messaging gateway application for external clients and UI applications. With gateway application service in place, internal complexity of distributed system is hidden from external clients. The presentation data service could be seen as single point of failure. It required flexible design that can cope with such problem. Sought after properties of such gateway application service are: • • • • •

Robustness Responsiveness Maintainability Extensibility Systematic approach to concurrency These properties influence service design. Applied design must be

• modular • extensible • relieve application developer/maintainer from need to deal with concurrency issues

172

N. Lecek and I. Dzafic message bus (Event Mediator)

Data Layer

Scada Plugin

DDS Plugin

NameService Plugin

IPC Plugin

other Plugins ...

Fig. 3 Presentation Data Service—plugin architecture

3 Event-Based Framework A number of services and expected changes in configuration and requirements make plugin architecture and middleware based on Messaging [9] an attractive choice for internal architecture of Presentation Data Service, as Fig. 3 shows. Considered option was to use event-based framework based on XML messages, SOAP, web services. This was abandoned for the following reasons: (a) parsing XML messages significantly degrade service performance; (b) required is low level layer interface between clients of internal data providers and message handlers called from callback of third party solution; (c) in system is introduced additional dependency on third party solution. Middleware technologies, such as CORBA Event Service are also abandoned to avoid adding complexity. To avoid overhead of complex middleware technologies and dependencies on external products, we designed and implemented relatively light weighted event-based framework in C++, with support for asynchronous messaging, flexibility of loosely coupled data providers, event-typed notifications and high performance event delivery. Publish-subscribe event notification combined with active object design pattern [4] form the basis for event-based framework applied in Presentation Data Service. Each plugin is specialized to handle communication with dedicated service and contain client APIs to access that service. Plugins could be added or removed. This way, application data layers (i.e. application logic and data cache) are decoupled from data providers. Concurrency is necessity in case responsiveness is important. Concurrency is often achieved using multithreading. The key concern related to multithreading is software maintainability. Explicit design of threading significantly decrease system maintainability and must be avoided for that reason. The problem with explicit design of threads is that does not scale well beyond trivial cases with few shared resources. Data providers and data caches are shared resources. It becomes especially difficult in case data providers or data caches belong to different layers. This may lead to considerations to use recursive mutexes. In case of many data providers, complex interactions cannot be avoided. Concurrency and synchronization shall be addressed by design. Events can and should be handled in parallel if their processing does not require access to the same shared resource. System organization and approach to multithreading is crucial.

Applying Event-Based Framework

173

IEventListener +Dispatch(evtMsg)

EventListener PluginX

1 .. n

1 .. 1 1 .. 1 ActiveObject

Event Processor PluginX

1 .. 1

ClientApi (access to internal service) Internal Data Pool

Fig. 4 Event-based framework—plugin class diagram

Publish-subscribe event notification combined with active object design pattern [1] form the basis for event-based framework applied in Presentation Data Service. External clients and UI applications are integrated using plugins. This application service is message oriented middleware. Application service is event driven, and efficient handling of event messages through event-based framework is desired. Plugins are loosely coupled. They communicate and exchange data only through event messages. Figure 4 shows plugin class diagram. Plugins use active object design pattern [5] to receive and consume event messages. Active object has thread safe queue and use worker thread (consumer thread) to process messages taken from that queue. A private event processor is assigned to every consumer. Event processor handles event messages specific to plugin functionality. In worker thread, queued event message are taken out sequentially, in FIFO order and given to event processor. Event processor does event message processing, usually as synchronous call. Event processor lives in consumer’s private thread, and has access only to own data pool and does not share or have access to data accessible from other threads. Each plugin implements interface IEventListener and registers for event messages types it wants to receive. Chosen subscription granularity is event message type. An implementation class, denoted here as EventListenerPluginX contains active object. Incoming event message is put into internal event message queue of active object. Active object’s private worker thread takes message from the queue and passes it to event processor. The message is interpreted there and requested operation is executed. On completion of message processing, event processor may want to post a message to inform other interested plugins. Therefore, event processor needs access to event mediator, which provides public, thread safe, re-entrant method to publish event message to subscribers. It is necessary to allow creating and publishing event message from private thread of active object. Thus, all plugins have access to event mediator. Plugin publishes event message to subscribers by calling EventMediator.Dispatch(eventMsg). All public methods of event mediator are thread-safe and reentrant.

174

N. Lecek and I. Dzafic IEventMediator +Dispatch(eventMsg) +Subscribe(evtType, evtListener) +Unsubscribe(evtType, evtListener) Dispatch(eventMsg)

Data Layer

Scada Plugin

DDS Plugin

NameService Plugin

IPC Plugin

other Plugins...

Fig. 5 Event-based framework—overview

The main reason to use event-based framework is to ensure clean, easy to use and highly responsive multithreaded solution and at the same time to solve multithreading issues by design. Important feature of designed event-based framework is that multithreading is hidden from application developer and this is achieved almost without performance penalties. Efficient event delivery assumes that event objects are not copied when passed from event producer to event consumers. This application service is implemented in C++ and copying is avoided by disciplined use of thread-safe smart pointers [10]. Efficient event-based framework has been designed as backbone for presented gateway service. Figure 5 shows overview of event-based framework used in Presentation Data Service. Internal data providers are accessed with proprietary APIs. Figure 5 depicts involved data providers: Data Distribution Service (DDS), Name Service, Supervisory Control and Data Acquisition (SCADA), Interprocess Communication (IPC) and Data Layer for internal application logic and data cache. EventMediator provides publish-subscribe interface for event delivery from one publisher to n subscribers. Event mediator implements Mediator and Observer design patterns 7. Event-based framework consistently uses asynchronous messaging, without blocking. Framework does not even offer functionality to block and wait for request completion. Synchronous, blocking requests are not used at all. This leads to issue how to map external request and response. Figure 6 shows Event Mediator used in Presentation Data Service. Event mediator supports asynchronous request/reply monitoring. In method Dispatch(), event mediator examines event message type and does bookkeeping for asynchronous request/response messages. Figure 7 shows sequence diagram for asynchronous request/response handling within Presentation Data Service application. If message is asynchronous request, a Correlation Identifier [9] is extracted from event message. This correlationId (id1) is added to OpenAsyncRequestList and event is published to subscribers. The purpose of CorrelationId is to hold unique mapping between asynchronous request and response. In case message is asynchronous response and CorrelationId is found in OpenAsyncRequestList the

Applying Event-Based Framework

175

IEventMediator +Dispatch(eventMsg) +Subscribe(evtType, evtListener) +Unsubscribe(evtType, evtListener) +RemoveListener(evtListener) +RemoveAllListeners() +AddTimer(timerDescription) +RemoveTimer(timerDescription)

1 .. 1 CEventMediator -AddAsyncRequest(CorrelationId) -RemoveAsyncRequest(CorrelationId); -RemoveAllAsyncRequests(); -bool AsyncRequestExists(CorrelationId);

1 .. 1

Open AsyncRequestList TimerList (contains a list of active timers)

1 ..n

Async CorrelationId

1 ..n

Timer Description

1 ..1 TimerThread

Fig. 6 Event-based framework—event mediator

EventProcessor

eventMediator

openAsyncRe questList

ipcPlugin

scadaPlugin

DoProcessing(evt)

createInternalEvent(evtReq1)

createInternalCorrelationId(id1) evtReq1.assignCorrelationId (id1, START_TRANS) Dispatch(evtReq1) isAsyncRequest() Add(id1) Dispatch(evtReq1) ReadScadaSMMF

createInternalEvent(evtResp1) evtResp1.assignCorrelationId (id1, END_TRANS) Dispatch(evtResp1) Remove(id1) Dispatch(evtResp1)

Fig. 7 Sequence diagram for asynchronous request/response

message is delivered to subscribers and CorrelationId (id1) is removed from OpenAsyncRequestList. In case message is asynchronous response and CorrelationId cannot be found in OpenAsyncRequestList the message is not forwarded to subscribers and is logged for later analysis. Such response message is either late or duplicate. Event mediator uses private timer thread to periodically check OpenAsyncRequestList and remove expired requests from the list. In case expired asynchronous request has been found, timer thread creates timeout response message and publish it to subscribers. This way requestor is also notified about request timeout.

176

N. Lecek and I. Dzafic

IpcPlugin

1 .. 1

IpcClient

1 .. 1

IpcMsgHandler

1 .. 1

1 .. 1

correlationIdSet + saveIpcCorrelationId(id, ipcId) + removeIpcCorrelationId(id)

eventMediator

0 .. n correlationIdMapper -CorrelationId -ipcCorrelationId

Fig. 8 IpcPlugin class diagram

External clients send requests to Presentation Data Service as asynchronous request/response IPC messages. For convenience of client’s application implementation, external clients use client-side Broker library (PDS ClientLib). This library can use blocking calls, i.e. block own, caller’s thread and wait for response, using Half-Sync/Half-Async [11] design pattern. Inter-process communication (IPC) between client and service is based on interprocess message queue. IpcPlugin is responsible to handle IPC communication and accept all asynchronous requests issued by external clients. The problem arise how to map client’s asynchronous request/response to internal handling of asynchronous request/response within Presentation Data Service. Figure 8 depicts class diagram for IpcPlugin and required bookkeeping classes. IpcClient handle low level IPC communication. IpcMsgHandler deserialize and process incoming IPC messages in private thread (IPC callback thread). Within IpcMsgHandler are created internal event messages and published to subscribers using event-based framework (Fig. 9). IpcPlugin receives and de-serialize IPC messages within IpcMsgHandler, in IPC callback thread (t1). External correlationId (IpcCorrelationId) is extracted from incoming IPC message. For tracing asynchronous transaction within eventbased framework, internal CorrelationId object is created. Internal CorrelationId is unique within Presentation Data Service. Data members of IpcPlugin are not accessible from IPC callback thread. This forces us to create event evtCorrelationId with mapping of external and internal correlation Id and publish it to IpcPlugin. Event evtCorrelationId contains internal CorrelationId, IpcCorrelationId, transactionState, transactionStartTime, and transactionDeadline. After evtCorrelationId is published, internal CorrelationId is assigned to internal event message and event is published to event mediator. IpcPlugin is subscribed for asynchronous reply event message. In worker thread of IpcPlugin is received event message asyncResponse and executed search through set of mapped correlation IDs. If

Applying Event-Based Framework IpcMsgHandler

177

eventMediator

openAsyncRe questList

ipcPlugin

scadaPlugin

IPC msg (async request)

DecodeMsgBuffer createInternalEvent(evtReq1) createCorrelationId(id1) Dispatch(evtCorrelationId1) Dispatch(evtCorrelationId1) evtReq1.assignCorrelationId (id, START_TRANS)

saveIpcCorrelationId

Dispatch(evtReq1) isAsyncRequest() Add(correlationId) Dispatch(evtReq1) ReadScadaSMMF

Dispatch(evtScadaValues) Dispatch(evtScadaValues1) Remove(correlationId) findIpcCorrelationId Serialize(evtScadaValues1) SendResponse(ipCorrelationId, evtScadaValues1) removeIpcCorrelationId

Fig. 9 Mapping client asynchronous transaction to internal asynchronous request/response

correlationId is found within correlationIdSet, IpcCorrelationId is known and client queue that receive response is identified. In case transaction state is received as completed, correlationID info is removed from correlationIdSet. Otherwise, in case correlationID cannot be found within correlationIdSet is logged for later analysis and message is not delivered to client. Periodical timer event triggers checking whether asynchronous request expired. If correlationId is expired, it is removed from correlationIdSet and timeout event message is published. The requestor is a registered listener for such timeout event message and instead of response it receives timeout notification message with correlationId from request message.

4 Key Design and Implementation Issues The event message granularity is one of the most important aspects of asynchronous messaging. Fine grained event message types require increased number of events to be handled and lead to complicated interaction between them. As a consequence application logic could be difficult to follow. In some cases this lead to decision to model application logic with state models although behavior is essentially stateless. Another extreme could be too coarse event types. Such event messages typically contain complex data structures that are difficult to use, and complex and time consuming to serialize and de-serialize. Experience proved that

178

N. Lecek and I. Dzafic

is better to start with small number of events and keep them symmetrical. E.g. in case exist event message type EVT_OPEN, then EVT_CLOSE should be also provided. The exact level of required event granularity and data members associated with event message is domain specific decision. In absence of exact method to guide this decision, good taste and experience is needed. Symmetry, no duplicated event types, easy understandable event purpose from event name indicates appropriate level of event type granularity. Important impact on efficient event delivery has copy policy for event messages. The framework should not copy event messages while dispatching event messages from event producers to event consumers. Copying complete event messages would degrade performance and require event sizing policy. Our framework solves these issues with disciplined usage of thread-safe, reference counted, immutable smart pointer objects to event messages. Event producers create and dispatch smart pointer object to framework and framework forwards such object to event consumers. Smart pointer objects are typically much smaller than event message objects and introduce very small overhead during event delivery. Smart pointer objects ensure efficient event delivery and proper resource management of event messages. Event mediator has additional task, i.e. it handles timers. Timers in this eventbased framework are asynchronous and non-blocking. The timer thread periodically checks for expired timeouts and publish timeout events for expired timers. In case timer expired and is one-shot timer, it is removed from TimerList. In case of periodical timer, timeout event is delivered and timer is rearmed. Here is taken special care that timers are not drifting. These periodical timeout events are used to trigger processing at regular intervals, such as polling of some data provider, client heartbeat checks and periodical calculations.

5 Integration with External Systems Integration with external systems is supported through IpcPlugin. IPC communication is used for performance reasons. External clients communicate with Presentation Data Service using Inter Process Communication (IPC), i.e. using inter-process message queues. External clients establish point-to-point communication with Presentation Data Service. Presentation Data Service uses IPC plugin to read incoming messages from own, dedicated IPC queue, denote it PDS queue. IpcPlugin sends responses and event notification messages to external clients into their queues. External clients are expected to support following communication tasks: • Establish connection to IPC queue (PDS queue). • Periodically send heartbeat message to PDS queue so that service knows that client is still alive. • Handle reconnect requests from client side.

Applying Event-Based Framework

179 External Client1 PDS ClientLib

Presentation Data Service

IpcPlugin

Other Plugins ...

External Client2

IpcDaemon (holds IPC queues)

1 .. 1

PDS queue

PDS ClientLib

1 .. n

Client Queues

Fig. 10 Communication between PDS and external clients

• Serialize and efficiently send messages to PDS queue. • Create own queue to receive event response and notification messages. • Parse received event response and notification messages and forward them to callback objects for further processing. • Provide methods to send message using SendNotification(), SendSynchronousRequest() or SendAsynchronousRequest(). • Cleanly close connection. Every external client (and UI client) should handle these tasks. In order to avoid redundant implementations, a client-side Broker library (PDS ClientLib) was designed and implemented. This library implements communication to server-side Broker (PDS) and provides C++ interface to send and receive data using application level event messages. Figure 10 depicts communication between presentation data service and external clients. External clients use PDS ClientLib library to communicate with Presentation Data Service. PDS ClientLib handle details of IPC communication and offers convenient C++ interface to external clients. To send request to PDS, external client need to create message object and pass it to the library. The library does object serialization and place serialized message buffer into PDS queue. In response direction similar activity take place. IpcPlugin serialize response message and place it to client’s queue. PDS ClientLib provides notification callback. This callback is called in case IpcPlugin sends notification (fire and forget) messages to client. IPC communication uses proprietary protocol to avoid performance loss in serialization and deserialization of open protocols based on XML messages. In case other protocols should be supported, event-based frameworks allows to extend Presentation Data Service with plugins providing these protocols (e.g. open XML protocols, such as SOAP, WS-* and others).

180

N. Lecek and I. Dzafic

6 Conclusion Integration of one complex distributed system with another system is difficult. Perhaps the most important factor is to provide usable and simple interface to external clients. The internal system complexity is hidden behind gateway application service designed for integration purposes. The main tasks of such service are to provide consistent data presentation and simplify communication with internal system at the same time. In this paper it is described how event-based framework was used to create flexible, robust soft-realtime presentation data service. The central building blocks of presented eventbased framework are publish-subscribe event message delivery combined with active objects. Publish-subscribe event delivery is a key element to achieve high cohesion and loose coupling for data providers. A robustness of application service is achieved by moving data providers into active objects. Active objects have very minimal interface and such narrow interface contribute to service testability. Data providers are cleanly separated from the rest of the service. Such separation addresses service robustness and also testability. Valuable property of event-based framework is possibility to use dummy data providers (stubs). Furthermore, clean threading model was applied. Data providers are visible only within worker threads of active objects and they do not have access to shared resources. Application logic is considered as internal data provider and placed into one plugin. The benefit of such design is that service developers do not need to deal with multithreading and can still create, multithreaded, highly responsive service. Asynchronous messaging is chosen to assure service responsiveness. At the same time, convenience of client side programming and integration is provided by simple interface of client library. Presented event-based framework is in production use and provides required functionality and flexibility.

References 1. Buschmann F, Henney K, Schmidt DC (2007) Pattern oriented software architecture: a pattern language for distributed computing, vol 4. Wiley, Chichester. ISBN-13: 978-0-47005902-9 (hbk) 2. Huang Y, Gannon D (2006) A comparative study of web services-based event notification specifications. In: ICPP workshops, IEEE Computer Society, pp 7–14 3. Henning M (2006) The rise and fall of CORBA. ACM Queue 30 June 2006, pp 29–34 4. Samek M (2002) Practical statecharts in C/C++ quantum programming for embedded systems. CMP Books. ISBN 1-57820-110-1 5. Lavender RG, Schmidt DC, Active object—an object behavioral pattern for concurrent programming. In: Proceedings of pattern languages of programs. http://www.cs.wustl.edu/ *schmidt/PDF/Act-Obj.pdf 6. Petitpierre C, Eliëns A (2002) Active objects provide robust event-driven applications, SERP’02 7. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: elements of reusable object-oriented software. Addison-Wesley, Reading

Applying Event-Based Framework

181

8. Zdun U, Voelter M, Kircher M, Design and implementation of an asynchronous invocation framework for web services, conference: ICWS-Europe’03 9. Hohpe G, Woolf B (2008) Enterprise integration patterns, designing, building and deploying messaging solutions. Addison-Wesley, Boston, ISBN: 0-321-20068-3, January 2008, 11th Printing, pp 163–169, 472 10. Boost web site. Smart pointers. http://www.boost.org/doc/libs/1_47_0/libs/smart_ptr/ shared_ptr.htm 11. Schmidt DC, Stal M, Rohnert H, Buschmann F (2000) Pattern-oriented software architecture: patterns for concurrent and networked objects, vol. 2. Wiley, Chichester. ISBN 0-471-60695-2

Utilizing Converged Networks for Green Network Operations Xiaodong Yang

Abstract The Converged network is the trend of network development. It can not only provide advanced new features to the end users, but also provide a novel way of green computing to the network operators. This paper presented the idea of utilizing converged networks for green network operations. The converged network based solution is presented and the benefits of the proposal are discussed. Cobb–Douglas model is for the first time used for analyses and the results show that the converged networks consume less network operation resources compared with non-converged networks and converged networks also perform better when the network operation scale is increased. Some open research issues and the future work are further discussed in the conclusion part of this paper. Keywords Converged network

 Green computing  Network operation

1 Introduction Green computing has already become a hot research topic recently. The word ‘‘green’’ here usually means ‘‘low energy, low carbon’’. Most of the research works being done are focusing on data center power efficiency [1], network operation algorithms [2], etc. Few research works are done for designing telecom network operations. Besides, green computing refers not only to consumed operation power reduction, but also to the economic use of operation construction materials and other network resources.

X. Yang (&) School of Information and Electronic Engineering, Zhejiang Gongshang University, Hangzhou, China e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_17,  Springer Science+Business Media B.V. 2012

183

184

X. Yang

Fig. 1 Converged networks of DVB-H and UMTS

Therefore, this paper will, for the first time, present and analyze a novel approach for green network operations-utilizing the converged networks. During the analysis, Cobb–Douglas model is for the first time used for green network operations. Converged networks refer to the networks where different kinds of networks are converged and the services or infrastructures are somehow shared and optimally co-utilized. In another word, converged networks make fully use of a common network infrastructure to serve the operation functionalities of two or more different networks. For example, the converged broadcast and telecom networks are making fully use of a common IP core network and the related service infrastructures, thus saving operation cost and reducing power/cooling energy consumptions. In this sense, even the network construction materials can be used more economically and the relevant costs are thus saved. On the other hand, the converged networks have for a long time been studied and proposed to provide users with high speed services and more advanced interactive features efficiently and economically [3]. As an example, a typical converged network between DVB-H and UMTS is shown in Fig. 1 [3], where DVB-H is the European standardized mobile broadcast standard and UMTS is one of the Third generation mobile telecommunications standards. In Fig. 1, the converged network coverage is an overlap cellular structure where the DVB-H cells and the UMTS cells are overlapping each other. The service head-end part is the same for both DVB-H and UMTS networks. And the two networks also share a common IP core network. The terminal used in the converged networks is also a kind of converged terminal thus saving the cost on user devices by avoiding using two different devices. This paper will focus on the green network operation features provided by the converged networks.

Utilizing Converged Networks for Green Network Operations

185

The converged networks could be composed of many different kinds of networks. For easy description and without loss of generality it is assumed in this paper that the converged networks are converged DVB-H/UMTS networks. This paper tries to exploit the green features of the converged networks. The solution of utilizing converged networks for green network operations are proposed and analyzed. The green features here mean that the network operation resources such as power energy, network construction materials, man power, etc. are consumed much less thus producing ‘‘green’’ effect. This paper is organized as follows: Section 2 describes the research backgrounds for green computing solutions. Section 3 presents the proposed novel solution of utilizing converged networks for green network operations. Section 4 focuses on the analyses of the network operation resource consumptions and the results are obtained by using Cobb-Douglas model and relevant comparisons are made. Section 5 concludes the paper and also describes some open issues and the future work.

2 Research Background For some time, people have been only interested in devoting the efforts to increase the processing power of equipment and inventing algorithms to realize it. The resources consumed by the equipments such as power energy and construction materials have for a long time been assumed being always available and affordable. However, with the idea of low carbon economy and especially with the establishment and development of data centers, people begin to change their minds. And for this reason, green computing has become a very hot research topic. Nevertheless, most of the previous research works about green computing were only focusing on the energy consumption reduction in data centers. This is, of course, partly due to the considerable power consumptions of the data centers and partly due to the still growing power demand of the larger data centers [4]. As it is already said that the solutions for green computing were first developed to address the power consumption issues in data centers, we have to make it clear that the technology behind the enormous power consumption is mainly cloud computing. Cloud computing is the collection of scalable, virtualized resources, which is capable of hosting applications and providing required services to the users and can charge on a per-user basis [5]. Cloud computing makes use of hundreds and thousands of computers to run parallelly thus consumes considerable amount of power energy for processing and cooling [6]. And this kind of energy consumption cost due to power usage for processing and cooling usually accounts for a great percent of the whole cloud computing operation cost. Thinking from another side of the data center network, paper [7] presented the thin clients and their features of being environmentally responsible than desktop computers most obviously regarding to power consumptions. Moreover, paper [7] mentioned that thin clients have far less parts than a desktop computer and therefore require also far less resources to manufacture and contain far fewer

186

X. Yang

hazardous materials to the surrounding environment. In this respect, paper [7] introduced other resources consumption issues besides the popular power consumption issue. Again regarding to data centers, paper [2] divides the costs of data centers into different categories such as servers, infrastructures, powers and networks. It proposed approaches to reduce these different categories of costs. It also argued that networking and systems innovation could be the key to reduce data center costs. Paper [8] pointed out that the energy expense per unit of computation has location dependent features, in spite of the fact that energy expenses are becoming an increasingly important fraction of data center operating costs. Therefore, as complementary to the conventional energy reducing approaches, paper [8] proposed to use energy cost-aware routing algorithms to reduce the power consumption cost of data centers by exploiting the variations due to fluctuating electricity prices. As presented above, most of the previous research works are addressing the issues of resource consumptions regarding to data centers. In this paper, it is argued that the issue regarding to data centers is not the only research issue of green computing, though the application to data centers is indeed one of the important application areas. By arguing this, this paper tries to point out that the converged networks also match very well to the idea of green computing by saving different network operation resources such as network infrastructure cost and network operation cost compared to the non-converged networks. The converged network itself is not a new idea. Converged networks provide a lot of advanced features to the existing services. Actually this is exactly what converged networks are designed to be at the beginning [9]. And for this reason, most of the research works regarding to converged networks are focusing on the services provided by converged networks and the improved converged network infrastructures. For example, paper [3] addressed the intersystem soft handover issue in the converged DVB-H and UMTS networks. However, no research works are done to exploit the converged networks for green computing up to the writing of this paper. Exploring the green features of converged network operations is a novel issue. This issue will surely expand the domains of green computing research and provide another incentive for adopting the converged networks. The details are presented in the following sections.

3 The Proposed Solution Exploring the features of green computing in converged network operations is a novel issue. Regarding to the converged DVB-H and UMTS networks, the proposed solution can be illustrated in Fig. 2. Figure 2 takes the converged broadcast (DVB-H) and telecom (UMTS) networks as an example. However, the proposed solution can be easily extended to other converged networks, in spite of the fact that most of the following aspects are specific to the converged broadcast

Utilizing Converged Networks for Green Network Operations

187

Fig. 2 Illustration of green features in converged networks

and telecom networks. The different green computing features of the converged networks are shown in Fig. 2.

4 Analyses and Results In this section, the comparison between the network operation resource consumptions for converged networks and non-converged networks will be compared. First, the relationship between network operation resources consumptions as output and related different variables as inputs will be estimated. The network operation resources here mean processing power and cooling energy, construction materials, man power, etc. and can be divided into fixed and variable resources consumptions. Fixed consumptions are not sensitive to the change of the volume of input network resources, but variable consumptions do. It is also reasonable to believe that the variable resources consumptions are affected by typically the following different variables: distance of the transmission, type of network, operator strategy. • Distance of the Transmission: No matter the network transmission is wire based or wireless based, the longer the transmission distance is, and the more transmission power will be consumed. Longer distance also means more network construction materials being consumed and more man power being used for repair and maintenance.

188

X. Yang

• Type of Network: Normally wired networks consume more resources than wireless networks. Moreover, different wired or wireless networks consume different resources. For the same service, different network modes such as unicast and broadcast consume also differently. Even for the same kind of network, different services on the network will consume different amounts of resources. For example, video services consume more resources than voice or text based services. • Operator Strategy: Each network operator usually has its own network operation strategies based on the company policy, which may lead to differences in network operation resources consumptions. For example, the different network technologies and standards being used, clean or non-clean power being used, even different network equipments being used could mean different resources consumptions. In order to make the analyses quantitatively, the above three input variables as parameters will be expressed as follows: • D: Distance of the transmission, DC refers to the value for converged networks and DN refers to the value for non-converged networks. • T: Type of network, furthermore, TC refers to the value for converged networks and TN refers to the value for non-converged networks. • O: Operator strategy, furthermore, OC refers to the value for converged networks and ON refers to the value for non-converged networks. For easy mathematical calculations and without loosing generality, the value of D, T, and O can be defined and to be set in the range from 0 to 1 depending on the resources consumptions of the network operations caused by the variation of the different corresponding variables. The less the network operation resources are consumed, the smaller the value D, T and O will be. Based on the descriptions in Sect. 3, for converged networks, statistically the following equations exist: DC hDN ; TC hTN ; OC hON : Suppose R is the total network operation resources consumptions, R can be calculated by using the following equation: R ¼ b0 þ b1 D þ b2 T þ b3 O:

ð1Þ

Where b0 is the fixed network operation resources consumptions; D is parameter for the network transmission distances; T is the parameter for the type of networks; O is the parameter for operator strategies. The coefficient bi (i = 1, 2, 3) is the elasticity of network operation resources consumptions with respect to different independent variables D, T and O. It shows the percentage of change in total resources consumptions resulting from a 1% increase. Let RC refer to the network operation resources consumptions of converged networks and let RN refer to the network operation resources consumptions of nonconverged networks, then we have the following equations:

Utilizing Converged Networks for Green Network Operations

189

RC ¼ b0 þ b1 DC þ b2 TC þ b3 OC :

ð2Þ

RN ¼ b0 þ b1 DN þ b2 TN þ b3 ON :

ð3Þ

By using DC hDN ; TC hTN ; OC hON : (2), and (3) it is easy to get the following: RC hRN :

ð4Þ

From (4) it is easy to see that the converged network operation resources consumptions are less than those of the non-converged networks. In this way, converged networks produce ‘‘greener’’ effects. Applying simple mathematical transformations to (1), the following equation is obtained: e R ¼ e b0 e D


b1

eT


b2

eO


b3

:

ð5Þ

:

ð6Þ

Again, (5) can be transformed to (6) as follows: e R ¼ e b0 e D


b1

eT


ðb2 þOTb3 Þ

Equation (6) is obviously a best fit to the Cobb–Douglas model function. Cobb–Douglas function [10] takes the form as follows: PðL; K Þ ¼ bLa K b :

ð7Þ

Where P is total production, L is labor input, K is capital input, b is total factor productivity, a and b are the output elasticities of labor and capital. And if a þ bh1:

ð8Þ

then the return to scale is decreasing. In another word, with increasing of the input, the output is decreasing. Otherwise, the return to scale is constant or increasing. Cobb–Douglas functions are production functions and are widely used to represent the relationship between an output and inputs, like vehicle operation cost analysis [11]. It is reasonable to assume that the Cobb–Douglas model can be used for the network operation resource consumption calculations. Specifically the Cobb–Douglas models for converged networks and nonconverged networks with input variables D and T can be expressed as follows: RC

e

¼e

b0

e


DC b1

e

TC

 
b2 þOTCC b3

:

ð9Þ

 
b2 þOTNN b3

:

ð10Þ

.

eRN ¼ eb0 eDN


b1

e TN

190

X. Yang

Regarding to the network type parameter T, for easy comparisons, reasonably suppose TC = TN, then according to (3), b1 þ ðb2 þ

OC ON b Þhb þ ðb2 þ b Þ: TC 3 1 TN 3

ð11Þ

As R is proportional to eR, According to the general Cobb–Douglas function model (7), (8) and network resource consumptions model (9), (10), (11), the return to scale feature of RC is comparatively decreasing than that of RN. This means, with the proportional increase of input variables such as D, the consumed resources for converged network operations RC are increased with less proportion than that of non-converged networks RN. In another word, with the increasing of the network operation scale, the converged networks perform better than nonconverged networks regarding to network operation resource consumptions. This result shows that the converged networks not only consume less network operation resources than non-converged networks, but also have ‘‘greener’’ effects when the network operation scales are increased. It is to be noted that (9) uses eD and eT as the input variables. If eD and eO or eT and eO are used as input variables, similar results can obviously be obtained. The analyses above are based on the theoretical calculations. Real network operation data are needed to further verify the feasibility of the network operation resource consumption models being used. In spite of a lot of further work regarding to the analyses and test are planned, the results obtained from the analyses here in this section have already showed that the converged network operations are ‘‘greener’’ than non-converged network operations.

5 Conclusions Telecom operators consistently rank among the top three electricity consumers nationally in most EU countries [12]. To keep ahead of technology development and user service requirements, telecom operators are again building and expanding costly new networks, such as 3G, even 4G networks. On the other hand, the converged broadcast and telecom networks utilize most of the existing network infrastructures and are capable of providing similar user services. However, the features of green computing of the converged networks are rarely discussed. For this reason, this paper raised the issue of green network operations and proposed a way of utilizing the converged networks to address such issue. The analyses based on Cobb-Douglas model are first used for the proposed solution. The results showed that the network operation resources consumptions in converged networks are less than those of non-converged networks. And with the increasing of the network operation scale, the converged networks perform better than the non-converged networks. Further evaluation of the Cobb-Douglas model for the network operation resource consumptions using real data from operators are needed.

Utilizing Converged Networks for Green Network Operations

191

Lab tests and field trials are needed and hopefully the operators could plan and integrate the solution into their commercial operations in the near future. Acknowledgments This work was supported by Zhejiang Qianjiang Scholars Program D of China (Grant No. QJD1002004) and Zhejiang Provincial Natural Science Foundation of China (Grant No. Y1111103).

References 1. Liang L, Hao W, Xue L, Xing J, Wen Bo H, Qing Bo W, Ying C (2009) Green Cloud: A new architecture for green data center. In: Proceedings of the 6th international conference industry session on autonomic computing and communications industry session, pp 29–38 2. Greenberg A, Hamilton J, Maltz DA, Patel P (2008) The cost of a cloud: research problems in data center networks. ACM Sigcomm Comput Commun Rev. 39(1):68–73 3. Xiaodong Y, Owens TJ (2008) Intersystem soft handover for converged DVB-H and UMTS networks. IEEE Trans Veh Technol 57(3):1887–1898 4. Kurp P (2008) Green computing. Commun ACM 51(10):11–13 5. Manish P, Jong SP (2009) Cloud computing: future solution for E-governance, In: Proceedings of the 3rd international conference on theory and practice of electronic governance, pp 409–410 6. Weiss A (2007) Computing in the clouds. netWorker 11(4):16–25 7. Ritschard MR (2009) Thin clients: make them work for you. In: Proceeding of the ACM SIGUCCS fall conference on user services conference, pp 41–45 8. Asfandyar Q, Rick W, Hari B, John G, Bruce M (2009) Cutting the electric bill for internetscale systems. In: Proceedings of the ACM SIGCOMM 2009 conference on data communication, pp 123–134 9. Xiaodong Y (2008) Handover in DVB-H—investigations and analysis. Springer, Heidelberg, ISBN: 978-3-540-78629-0 10. Tan BH (2008) Cobb–Douglas production function. Teaching Materials, Carnegie Mellon University 11. Maryam H (2004) Operating costs for commercial vehicle operators in Minnesota. Thesis, University of Minnesota 12. Diala A, Melissa B, Nancy B, Davide Va (2009) Telecom operators in a carbon constrained world, Arthur D. Little report (2009)

A Ring and Clustering Routing Scheme Based on Data Field in Sensor Network Wenwen Dai, Kaihua Xu, James J. (Jong Hyuk) Park, Naixue Xiong and Yuhua Liu

Abstract This paper proposes a routing algorithm named RCBDF (Ring and Clustering Based on Data Field).The algorithm uses the concept of physical field to form clusters on the ring network model. The routing algorithm mainly considers the remaining energy of nodes and the coverage of the cluster to balance energy consumption in network. Simulation analysis shows that: the routing policy will balance the energy consumption of the network at a large extent and greatly extend the network lifetime.







Keywords Wireless Sensor Networks Clustering Data Field Communications

W. Dai (&)  Y. Liu Department of Computer Science, Huazhong Normal University, Wuhan, China e-mail: [email protected] Y. Liu e-mail: [email protected] K. Xu College of Physical Science and Technology, Huazhong Normal University, Wuhan, China e-mail: [email protected] J. J. Park Department of Computer Science and Engineering, Seoul National University of Science and Technology, Seoul, Korea N. Xiong Department of Computer Science, Georgia State University, Atlanta, USA e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_18, Ó Springer Science+Business Media B.V. 2012

193

194

W. Dai et al.

1 Introduction The wireless sensor network is a self-organizing network system, which is formatted by multiple hops radio communications among the tiny sensor nodes. Those sensor nodes are deployed in the monitoring area to perceive collection and dispose the information from the monitoring objects in the network, and send the information to observer [1]. Because of the sensor nodes have restrictions due to energy and communication ability, the research on routing technology on the network layer is pivotal in the architecture of wireless sensor networks. In very large scale monitoring sensor networks, clustering technique has been proposed to address routing scalability and energy conservation issues. LEACH is a milestone of hierarchical routing. However, it will shorten the network lifetime, which because of it does not take into account the current residual energy of the candidate node when select cluster head. Meanwhile, LEACH doesn’t consider the manner of non-cluster head to join the cluster and the communications among clusters. All that is likely to cause uneven cluster and easily result in death of cluster head far from base station due to consumption of excess energy. So, this article proposes a ring and clustering routing algorithm based on the data field to balance the energy consumption and prolong the network lifetime.

2 Related Work There are many literatures about the optimization target of WSN energy efficiency. It is proved [2] that the non-uniform energy consumption of nodes is inevitable under the conditions of the non-uniform node placement and data reporting continuous in multi-hop WSN. Previous work, such as [3] and [4] pointed out that it could improve energy efficiency in network by hierarchical node deployment and data fusion. [5]’s studies focus on energy efficient clustering algorithm in distributed heterogeneous WSN. The elected cluster head node is based on the ratio of the residual energy and the average energy of the nodes in network. It makes the nodes have high initial energy and residual energy and have more chance to become a cluster head. It can balance the network energy consumption and prolong network lifetime. However, this approach has a flaw: all the remaining energy of nodes will reduce when the network running for some time, and it leads to the number of elected cluster head node becomes less and less for each round. Eventually leading to unbalanced network energy consumption and reducing the network lifetime greatly. Therefore, this paper analyzes this problem and uses data field theory to form cluster on the ring network model. It takes into account not only the global residual energy but also the coverage of clusters and distance among nodes. Meantime, it uses Dijkastra algorithm to establish the shortest path tree at data communication phases to eliminate the link cost by single-hop communications.

A Ring and Clustering Routing Scheme

195

Fig. 1 The ring base network model

3 Network Model and Problem Description 3.1 Network Model Nodes deploy in a circular area of radius R randomly. This paper divides the monitoring area into S rings evenly to facilitate the establishment of multi-hop path. The center of the ring is the sink node and the width of each ring is h (units of meter, the same below), shown in Fig. 1. This article assumes that the sensor network has the following properties: (1) the sensors will work independently and be stationary once deployed; (2) the width of each ring is same; (3) each node has a unique identifier ID; (4) transmission strategy is so perfect that there is no conflict and retransmission; (5) all of the nodes have simple data fusion capabilities and there always have data to sent.

3.2 Energy Model The Literature [6]’s energy consumption model is used for the analysis and simulations throughout the rest of this paper. Thus, to transmit k-bit message a distance d, the radio expends:  kEelec þ kefs d2 d\d0 ð1Þ ETx ðk; dÞ ¼ ETelec ðkÞ þ ETxamp ðk; dÞ ¼ kEelec þ keamp d 4 d  d0 and to receive this message, the radio expends: ERx ðkÞ ¼ ERxelec ðkÞ ¼ kEelec

ð2Þ

The electronics energy, Eelec , depends on factors such as the digital coding, modulation, filtering, and spreading of the signal, eamp and efs are amplifier

196

W. Dai et al.

magnification, k is the number of bytes by sensor node sent or received, d is the distance between the transmitter and receiver. d0 is the threshold of the distance: if the distance is less than the threshold, the free space model (d2 power loss) is used; otherwise, the multi-path model (d4 power loss) is used.

4 Analysis of RCBDF 4.1 The Value Selection of Threshold LEACH protocol does not take into account the node’s residual energy and makes the selected cluster head may not be the optimal cluster head. An approach to the solution of this problem is presented by literature [7], it adjusts the value selection of threshold as formula (3) shows: TðnÞ ¼

1

p ðr

p En  mod 1=pÞ En

residual ðrÞ average ðrÞ

ð3Þ

Where En residual ðrÞ denotes the current residual energy of nodes, En average ðrÞ is the initialized energy of the nodes. This ensures that the higher residual energy has the higher probability to be cluster head and it will improve the network lifetime. So, this paper uses the revision of the formula in the simulation, the revision is: Tk ðnÞ ¼

pk Ekn  1  pk ðr mod 1=pk Þ Ekn

residual ðrÞ average ðrÞ

ð4Þ

Where, Tk ðnÞ is the select threshold in ring k; pk is the percentage of cluster head in ring k; Ekn residual ðrÞ denotes the current residual energy of nodes in ring k; Ekn average ðrÞ is the initialized energy of the nodes in ring k.

4.2 The Choice of the Radius of Clustering In order to ensure that the clusters can cover the region of the ring Maximum and make a more balanced distribution of the cluster head on ring network model, this paper derives the proper radius of cluster as follows: The area of ring k is: Sk ¼ pðkhÞ2  ½pðk  1Þh2 ¼ pð2k  1Þh2

ð5Þ

And the area of a single cluster in ring k is: SchðkÞ ¼ ph2chðkÞ

ð6Þ

A Ring and Clustering Routing Scheme

197

In theory, all the clusters’ area in ring k is equal to the area of ring k, that is: Sk ¼ mk SchðkÞ

ð7Þ

So, from the equations of (5–7), we can obtain the radius of cluster in theory, which is: rffiffiffiffiffiffiffiffiffiffiffiffiffi 2k  1 h ð8Þ hchðkÞ ¼ mk However, it can not guarantee the cluster’s radius will distributed with the theory value in real environment. Hence, this paper revises the radius as follows: h0chðkÞ ¼ xhchðkÞ ðx [ 1Þ

ð9Þ

The value of the distance modified by coefficient x, which is usually a value of greater than 1. This paper uses it as 2 according to the path attenuation factor.

4.3 Cluster Formation Based on Data Field Physics problem is a physical quantity related to distribution in a certain regions of space called ‘‘field’’. Usually, the interaction effect among neighboring particles and the concept of the ‘‘field’’ method are introduced to a few domain spaces, which is used to describe the distribution law of various physical quantity or mathematical function [8]. Nodes will influence each other in wireless sensor networks. The candidate cluster head with much remaining energy and more nearly from the base station have a greater influence while other candidate cluster head is smaller. Therefore, we take the advantage of this feature for cluster formation in wireless sensor networks, consider the area of each ring as a potential field and use the theory of data field to make cluster. Potential field is a scalar field. The criterion of potential requires its function must be smooth, continuous, limited and monotonous. So, in order to better show the characteristics of clusters, this paper uses normal distribution function that has good mathematical properties to define the potential in field data [9]. Definition 1 There are n objects fx1 ; x2 ; . . .; xn g in space X, so when xi stands of the position of the candidate cluster head, the potential at node x generated by the candidate cluster head xi in the data field is as follows: jjxxi jj 2 Ui ðxÞ ¼ mi e o ð10Þ Where, jjx  xi jj is the distance between node and the candidate cluster head. This paper uses the value of RSSI between node and candidate cluster head node as the

198

W. Dai et al.

distance. mi is the quality of the candidate cluster head, here, we uses the ratio of the residual energy of candidate cluster head and the residual energy of all nodes in the ring. Influence factor o is used to control the degree of interaction between objects. The value of o will affect the intensity of cluster: the smaller the value the more compact cluster; the other hand, the sparse cluster [10]. Here the value is 5.

4.4 Inter-Cluster Communication Mechanism The communications among clusters has a great impact on the balancing of energy consumption after the establishment of the cluster. In LEACH, cluster heads communication with each other by single-hop, which causes the cluster head node consumes large amounts of energy by Long-distance transmission of data and dies early. Hence, the algorithm of Dijkastra is used in this paper to establish the minimum cost routing tree as the inter-cluster communication routing tree.

4.5 Algorithm Description Summing up the above analysis, the algorithm implementation steps are as follows: Step1. Network Initialization N nodes are deployed randomly in the monitoring area. The sink node broadcasts the information including the ring width h, the total number of nodes and the area radius R. Nodes in the network calculate its ring number according to the information sent from base station; Step2. Every node calculates the threshold Tk ðnÞ by formula (7) and generate a random number x (0\x\1). Then, the node determines whether they can be a cluster head by compare x with Tk ðnÞ; if the value of x is more than Tk ðnÞ; the node is to be candidate cluster head, on the other hand, to be non-cluster head; Step3. Candidate cluster head node sends out a message in its broadcast radius obtained by formula (10). The message includes its own ID and location information, if there are multiple candidates within the broadcast radius of the cluster head node, then select the candidate cluster head node who has large remaining energy as cluster head and the other must abandon the competition; Step4. The non-cluster head obtains its potential generated by each cluster head node with the formula (10) and selects the large potential as its cluster head node; Step5. Establish the minimum cost routing tree for communications by Dijkastra; Step6. If all the nodes are dead, the network is over; otherwise repeat the process 3–7.

A Ring and Clustering Routing Scheme Table 1 Experimental parameter values Parameters Value Monitoring area Base station coordinates Number of nodes Node initial energy Packet size Rings

(-240, -240) * (240, 240) (0, 0) 1000 2J 2000 bit 3

199

Parameters

Value

x h emp Eelec EDA efs

2 5 0:0013 pJ/bit/m4 50  109 J=bit 5 nJ/bit/signal 10 pJ/bit/m2

5 Simulation Analysis 5.1 Simulation Environment and Parameters For our experiment, we divide the monitoring area into three rings evenly. The ring width h = 80, Cluster head node broadcasts a radius of 2h: Other parameters are shown in Table 1.

5.2 Performance Analysis This paper compares RCBDF with the classic clustering routing algorithm LEACH from energy efficiency and network lifetime. 1. Network lifetime The network lifetime is defined by the number of nodes in each round of survival in this paper. We compare the RCBDF protocol with the original LEACH protocol after 100 tests, as is shown in the Fig. 2, for LEACH, the first node died at 540 rounds and all nodes died at 1400 rounds, but, in RCBDF, nodes start to die at 1080 rounds and extend the network lifetime to 2500 rounds, it prolongs the network lifetime greatly by the algorithm of RCBDF compared with LEACH. This explains that the single-hop communications in LEACH will consume large energy in the large area with high-density nodes. However, RCBDF divides the area into many small ring areas, and uses the heterogeneous cluster formation in each ring. This strategy makes the RCBDF algorithm effective extend the network lifetime much more than LEACH protocol. 2. Energy efficiency In Fig. 3, it shows that the energy efficiency by LEACH and RCBDF routing algorithm. We can clearly see that the energy consumption of RCBDF significantly lower than LEACH. Thus, RCBDF can balance the node energy consumption distribution more effectively, reduce the network’s energy

200

W. Dai et al.

Fig. 2 Network lifetime

Fig. 3 Energy consumption per round

consumption greatly, and extend the network life cycle in effect; it is suitable for Advanced Systems which is better in intrusion detection and firewall monitoring.

6 Conclusion In this paper, we focus on reducing energy consumption of wireless sensor network, so we propose the RCBDF. The algorithm uses the concept of data field for cluster formation on ring network. Cluster by data field will not only ensure the coverage of the monitoring area but also adjust the density of cluster according to actual need. In selection of cluster head, the remaining energy of nodes and the distance between the transmitter and receiver are considered. And, we build the

A Ring and Clustering Routing Scheme

201

multi-hop communication routing tree by Dijkastra at stable communication phase. So, this algorithm can effectively reduce the link cost and greatly balance the node energy consumption. The most important, the algorithm has good scalability to adapt to large-scale network.

References 1. Liu Y, Luo Z, Xu K, Chen L (2010) A reliable clustering algorithm based on LEACH protocolin wireless mobile sensor networks. IEEE Computer Society, Singapore, 10–12 Sept 2010 2. Wu X, Chen G, Das SK (2008) Avoiding energy holes in wireless sensor networks with nonuniform node distribution. J IEEE Trans Parallel Dist Syst 19(5):710–720 3. Li J, Mohapatra P (2005) An analytical model for the energy hole problem in many-to-one sensor networks. In: Proceedings of 62nd IEEE vehicular technology conference. IEEE Communications Society, NewYork, pp 2721–2725 4. Li J, Mohapatra P (2007) Analytical modeling and mitigation techniques for the energy hole problems in sensor networks. Perv Mobile Comput 3(3):233–254 5. QING Li, Qing-Xin ZHU, Ming-Wen WANG (2006) A distributed energy-efficient clustering algorithm for heterogeneous wireless sensor networks. J Softw 17(3):481–489 6. Heinzelman WR, Chandrakasan A, Balakrishnan H (2002) An application-specific protocol architecture for wireless microsensor networks.IEEE Trans Wirel Commun 1:660–670 7. Handy MJ, Haase M, Timmermann D (2002) Low energy adaptive clustering hierarchy with deterministic cluster-head selection. In : Proceedings of the Fourth IEEE conference on mobile and wireless communications networks, Stockholm, erschienen, S 368–372, ISBN 07803-7606-4. World Scientific, Singapore, Sept 2002 8. Xu K, Meng Y, Liu Y, Xiong N, Yang LT, Zheng M (2010) An effective method to improve the resistance to frangibility in scale-free networks. The 13th IEEE international conference on computational science and engineering (CSE-2010). IEEE Computer Society, Hongkong, China, 11–13 Dec 2010 9. Zhang K-B, Orgun MA, Zhang P, Zhang P (2006) Hypothesis oriented cluster analysis in data mining by visualization. ACM, Venezia, Italy, May 254–257 May 2006 10. Dai X, Gan W, Li D (2004) Study of image data mining based on data field. J Comput Eng Appl

A Case Study on Cooperative Real-time Quality Monitoring Service Utilizing Sensor Networks Hangbae Chang, Jonggu Kang, Yanghoon Kim, Jaehwan Lim and Youngsub Na

Abstract Introduced an ERP system for increasing their profitability and a point of production system, shop floor controller system etc. for managing sites, however, data has still been managed manually in production sites, and the difficulty lies in the real-time process control due to limitations of the introduced information system. Therefore, this paper studied examples of cooperative real-time quality monitoring services utilizing sensor networks that could manage the real-time production process by integrating various technologies of ubiquitous sensor networks together with RFIDs considering characteristics for each workplace as well as for each industry. Keywords RFID


Sensor
Network
ERP
Management

H. Chang (&)
J. Kang
Y. Kim
J. Lim
Y. Na Department of Business Administration, Daejin University, Hogukro 1007, Pocheon-Si, Gyeonggi-Do, Korea e-mail: [email protected] J. Kang e-mail: [email protected] Y. Kim e-mail: [email protected] J. Lim e-mail: [email protected] Y. Na e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_19, Ó Springer Science+Business Media B.V. 2012

203

204

H. Chang et al.

1 Introduction Manufacturing companies have an insufficient internal capacity following to adoption of a mass production strategy mainly with end products, lack experience of changing into advanced business based on knowledge such as core component technologies and knowhow etc., the production is moved to China and Southeast Asia of a low-wage system according to a level of labor-cost dependent manufacturing services, small and medium businesses have a vulnerable technological level, and businesses are mainly promoted by large companies. General manufacturing companies have introduced an enterprise resource planning (ERP) system to the production sites, and bar codes etc. are attached to use at a level of component packaging units by combining electronic devices with the existing installed manufacturing facilities. It is required studies on a method to manage production processes in real time by applying various ubiquitous sensor network technologies together with RFIDs considering characteristics for each workplace as well as industrial characteristics. Therefore, this paper would like to study on examples of a cooperative real-time quality service utilizing sensor networks by not restricting only to the existing static electronic tag (for example, RFID) [1] technologies and by applying the entire new IT technologies such as ubiquitous sensor networks etc. suitable to each site and situation.

2 Related Studies 2.1 Concepts of Sensor Networks Sensor networks are composed of smart sensor node hardware, sensor node platform software, network infrastructure, service server system for application services and so on. The smart sensor node hardware consists of general hardware sensors, which obtain (sense) surrounding environmental information, sink nodes, which are the link nodes for interworking with external networks, especially internet, and gateways [1]. This hardware has a characteristic that should be designed as a robust structure with low-power because it is difficult to maintain and repair and it has to be worked for from a few days to a few years once it is deployed [2]. It may become different significantly according to its application field and should have flexibility and modularity to use effectively even for any structure, consequently, it has a limit that could attach various sensors and easily replace by hardware fitted for the purpose [3]. The sensor platform software efficiently enables to operate sensor nodes and networks normally, and to implement sensor network based application services. It could be divided into a middleware for a sensor node itself (node middleware) and a middleware for providing and managing services of the sensor platform (network middleware.) The network infrastructure is composed of numerous sensors, ad-hoc and multi-hop communication between nodes [4]. It has a characteristic different from the existing wireless local area network (LAN) or cellular networks [5]. In addition, it is

A Case Study: Cooperative Real-time Quality Monitoring Service

205

Fig. 1 Advantages of sensor network

constructed with tens to hundreds of sensor nodes, and it has also a function to cooperate each other by installing numerous sensor nodes within a narrow area.

2.2 Features, Advantages and Disadvantages of Sensor Networks The major features are that each node has a simple structure of resources restricted to a low-power and low-price system operated with batteries that is not a network for a general purpose that is data-centric in the communication, which supports wireless communications between heterogeneous nodes and so on [6]. In addition, it has features that carries out intelligent functions by giving a computing power, extends the installation domain even to various areas where could not be installed, numerous sensor nodes could be processed in the concentrated area, and sensor nodes are cooperated each other to perform the function. In addition, phenomena in the real world could be observed more accurately and reliably by merging information collected from the individual node, the compact system could reduce costs compared to the conventional sensor systems, and it could collect data observed in detail from various perspectives because a single task is carried out by dividing it into several nodes (Fig. 1).

3 Cooperative Real-time Quality Monitoring Service Utilizing Sensor Networks 3.1 Demand Survey on Applications of Sensor Networks The purpose of this study is not to ask the intention of introducing high-level sensor networks for manufacturing companies having no IT capability, but to conduct an exploratory study on applicability of sensor network technologies in

206

H. Chang et al.

terms of resolving insufficient unmet needs on the current company’s information service for manufacturing companies having a reasonable level of IT capability, so that the number of companies for surveying is limited to 37. For service models to be designed, a sensor network service is first drawn based on the result considering all the 3 sectors such as the demand company’s intention of introducing new technologies such as sensor networks etc., possibility of introducing on the production process, possibility of cooperating with partners. As a result of aggregating scores for each item, a cooperative real-time quality monitoring service model utilizing sensor networks is first designed for automobile and process industry.

3.2 Cooperative Real-time Quality Monitoring Service in the Automobile and Process Industry Domestic manufacturing companies focus on the quality control guaranteeing a high quality for responding more effectively and strategically to the reduced demand (market) in the fierce competition environment following an economic crisis. Because quality competitiveness for finished products is determined by a quality control level of subcontractors for components as well as a quality control level of the production company for finished products, the production company for finished products has developed a horizontal quality control activity by conducting guidance on quality and evaluation regularly for improving quality of subcontractor’s components. Because there are thousands to tens of thousands of components due to the characteristics of automobile and process industry, the existing regular visit or passive management method could not but have various limits in terms of management levels, effects and costs etc.

3.3 Process Analysis on the Cooperative Real-time Quality Monitoring Service in the Automobile and Process Industry Examining features of the component’s quality control process for the cooperative real-time quality monitoring service in the automobile and process industry, each subcontractor in the supply chain has a feature carrying out a similar process to inspect quality (Fig. 2). First, when raw and subsidiary materials etc. are stocked, a warehousing inspection is carried out, an inspection (quality inspection for semifinished products) is conducted between processes during the production (processing, assembly) process, and then a product inspection is performed for the end products. In conducting the warehousing, inter-process and product inspections like this, a result of the quality inspection is mostly made manually to enter and store it, and then its own shipment result report is written to share the quality information when delivering the end products. Therefore, because it is difficult to

A Case Study: Cooperative Real-time Quality Monitoring Service

207

Fig. 2 Current process of car and device industry

expect the exact quality information and the cooperative infrastructure is also insufficient for remaining the quality competitiveness, there are many problems on effective guidance of improving the quality and traceability of the quality history. Like this, to resolve problems when monitoring a quality cooperatively in real time in the automobile and process industry, a service process should be required to design by applying sensor network technologies.

3.4 Process Design of the Future Service in the Automobile and Process Industry In order to promote a cooperative quality management policy effectively in the automobile and process industry, it is designed to construct environments for the preventive quality, and to obtain and analyze the quality information in real time. For improving a current limitation due to connecting the quality instrument with the measurement hub, which collects the measured data, via a wired network, it is designed by applying the Zigbee technology as Fig. 3 illustrates. It is designed to introduce the interworking technology between the quality instruments, which use the Zigbee technology, and the measurement hub, the low-level protocol converting technology for delivering the quality measurement information directly to the data server, the middleware platform technology for integrated management of heterogeneous data collecting devices and connections of applications, the Zigbee localization technology for managing the instruments and data’s history

208

H. Chang et al.

Fig. 3 Improves process design of can and device industry

information efficiently, and the real-time quality information monitoring and analyzing system at each point.

4 Verification of the Cooperative Real-time Quality Monitoring Service in the Automobile and Process Industry 4.1 Evaluation of the Attractiveness Because there are very few preceding studies that designed the business service evaluation model utilizing the sensor network technology, it is evaluated the model derived by using the ubiquitous service evaluation model. For the service model suggested in this paper, it is conducted the expert evaluation of 3 persons in the academy and 3 persons in the industry according to detailed evaluation items such as technical considerations, feasibility and acceptability etc. The evaluation’s result is as Table 1, and it is measured by 5-point measurement unit. The technical considerations for the designed service obtains relatively lower score than other evaluation domains, and such a result shows that the development level for core technologies is still insufficient for its commercialization. In addition, the feasibility for the designed service obtains higher scores except some services in general, and such a result is based on the expectation that the potential

A Case Study: Cooperative Real-time Quality Monitoring Service

209

Table 1 Result of the attractiveness evaluation on the service process of sensor networks Evaluation item Detailed item Evaluation result Technical considerations

Feasibility

Cost validity for core technology Development level of core technology Current dissemination level of core technology Reliability of core technology Level of service values Availability of core resources Concreteness of the target market Size of the target market Definiteness of principal investment agents Profitability of the service

Score

Total

3.4 4.1 3.8 3.3 3.9 4.1 3.7 3.9 3.7 3.9

14.6

23.2

market for sensor networks would continue to grow in the future. Finally, the acceptability for the designed service is inspected that there is no aversion about applying the service to the production sites actually, and such a result is considered because the sensor network technology is not the emerged form of new technologies but is developed for improving a limitation of the existing technology (RFID.) Like this, as a result of the attractiveness evaluation for the designed service, it could obtain the result more than ‘average’ in terms of technical considerations, feasibility and acceptability.

4.2 Expected Effects of the Cooperative Real-time Quality Monitoring Service in the Automobile and Process Industry By applying sensor networks to the cooperative real-time quality monitoring service in the automobile and process industry, it could be obtained the qualitative expected effects as follows. The huge amount of data could be collected and backed up for quality, and the process and quality could be visually managed in real time. In addition, it could build the prior quality prevention and quality guarantee system, and the horizontal quality management system could be built by sharing and utilizing the quality information. In addition, the lead time for improving the quality such as an action for unsuitable components could be reduced quantitatively, and the input time could be decreased by entering the quality information in real time. And it could have an effect that the quality control cost is reduced by sharing the real-time quality information.

210

H. Chang et al.

5 Conclusion Even though the manufacturing companies have introduced an ERP system in the production sites, and they use bar codes attached in the component packaging unit level by combining electronic devices with the existing installed production facilities, information is not flowed efficiently between the enterprise management system and the production system. Not carrying out procurement, distribution and sales by collecting information on the production process in real time brings about another inefficiency. Therefore, in order to collect information on the production process in real time, this paper designs the method to do the cooperative real-time quality monitoring by applying various ubiquitous sensor network technologies together with RFIDs considering characteristics for each workplace as well as industrial characteristics, and the evaluation is conducted for experts. The propose service model could provide an integrated quality management system to general companies manufacturing products by operating several workplaces (factories) as well as the industry constructing a supply chain like the heavy industry field such as automobile, shipbuilding and process industry etc. In addition, if this service model is applied also to the industries handling large semiproducts such as the shipbuilding and process industry etc. rather than compact components like automobile parts, it is expected that there would be a better effect.

References 1. Bhattacharyya D, Kim T, Pal S (2010) A comparative study of wireless sensor networks and their routing protocols. Sensors 10(12):10506–10523 (MDPI AG, Kandererstrasse 25, CH4057 Basel, Switzerland) 2. Lee J, Kapitanova K, Son SH (2010) The price of security in wireless sensor networks. Comput Netw 54:17:2967–2978 (Elsevier Science BV, Amsterdam) 3. Raymond D, Marchany R, Brownfield M, Midkiff S (2006) Effects of denial of sleep attacks on wireless sensor network MAC protocols. In: Information assurance workshop, 2006 IEEE, pp 297–304 4. Liu Y, Wang C, Qiao X, Zhang Y, Yu C (2009) An improved design of ZigBee wireless sensor network. In: International conference on computer science and information technology 2009 (ICCSIT 2009) 5. Lei Y, Zhang Y, Zhao Y (2009) The research of coverage problems in wireless sensor network: wireless networks and information systems. In: International conference on WNIS ‘09 6. Xia J, Law CL, Zhou Y, Koh KS (2010) 3–5 GHz UWB impulse radio transmitter and receiver MMIC optimized for long range precision wireless sensor networks. IEEE Trans Microwave Theo Tech 58(12):4040–4051

High Availability Archive for Time-Based Data Tobias Haag, Thomas Meier and Izudin Dzafic

Abstract An approach for archiving time-based data is presented. The basic idea is to split data intelligently into separate time-dependent areas having different logical and physical properties. It is especially useful and sustainable when dealing with just a few but very big data tables, when all-time availability becomes a principal issue or when it is necessary to have a fast restore in case of a disaster recovery—even if the database reaches sizes over one TB. Keywords Archive systems storage Backup and restore Area





Power systems
Huge database
Oracle
Data
Short Term Area
Mid Term Area
Long Term

1 Introduction Generally historical data collected in industry processes is becoming more and more important for direct operational work, optimization decision support systems [1], disturbance studies and even quantitative risk analysis [2]. Over the past years the number of measurements increased significantly and so did the total amount of data to be stored. T. Haag (&)
T. Meier
I. Dzafic Siemens AG, Humboldt str. 59, 90459 Nuremberg, Germany e-mail: [email protected] T. Meier e-mail: [email protected] I. Dzafic e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_20,  Springer Science+Business Media B.V. 2012

211

212

T. Haag et al.

Short Term Area Raid 1 / 10

Move data

Mid Term Area

Import: reload / restore Mid Term

Long Term Area

Raid 5 Export file

Export

Fig. 1 Logical and physical data storage

Archiving terabytes of time-dependent data leads to some technical problems. Usually in such systems a huge amount of dynamic data is stored with a high throughput up to several thousands of records each second. Terabytes of numerical values, status data and system messages—all of them addressed by timestamp— might be stored in just a few big data tables. Static data like configurations and master data however is stored in a lot of tables, but referring to its total data volume it is more or less negligible for the overall size of the database. This is also true for Power Systems, where the described solution is already in use. On the one hand recently archived data is of greatest interest. For example a Power System’s operator uses trending based on very recently archived values in order to compare data and to control the power network. If necessary, this data must be edited manually. Data availability is vital and very time critical. On the other hand data must also be kept online for up to a few years in order to support reporting and other significant investigations. Appropriate queries are usually less time critical as the queries of the operators, but must be available for monthly or yearly reports. However manual changes are quite unlikely for nonrecent data. Finally old data must—in any way—still be available for several years for legal reasons. 10 years and more are quite usual. It is commonly accepted that this old data can be exported out to files and stored somewhere, for example on tapes. If necessary, it must be reloaded to the archive system first. Another requirement is that the database must be available again in after short time in case of a failure—even if the database is restored through a backup. One possibility to get a recovery faster is to reduce the size of the backups. Since still all data must be backed up, the data which shall be available soon have to be backed up in a different way as older data. This means in case of Power Systems, the data of for example the last three months has to be available soon and the system can store new values into the database. The restore of older data may take more time, since it is not critical for operational use cases. Because the old data has to be exported to files for offline archiving, these files provide the possibility to be used as backup (Fig. 1). For the outlined solution the database is split into three parts (logical locations): 1. Short Term Area: contains the most recent data. 2. Mid Term Area: contains older data. It is still online, but data is considered to be read-only. The Mid Term Area should be stored on separate disks.

High Availability Archive for Time-Based Data STORAGE_UNIT

ARCHIVE_TABLE

PK

STORAGE_UNIT_ID

PK

FK1

TABLE_ID PARTITION_NAME

FK1

TIME_STAMP_FROM

TIME_STAMP_TO TRANSFER_STATUS

VERSION

213

TABLE_ID TABLE_GROUP_ID TABLENAME TABLESPACE LOCATION TIME_STAMP_FROM TIME_STAMP_TO PARTITION_RANGE

ARCHIVE_TABLE_GROUP PK

TABLE_GROUP_ID

TABLE_GROUP_NAME TRANSFER_RANGE_HOURS

EXPORT_DIRECTORY IMPORT_DIRECTORY

PARTITION_RANGE_UNIT

Fig. 2 Entity-relationship diagram: archive tables

3. Long Term Area (LTA): consists of data having been exported to external files. It has to be reloaded, if a user wants to access it. Because the files are already created when moving data from Short Term Area to read-only Mid Term Area, they incidentally also constitute a backup of the Mid Term Area. This split implies some severe logical consequences. As the data is no longer stored in one single relational table, the actual (and varying in time) location must be known for access. This task can be solved by an API, which defines an interface and returns the data from the correct tables. It can also be used to initiate a reload of Long Term data, in case the requested time range is stored completely or partly in the Long Term. Since the data of Mid Term Area was already exported to Long Term Area files, it can no longer be allowed to modify the data in the Mid Term Area—otherwise these changes would get lost in case of a recovery. Because the API is also used for writing/editing data, it must also take care of this important restriction. The Mid Term Area can be stored on a RAID 5 (‘‘RAID’’ = Redundant Array of Independent Disks) because there is mainly read access, except the movement of data from Short to Mid Term and from Mid to Long Term. The Short Term Area should be stored on a RAID 10 or RAID 1 for best read and write performance.

2 Algorithm of Data Transfer The described solution is implemented based on Oracle Enterprise Edition [3] with the Partitioning Option [4]. Although Oracle already provides all necessary basic functionality, the actual solution must still be implemented. Currently there is not any feature like ‘‘Short, Mid or Long Term solution’’ directly offered by Oracle. As the API and also the internal management of the data must know where the data is located, some bookkeeping tables are used storing the necessary information in relational tables (Fig. 2). The information is split into following parts: 1. Data archives can be spread over Short, Mid and Long Term. The table group joins these areas logically together. Therefore every data archive is assigned to one table group consisting of Short, Mid and Long Term parts.

214

T. Haag et al. STORAGE_UNIT

STORAGE_UNIT_IN_LTA_UNIT

PK

STORAGE_UNIT_ID

PK,FK2 PK,FK1

FK1

TABLE_ID PARTITION_NAME TIME_STAMP_FROM TIME_STAMP_TO TRANSFER_STATUS VERSION

STORAGE_UNIT_ID LTA_UNIT_ID

LTA_UNIT

LTA_FILE PK

LTA_FILE_ID

FK1

LTA_UNIT_ID FILE_NAME ARCHIVED RESTORE_STATUS

PK

LTA_UNIT_ID

FK1

TABLE_GROUP_ID TIME_STAMP_FROM TIME_STAMP_TO EXP_TIME_STAMP_FROM EXP_TIME_STAMP_TO USER_COMMENT STATUS

LTA_RELOAD_JOB PK

LTA_JOB_NO STATUS NAME TIME_STAMP_FROM TIME_STAMP_TO USER_ID EXPIRATION_DATE RESTORE_FLAG CANCEL_FLAG IMP_DIRECTORY

LTA_UNIT_IN_JOB PK,FK1 PK,FK2

LTA_JOB_NO LTA_UNIT_ID

Fig. 3 Entity-relationship diagram: long term archiving

2. The underlying data table. In case of the Long Term Area this can also be a ‘‘logical’’ table, which does not exist physically in the relational database. It is nothing but a bookkeeping entry. Thus a table group consists of the following tables stored on different physical locations: (a) (b) (c) (d) (e)

One physical Short Term Area table One physical Mid Term Area table One ‘‘logical’’ Long Term Area table (just a bookkeeping entry) 0 to n only temporarily existing intermediate transfer tables 0 to n real reload tables, if data was reloaded from LTA to the online system

3. Each table consists of 0 to n storage units, which reflects the Oracle partitions for the Short Term and Mid Term tables. The time range of the data contained by a particular storage unit gets recorded. This time range is left-justified (end time is excluded), because Oracle’s range partitioning is used and the partitions are created with the ‘‘less than’’ statement. The storage unit’s current status is changing during the move from Short Term to Long Term. This must be recorded very detailed in order to enable the process to resume in any case without manual actions. 4. The files of Long Term Area (LTA) are grouped in LTA units. One LTA unit consists of one or more files and one or more storage units. Usually one LTA unit consists of one file only, but it is possible to limit the size of the LTA files. Also the parallel export option from Oracle could be used, if a LTA unit consists of more than one file (Fig. 3). The whole mechanism is working in the background. All background jobs are implemented in PL/SQL and started by the Oracle Scheduler [5, 6]. The following steps have to be executed regularly:

High Availability Archive for Time-Based Data

215

1. The partitions for the Short Term tables must be created and the information must be stored in the storage units table. As in Power Systems new values get the current timestamp, it is not necessary to create too many partitions in advance. But nevertheless at least partitions for some days should be available. So there is time enough to react to potential technical problems—for example no space left on disk—without disturbing the running applications. 2. The partitions must be analyzed for better performance. The statistics on the tables/partitions are used by the Oracle optimizer to get the best read performance. Analyzed but empty partitions, which are filled up later, decrease the performance until the partitions are analyzed again. A separate background job is used for statistics gathering which guarantees that only filled partitions are analyzed, as it is known, when the data is stored. 3. The data must be moved from the Short Term Area to the Mid Term Area as soon as the Short Term retention period is reached. This is the core functionality and it must be done in the following steps: (a) Merge partitions to bigger partitions. This step is optional but it helps to reduce the number of partitions in Mid Term. It can be used to increase the retention period of the Mid Term because there is a restriction in Oracle concerning the maximum number of partitions in one table (about 32000 partitions). But the experience shows, that the access to partitioned tables slows down already, if the table has more than 10000 partitions. (b) Create a table with the exact physical layout as the partition. This table will be used later on for the export to files and also for the move to Mid Term. There has to exist exact one table for each partition, which has to be moved. For easier maintenance it gets the same name as the partition. (c) Exchange the data between the partition and the table. Using the command ‘‘ALTER TABLE … EXCHANGE PARTITION …’’ it is only an internal bookkeeping change from Oracle. The extents of the partition are assigned to the table and vice versa. This also generates nearly no entry in the redo logs and therefore also not in the archived redo logs. This helps to reduce the required space for archived redo logs during transfer from Short Term to Mid Term. (d) The Long Term archive files have to be created now. This is just an export of the transfer tables using the Oracle Data Pump export [7]. Data Pump has a PL/SQL interface [8], this makes it easy to use in the background job scheduler with the Oracle Scheduler. The export has to update the LTA unit information. Creating the LTA files has to be finished before moving the data to the Mid Term, otherwise there would be no backup of this data. (e) The next step is to move the transfer tables to the Mid Term. It is a physical move using the Oracle ‘‘ALTER TABLE … MOVE …’’ command. This statement also produces nearly no entries in the redo logs. It may decrease the fragmentation of the data segment because originally inserted data might not have been inserted concurring to the index.

216

T. Haag et al.

(f) When the tables are moved, the partitions must be created on the Mid Term table. (g) Afterwards the Oracle ‘‘EXCHANGE’’ command is used to exchange the data from table to the partition in Mid Term. If there are indexes on the tables, they have to be rebuilt then, otherwise they will be not usable. In case of index-organized tables the rebuild is not necessary. (h) Drop the empty transfer tables. (i) Analyze the transferred partitions in Mid Term, as the optimizer statistics get lost during transfer and would be also no longer valid as the move command results in an optimized storage of data by Oracle. 4. When the retention period in Mid Term is reached, the partitions have to be dropped. As the Long Term archive files are already created, just the bookkeeping must be updated, that the data is now in Long Term.

3 Backup and Restore There are also some additional commands necessary for backup and restore to make this approach working [9, 10]. When the tablespaces for the Mid Term are created, they have to be excluded from the backup. RMAN’s command ‘‘CONFIGURE EXCLUDE FOR TABLESPACE …‘‘excludes a tablespace for all backups made after execution of this command. When a restore is necessary, the Mid Term tablespaces must be dropped first, otherwise RMAN will try to restore them. This can be done by SQL command ‘‘ALTER DATABASE DATAFILE ‘…’ OFFLINE FOR DROP’’ before the restore command. The tablespaces must be also excluded at recovery: ‘‘RECOVER DATABASE SKIP TABLESPACE …’’. After having successfully restored the Short Term, the Mid Term tablespaces must be dropped and recreated. Finally the configuration to exclude the tablespace from backup must be executed again, because this configuration was lost with the drop of the tablespace. Some PL/SQL procedure recreates the Mid Term tables and updates the internal bookkeeping tables. Since logically for the API the overall time range covered by Mid Term must be contiguous, the restore has to be started with the newest data and has to be continued in direction from newest to oldest—an important restriction of Mid Term restoration. With this approach the normal move from Short Term to Mid Term can be done already, even if the Mid Term is not yet completely restored. If the restore takes long, it is possible, that some data has already reached the retention period and thus has no longer to be restored. Because range partitions are used and created with the ‘‘less than’’ statement, the partitions have to be split during restore of the Mid Term. This is supported by Oracle with the ‘‘ALTER TABLE … SPLIT PARTITION …’’ command. Because a split of partitions causes an internal renumbering of the partitions by Oracle, this approach will slow

High Availability Archive for Time-Based Data

217

down the more data is restored in Mid Term and the split command has to be executed. Therefore all necessary partitions are already created when recreating the Mid Term. In case the restore of Mid Term takes some time and the retention period of an empty partition is reached, the partition will be just dropped by the normal background process and the status of the storage unit will be updated.

4 Restore/Reload of Mid Term Area Reload and restore are mostly automated. The only exception is that the necessary files are not loaded automatically from a backup medium like tape—some user interaction is still needed. The necessary files will be listed by restore/reload. The load can be also automated, but as there are different solutions available for the archiving a generic method is not available. The reload of Long Term data and restore of Mid Term Area are nearly the same, there are just a few more steps necessary in case of a restore. The following steps are executed during restore/reload: 1. Start of the reload or restore. The time range and the type of data must be specified. In case of a reload this information is already passed by the API. Internally a reload/restore job will be created. The jobs are stored in tables, so it is possible to check the status in different sessions. A background job, which is implemented in a way, that it can be restarted in case of any issues, takes over the work. 2. First the time range of the reload/restore will be adapted to fit to the partition size because only partitions as a whole can be imported. 3. Now the files to import will be identified using the LTA bookkeeping tables. As long as the necessary files are on disk, the reload/restore imports them step by step. If there are one or more files missing, the background job lists them in a way, that the user can see them through the UI. As long as the reload/restore job is not cancelled it will retry to import the files after a configured time. 4. In case of a reload, the job is finished after the import. The background job inserts for each imported table (which was a partition before) an entry in ARCHIVE_TABLE, so the API can find the reloaded data. 5. In case of a restore, the reloaded table will be exchanged with the corresponding partition in Mid Term. The reloaded table will be dropped afterwards. The status of the storage unit will be updated, that the data is available in Mid Term. At each start of a reload job, the user can specify how long the reloaded data should be kept in the online system. It is possible to modify this date as long as the data is online. It may happen that the same storage unit is part of more than on reload job. In this case, the data will be only unloaded, if all reload jobs are expired. The unload itself is just a drop of the table and an update of the

218

T. Haag et al.

Table 1 Database sizes of two productively working reference systems Data inflow MB/min Online retention period in months Size in GB 8 3.5

Total

Short Term

Mid Term

Short Term

Mid Term

24 4

3 2

21 2

980 300

6,800 300

bookkeeping table. The expiration of reload jobs is checked by the same background job, which also executes the reload. Furthermore a separate tablespace is used in order to limit the maximum reloaded data. This tablespace is not auto-extendable, so the maximum used space for reload is already allocated at installation. If the required space was underestimated, the tablespace size can be still modified using Oracle commands.

5 Result The Short/Mid Term solution provides the flexibility to fulfill different customer needs by also assuring a stable overall workload and an acceptable performance— even if Short and Mid Term data accesses are mixed. The following table shows two exemplary configurations, which are already in productive use for more than 2 years now. The configurations differ due to different use cases. The first system needs data over two years for some reports, which is not required for the second system (Table 1).

6 Conclusion and Risks This paper presented an approach for reducing the restore time of a large database. The goal was to provide an algorithm which • makes it possible to restore the most recent data fast; • does not accept data loss and ensures high data consistency of Short and Mid Term Area (by moving complete partitions and providing a detailed bookkeeping and an intelligent API for data retrieval); • can be automated; • just uses Oracle features only. The current version of Siemens’ Historical Information System (HIS) contains all these features already implemented and proven in practice. A very exact bookkeeping is essential for backup, restore and data retrieval by API. Any failure or even any inaccuracy (e.g. concerning left- or right-justified time ranges) can

High Availability Archive for Time-Based Data

219

cause significant data loss—logically (data retrieval by API) or even physically (backup/restore). By accessing the intermediate transfer tables if necessary, the API is basically designed to provide data access even while the Short to Mid Term data transfer is in progress. However, it cannot be guaranteed that data can be read without any gap in all potential circumstances without the API having logically locking the transfer. But HIS avoids any lock by intention, because it must not be possible at all to block the important (background) data transfer by simply reading data. The coded algorithm is critical concerning transactional locking issues, potential error handling and the possibility to restart broken jobs again. Since Oracle’s SQL data definition language commands like ‘‘ALTER …’’ already terminate currently open transactions, on the one hand no regular rollback is possible anymore and on the other hand any intermediate changes in bookkeeping tables become persistent. In case of failure all changes must be undone explicitly. This is why any real implementation must be very carefully checked and intensively tested.

References 1. Niu CL, Yu XN, Li JQ, Sun W (2005) The application of operation optimization decision support systems based on data mining in power plant. In: Proceedings of 2005 international conference on machine learning and cybernetics 2. Rastogi UJ, Srinivas VK, Guptan RG (2010) Databases and their analysis for applications in quantitative risk analysis of NPP. In: 2nd international conference on reliability, safety and hazard (ICRESH), 2010 3. Oracle Database, http://www.oracle.com/technetwork/database/enterprise-edition/overview/ index.html 4. Oracle Partitioning, http://www.oracle.com/technetwork/database/options/partitioning/index. html 5. Oracle Scheduler Concepts, http://download.oracle.com/docs/cd/E11882_01/server.112/e17120/ schedover.htm#i1106753 6. Oracle Scheduler: Scheduling of Jobs and Administration, http://download.oracle.com/docs/ cd/E11882_01/server.112/e17120/scheduse.htm#i1033533. http://download.oracle.com/docs/ cd/E11882_01/server.112/e17120/schedadmin.htm#i1007589 7. Oracle Data Pump, http://download.oracle.com/docs/cd/E11882_01/server.112/e16536/part_ dp.htm#i436481 8. Oracle Data Pump PL/SQL Interface, http://download.oracle.com/docs/cd/E11882_01/appdev. 112/e16760/d_datpmp.htm#i1007277 9. Oracle Backup and Recovery User’s Guide, http://www.oracle.com/pls/db112/to_pdf? pathname=backup.112/e10642.pdf 10. Oracle Backup and Recovery Reference, http://www.oracle.com/pls/db112/to_pdf?pathname= backup.112/e10643.pdf

A Multivariate Spatial Outlier Detection Method Based on Semantic Similarity Fatemeh Azam and Ahmad Baraani-Dastjerdi

Abstract Spatial outlier detection is usually done in local neighbourhoods. We use an interpretable similarity measure to calculate the similarity between the regions of influence of spatial objects using spatial and non spatial attributes and relations both in the semantic description and in the similarity measure. A systematic method for similarity threshold selection is presented that can be used to categorize objects by their behavioral pattern in semantically similar neighborhoods. This paper’s main contribution is developing a multivariate two level outlier detection method. Real world data is used to evaluate our method. Keywords Spatial outlier detection

 Semantic similarity

1 Introduction A spatial outlier is a spatially referenced object which its spatial or non spatial attributes are significantly different from other spatial objects in its neighbourhood. In this paper we’re interested in finding patterns of behaviour in spatial data sets and outliers are data objects that do not follow any regular pattern. As it is expected that objects located nearby tend to show similar behaviour, therefore outlier detection is usually done in local neighbourhoods. An Objects neighbours are usually determined by spatial distances [1–4]. Since spatial objects are F. Azam (&)  A. Baraani-Dastjerdi Department of Computer Engineering, University of Isfahan, Isfahan, Iran e-mail: [email protected] A. Baraani-Dastjerdi e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_21, Ó Springer Science+Business Media B.V. 2012

221

222

F. Azam and A. Baraani-Dastjerdi

influenced by their surrounding, therefore for any consistent analysis, spatial and non spatial attributes and relationships as well as the object’s region of influence need to be taken into consideration. A spatial objects’ region of influence contains underlying spatial or non spatial attributes and relations which can affect the objects’ behavior [5]. This paper proposes a method in which effective parameters and attributes on an object’s behavior are extracted from different data sources and are used to model the objects’ behavior in its region of influence. An interpretable similarity measure and a systematic similarity threshold selection algorithm will be used to categorize objects by their behavioral pattern. Therefore outlier detection will be conducted both globally and locally in semantically similar neighborhoods. The paper is organized as follows. In Sect. 2 we will present some related work in the area of spatial outlier detection. In Sect. 3 this paper’s proposed method is presented. Section 4 gives experimental results on real world data taken from www.mapcruzin.com and evaluates the method. Section 5 concludes the paper.

2 Related Work Spatial outlier detection methods fall in two main categories, methods based on multi dimensional spaces and graph based methods. The first models spatial data objects as points in a multi dimensional space and uses concepts such as distance [1, 2], density [3] and wavelet transformation [6, 7] for outlier detection. In these methods neighborhoods are defined according to the Euclidean distance. In the graph based methods neighborhoods are defined by the existence of edges between spatial objects or the adjacency spatial relationship [4, 8]. Other methods form a network of grids on the whole space and use KNN to find the neighboring objects [9]. In most of the mentioned methods neighborhoods are defined according to distance, spatial relations or clustering methods. These methods do not consider the spatial objects’ region of influence which can be used to model semantic relations between spatial object attributes. However [5, 10] define a method to form micro neighborhoods around each spatial object and use the Jacard coefficient to calculate the number of matches between non spatial features of any two adjacent micro neighborhoods and combine them to form macro neighborhoods if the Jacard coefficient meets a certain threshold. This method uses exact matches to define similarity between data objects and does not take into account the underlying semantic relations or dependencies between attributes. Other methods suggest the use of anthologies to define semantic similarity and figure causal relations between data objects [11]. But the use of heterogeneous anthologies for spatial and non spatial data as well as different conceptual levels of comparison and similarity calculation makes these methods fairly complicated in design and implementation. In another way outlier detection can be seen as univariate or multivariate. Multivariate outlier detection is able to detect irregularity along multiple or single attributes of the data object. The most common of these methods is based on the

A Multivariate Spatial Outlier Detection Method

223

Mahalanobis distance introduced by P. C. Mahalanobis in 1936. The Mahalanobis distance is basically for continuous data, although methods have been presented which extend it to be used in categorical data as well [12]. Other methods in multivariate outlier detection implement a similarity function between data objects and use KNN method for outlier detection [13]. In semantic outlier detection we need an expressive distance function that not only calculates the distance between multivariate data objects, but is also able to express and analyze the difference/ similarity between attribute values in the form of associate rules. Semantic similarity requires considering spatial and non spatial features and relations both in the semantic description and similarity measure. The mentioned requirements of this literature make each of the pre-mentioned methods inappropriate in some senses.

3 Proposed Method With the use of a semantic similarity measure for spatial data we intend to find similarly behaving data objects in their region of influence, and change the concept of neighborhood from spatial distance to semantic neighborhoods, containing similarly behaving data objects. In fact each semantic neighborhood is comprised of smaller neighborhoods, called the region of influence of the data objects. Since the notion of similarity is strongly affected by the density and distribution of other objects relevant to a data object [14], we argue that similarity threshold selection should be performed per object, according to the density of similarity measurement values, and a novel systematic approach for threshold selection is presented. Semantic neighborhoods are created by combining similar regions of influence. Outlier detection will be done both locally in each semantic neighborhood and globally across the whole space using separate novel scoring functions.

3.1 Region of Influence Creation The spatial objects under study for outlierness will be represented by a point in space. In order to define the spatial boundary of the region of influence for a data object we will use Voronoi cells [15]. The Voronoi diagram makes it possible to determine the position of the nearest cell generator to a point of query, thus we will use it to define the spatial boundary of the region of influence for the data objects. Semantic relations and associate rules between the data objects are extracted from the information layers describing the objects and their region of influence; therefore in order to create reasonable rules it is important that the selected attributes are correlated and interdependent. In spatial data mining a common practice is to materialize spatial relationships into traditional input columns [16–19]. Semantic similarity calculation

224

F. Azam and A. Baraani-Dastjerdi

3.2 Semantic Similarity Calculation There are two main assumptions about semantic similarity, commonalitydifference and spatial distance. In this paper the similarity measure will be based on commonality-difference between attribute values. In the proposed method the similarity between each two values of an attribute is evaluated based on the semantic relations between other attributes under the condition that these values hold. Therefore similarity between each attribute value pair is first calculated based on the conditional probability distributions of other attributes under the condition that those values hold. Then similarity between two data objects is calculated as the sum of similarity between their attribute values. Some of the possible measures that can be used for measuring similarity between probability distributions are Kullback–Leibler [20], Lin [21] and Rached [22]. In this paper the Kullback–Leibler will be used. In order to apply this measure to the mentioned problem in this paper, we will use [23]’s notation. First the difference between each pair of possible attribute values vi ; v0i of attribute of attribute A are calculated based on the conditional probability distribution of other attributes. This measure is defined as follows:    0 ! X       P v jv P vj jv j i uA vi ; v0i ¼ vj P vj jvi lg  0  þ P vj jv0i lg  i  ð1Þ P vj jvi P vj jvi The second step of the similarity measurement process is to calculate the difference between two data objects X and Y by summing the difference between the values of corresponding attributes Ai , and is calculated as follows: X u ðX; Y Þ ¼ u ðx ; y Þ ð2Þ i¼1 Ai i i

3.3 Similarity Threshold Calculation After similarity measurement a threshold value that signifies the values measured as similar or non similar is needed. As the distance between an object and its neighbors decreases its density increases, and as a result the perception of similarity between objects decreases [14, 24, 25]. This notion implies that for dense closely distributed values of u the distance threshold should vary from loosely distributed values of u. In fact we argue that for each object its threshold value depends of the range of u ðX; Yi Þ values. The proposed method for threshold selection in this paper is based on significant changes in similarity measures u ðX; Yi Þ in a sorted list of similarity measures between data object X and other data objects Yi . Our threshold selection algorithm implements the SD test by considering a normal distribution on u ðX; Yi Þ. The test accepts values less than l þ cr as values with low dispersion and the first value that is more than l þ cr fails the test because it increases the dispersion. In order to prevent large or unreasonable values for the threshold, the average percent of objects possible of being similar to a reference data object should be calculated and

A Multivariate Spatial Outlier Detection Method

225

Fig. 1 Threshold selection algorithm

Fig. 2 Semantic neighborhood creation

applied as a higher filter on the algorithm (we call it sx ). The threshold selection algorithm is presented in Fig. 1.

3.4 Semantic Neighborhood Creation After threshold selection, similarity lists are built for each data object X, by comparing u ðX; Yi Þ for each data object Yi with the threshold value. The similarity lists can then be merged according to the transitive property to form partitioned semantic neighborhoods with similar properties. The algorithm for creating semantic neighborhoods is shown in Fig. 2. Outlier detection After grouping similarly behaving objects in semantic neighborhoods, outlier detection can be conducted in two levels called global and local. Global outlier score (GOS): Global outlier detection detects the ‘‘outlierness’’ of a data object based on its positioning relevant to its semantic neighborhood and the rest of the data objects. A global outlier score is calculated for each data object and we define it as the distance between a data object X and the object that caused its threshold value Ythrsh ; compared against the number of objects in X’s semantic neighborhood. GOS is mathematically defined as:

226

F. Azam and A. Baraani-Dastjerdi

u ðX; Ythrsh Þ size of ðsem neighborhoodkx Þ

ð3Þ

Where u ðX; Ythrsh Þ, is the distance between data object X and the object that caused the threshold value for X, and sem neighborhoodkx is X’s semantic neighborhood. The GOS compares the distance that partitions X from the rest of the dataset against the number of objects neighboring X in its semantic neighborhood. The higher u ðX; Ythrsh Þ and the lower the size of ðsem neighborhoodkx Þ, the more outlier score an object achieves. This is because we expect a global outlier X to be significantly different from the rest of the data set, while belonging to a small sized semantic neighborhood. Ythrsh for X is a the first significantly different value in the sorted list of u ðX; Yi Þ and conceptually it partitions X from the rest of the objects, therefore this distance qualifies as an appropriate distance that is able to compare how significantly different X is from the rest of the data objects. The size of X’s semantic neighborhood is also another parameter that can score its ‘‘global outlierness’’. Comparing these two parameters by means of a fraction enables us to combine relevant affective parameters on global outliers in a single scoring function. According to this function objects belonging to single member or very small sized groups that have a relevantly high u ðX; Ythrsh Þ are expected to be more ‘‘globaly outliered’’. Local outlier score (LOS): Local outlier detection detects the ‘‘outlierness’’ of a data object in its local set. For each object in a local set an outlier score in defined as the minimum distance between data object X and the rest of the objects in its semantic neighborhood compared against the maximum distance between other objects in X’s semantic neighborhood and is mathematically defined as follows:   MIN u X; semneighborhood kxX   Max u Xisemnegihborhood k ; Xjsemneighborhood k xX



ð4Þ

xX

Where MIN uðX; sem neighborhoodkxX Þ is X’s distance   to the rest of the semantic neighborhood, and the parameter Max u Xisem neighborhoodk ; xX

Xjsem

neighborhoodk xX

ÞÞ is the maximum distance between other data objects in the set,

or mathematically put, the diameter of sem neighborhoodkxX . The local outlier of a set is a data object that is different enough to be considered an outlier and similar enough to be a ‘‘local outlier’’. LOS indicates that a data object is ‘‘locally outliered’’ if it is far enough from the rest of the set, while the rest of the set is close enough to each other. A high value of MIN uðX; sem neighborhoodkxX Þ means that  a local data object is far enough from  the rest of the set and a low value of Max u Xisem

neighborhoodk xX

; Xjsem

neghborhoodk

means that the other data objects

xX

A Multivariate Spatial Outlier Detection Method

227

in the set are close enough to each other. The combination of these two parameters in one fraction enables us to score the local outlier by the parameters that are affective on it.

4 Experimental Results and Evaluation In this section we will investigate the results of applying the proposed method on real world data. SQL Server 2008 was used for managing the spatial data. Matlab’s ‘‘voronoin’’ function was used for creating Voronoi cells and vertices. A code was written for turning the resulting cells and vertices into shape files. In order to evaluate the proposed method, experiments were conducted on real world data taken from http://www.mapcruzin.com. The data sets dams00x020 (United States dams), and county20thcentury (demographic information by county in the USA) were used. In the experiments the objects of study for outlierness are ‘‘the dams of California’’ and the behavior being investigated is ‘‘the hazard of the dams malfunctioning’’. The documentation provided with the dams00x020 dataset describes the attribute ‘‘Hazard’’. Based on this information the potential hazard of a dam is influenced by the regions’ population or significant infrastructure and agricultural resources in the region nearby the dam. Therefore the information layer added was the county20thcentury dataset showing demographic information by county in the USA. Data regarding the state of California was taken from these datasets. Attribute selection was based on correlation calculation between the ‘‘Hazard’’ attribute and other attributes. Continues attributes were transformed to categorical data using Weka’s MDL module. The correlation test was conducted using Chi-square test for dependency with 95% significance level. The added spatial relationship to the context was the adjacency relationship between dams, which was added as a new attribute based on the locus of points between a defined adjacency length and adjacency width. Using the algorithms presented in the proposed method threshold selection and semantic neighborhood creation were preformed. The parameters used in the algorithms were: sx ¼ 2%; c ¼ 1; a ¼ 99%; e ¼ 0:5

4.1 Evaluation Method Comparison between our method and the KNN method is done for global and local outlier detection. Our method for global outlier detection is compared with the KNN method and Mahalanobis distance by comparing how early they are able to find actual outliers. In the KNN method the outlier score is the distance to the kth nearest neighbor. To evaluate the methods, we count the number of true outliers in the top p portion of the sorted scores, where p ¼ dn; and n is the number of actual outliers. Let p0 be the number of actual outliers in the top p predicted outliers.

228

F. Azam and A. Baraani-Dastjerdi

Fig. 3 Global outlier detection, comparing proposed method with Mahalanobis and KNN

Table 1 Global outlier detection accuracy compared with the KNN method Method Accuracy for d ¼ 1 Accuracy for d = 2 Proposed method K=3 K=6 K = 10 Mahalanobis

7/10 5/10 5/10 5/10 6/10

10/10 6/10 6/10 6/10 8/10

Fig. 4 LOS in one of the semantic neighborhoods of proposed method

0

The accuracy of the method is measured as pp . 10 actual outliers added to the dataset were used for evaluation. Figure 3 shows method accuracy for d ¼ 1 (last 10) and d ¼ 2 (last 20). As can be seen in Fig. 3 the five global outlier scores series tend to show a higher slope at the tails of the series which are supposed to be outliers. But in Table 1 it can be seen that the proposed method shows better accuracy in finding the actual outlier in the sorted outlier score list. Another interesting observation was that all actual outliers were placed in single value sets, which further confirms the validity of single sets being global outliers.The score we proposed for local outliers was used to score the local outliers. Since local outlier detection fairly depends on local set creation we present our local outlier detection results in one of the different semantic neighborhoods created based on our semantic neighborhood creation method in Fig. 4.

A Multivariate Spatial Outlier Detection Method

229

5 Conclusion The main contributions of our method are using semantic similarity for spatial outlier detection and defining two conceptual levels of outlier detection each with its own score functions. Comparing our results with KNN and Mahalanobis scores, showed that our method is semantically more reliable and interpretable.

References 1. Ramaswamy S, Rastogi R, Shim K (2000) Efficient algorithms for mining outliers from large data sets. In: Proceedings of the ACM SIGMOD 2. Bay SD, Schwabacher M (2003) Mining distance-based outliers in near linear time with randomization and a simple pruning. In: Proceedings of the ACM SIGKDD 3. Hu T, Sung Y (2003) Detecting pattern-based outliers. Science Direct- Elsevier, Amsterdam 4. Shekhar S, Lu C, Zhang P (2001) Detecting graph-based spatial outlier: algorithms and applications (a summary of results). Technical report Computer Science and Engineering Department, UMN, pp 01–014 5. Adam NR, Janeja VP, Atluri V (2004) Neighborhood based detection of anomalies in high dimensional spatio-temporal Sensor Datasets. ACM symposium on applied computing 6. Barua S, Alhajj R (2007) A parallel multi-scale region outlier mining algorithm for meteorological data. In: Proceedings of the ACM GIS 7. Lu CT, Kou Y, Zhao J, Chen L (2006) Detecting and tracking regional outliers in meteorological data. Science Direct, Elsevier, Amsterdam 8. Jin W, Jiang Y, Qian W, Tung AKH (2006) Mining outliers in spatial networks. In: Proceedings of international conference on database systems for advanced applications 9. Kou Y, Lu CT, Chen D (2006) Spatial weighted outlier detection. In: Proceedings of the SDM 10. Janeja VP, Atluri V, Adam NR (2004) Detecting anomalous geospatial trajectories through spatial characterization and spatio-semantic associations. ACM symposium on applied computing 11. Janeja VP (2007) Anomaly detection in heterogeneous datasets. Rutgers, The State University of New Jersey, Newark 12. McCane B, Alber M (2007) Distance functions for categorical and mixed variables. Patt Recog Lett 29:985–993 13. Chandola V, Boriah SH, Kumar V (2008) Understanding categorical similarity measures for outlier detection. CS technical report 08-008, Computer Science Department, University of Minnesota 14. Yoo C, MacInnis DJ (2004) Same or different? how distance and variation affect similarity judgments. Psychol Mark 21:209–277. doi:10.1002/mar.20002 15. Okabe A, Boots B, Sugihara K, Chiu S (2000) Spatial tessellations: concepts and applications of Voronoi diagrams. Wiley, West Sussex, pp 291–410 16. Quinlan J (1993) C4.5: programs for machine learning. Morgan Kaufmann, San Fransisco 17. Barnett V, Lewis T (1994) Outliers in statistical data, 3rd edn. Wiley, West Sussex 18. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. In: Proceedings of very large databases 19. Jain A, Dubes R (1988) Algorithms for clustering data. Prentice Hall, Upper Saddle River 20. Kullback S, Leibler R (1951) On information and sufficiency. Ann Math Stat 22:786 21. Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inf Theo 37(1):145–151

230

F. Azam and A. Baraani-Dastjerdi

22. Rached Z, Alajaji F, Campbell L (2001) Rényis divergence and entropy rates for finite alphabet markov sources. IEEE Trans Inf Theo 47(4):1553–1561 23. Le SQ, Ho TB (2005) An association-based dissimilarity measure for categorical data. Elsevier Patt Recog Lett 26:2549–2557 24. Krumhansl CL (1978) Concerning the applicability of geometric models to similarity data: the interrelationship between similarity and spatial density. Psychol Rev 85:445–63 25. Appleman IB, Mayzner P (1982) Application of geometric models to letter recognition: Distance and density. J Exp Psychol Gen 111:60–100

Cohesion Between Size of Commit and Type of Commit Maryam Marzban, Zahra Khoshmanesh and Ashkan Sami

Abstract Predicting what type of work was performed by developers in OSS has attracted several researchers. Previous studies have shown that fixing bugs are usually done by modifying very few files. In this study, we try to predict the trends and characteristics of how developers commit source code to version control systems (SVN, CVS, etc.). We want to investigate if number of files that are committed together is an indication of type of activity. In other words, is number of files in a commit related to the type of activity? Interestingly, we found that a strong correlation exits between type of activity and number of files involved in a commit. To make sure our study is not biased for a specific project, we have analyzed ten projects. Parsing messages in commits and processing the results, we found in coherent with previous studies bug fixing involves few files in a commit and commits that affect several files are usually related to adding functionality. Keywords Version control system characteristic Cohesion between size of commits and activity


Type of

activity in OSS




M. Marzban (&)
Z. Khoshmanesh
A. Sami CSE and IT Department, Shiraz University, Shiraz, Iran e-mail: [email protected] Z. Khoshmanesh e-mail: [email protected] A. Sami e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_22, Ó Springer Science+Business Media B.V. 2012

231

232

M. Marzban et al.

1 Introduction Over the years, open source software has become an important cornerstone of the software industry, and the version histories of these systems are typically recorded in version control system such as SVN or CVS. This history is a valuable source of data for understanding the evolution process [1]. In the paper presented here we aim to better understand committer’s activity. We want to investigate if there is any relation between committer’s activity and number of files that are committed together. We feel that a better understanding of this process will allow us to predict a type of activities for the future maintenance tasks. Commits recorded by version control systems keep track of changes to the software system. These commits happen any time a committer add, delete or change something in a source code. We use a tool named CVSAnalY to analyze and extract data such as, whom and when the event happens—what was the revision number and the comments attached to the change, from version control systems and import them to the MySQL database. Here, we focus on ten open source projects changes. We collect all important data, such as a number of files for each project’s commits and then categorize them into five intervals from extra-small to extra-large. One of the other important data is describing message. When a change is committed developers add a message describing the change. By analyzing these messages we can find most common vocabularies over a commits. We calculate the distribution of vocabularies over the commits and present the most often used vocabularies over each size category of commits. The paper is organized as follows. Section 2 describes a related work. The details of our data are given in Sect. 3. The results of the Categorization performed and a distribution of vocabulary are presented in Sects. 3 and 4 discusses some final conclusions.

2 Related Work Commit changes presented here is stored in a subversion repository, and this repository saves rich information about the evolution of a project. It is obvious that researchers have an interest to work on different point of change. Oliver Arafat and Dirk Riehle studied a commit size distribution of open source software. They review the total distribution and distinguish three categories of code contributions using a size-based heuristic: single focused commits, aggregate team contributions, and repository refactoring. They also suggest that distinguishing these commit categories by size will benefit future analyses [3]. Purushothaman and Perry discuss the impact of small changes on software projects [4]. They observe that during maintenance, nearly 10% of all changes

Cohesion Between Size of Commit and Type of Commit

233

made to the project under investigation were one-line changes. Hindle et al. [5] studied release artifacts that can be accessed from the software repositories. They try to find release patterns that happen around a release. They found that MySQL documentation and testing commits occur more before a release than after. While the build commits increased, source code changes decreased around release time.

3 Tools In this chapter we extract the data necessary for the study from the subversion control systems of ten open source software. For this data collection we use a tool named CVSAnalY. Then we categorize the data; and finally we describe a relation between these categories and commit activities.

3.1 CVSAnalY Tool Source code repositories store metadata such as user-IDs, timestamps, and commit comments in addition to the source code differences across versions. This metadata explains the why, who, and when a source code change. Modern source-control systems, such as Subversion, preserve the grouping of several changes in multiple files to a single change-set as performed by a committer. Version-number assignment and metadata are associated at the change-set level and recorded as a log entry. CVSAnalY is a tool created by Libre Software Engineering Group at University Rey Juan Carlos that extracts statistical information out of CVS (and recently Subversion) repository logs and transforms it in database SQL formats. It has a web interface—called CVSAnalYweb—where the results can be retrieved and analyzed in an easy way. We use this tool for analyzing our ten open source software, and extract commit information from subversion repository. Figure 1 show a sample of subversion information that is extracted by CVSAnalY. As we can see in Fig. 1, Source code repositories store metadata such as committer, timestamp and a text message describing the change entered by the developer. In addition to table we indicate in Fig. 1, we have some other tables such as ‘action’ table that maintains information about commit id and all files involved in a commit or paths involved in a change-set (Fig. 2). As shown in the figure, for example, the changes in the files ‘trunk/src/bug-buddy.c’ and ‘trunk/src/gdbbuddy.c’ are committed together and the commit 80 is assigned to the entire change-set.

234

M. Marzban et al.

Fig. 1 CVSAnalY analyze subversion repository and import result data into a SQL file

Fig. 2 Commits and related files generated by CVSAnalY

3.2 MonoConc Pro Tool This tool is a fast text searching program that is used in the analysis of English or other texts. It generates a concordances or listings of all the occurrences of any words in a given text with a frequency of each word (alphabetical order and frequency order). As shown in Fig. 3, a combination of all words, translation, update, bug, remove, fix, configure, add and etc., creates a given text. The frequency of

Cohesion Between Size of Commit and Type of Commit

235

Fig. 3 Count of words calculated for a given text

occurrences of each word is calculated and indicates in ‘Count’ column. For example, the frequency of ‘translation’ is 301 or 4.9% of total words.

4 Data Collection In this section, first we obtain data from ten open source projects over large number of commits performed during at least 5 years and at most 8 years and then we categorize data and extract commit messages vocabulary. Finally we indicate a correlation between each categories and vocabularies.

4.1 Commit Categories We determine a number of files involved in a commit as commit size or we can say the total number of files that are added, deleted, and/or modified in a commit indicate a commit size. Calculating a commit size is not expensive. As we mention in previous subscription (CVSAnalY tool), whole commits and related files exist in ‘action’ table. Therefore, we can calculate a number of files changed for each commit. Our approach would indicate that the commit with a larger number of files is larger. For our study we select 10 open source projects (the name of projects and their evolutions are shown in Table 1): bug-buddy epiphany gconf-editor gedit gnome-desktop gnome-terminal

graphical bug reporting utility web browser based on the Webkit rendering engine application for the GNOME computer desktop environment official text editor GNOME desktop environment terminal emulator for the GNOME desktop environment

236 Table 1 Ten open source projects and the years of their evolution history

M. Marzban et al. Name

Duration

Bug-buddy Epiphany Gconf-editor Gedit Gnome-desktop Gnome-terminal Metacity Nautilus-cd-burner Sound-juicer Yelp

1999–2008 2002–2008 2001–2008 1998–2008 1998–2008 2001–2008 2001–2008 2002–2008 2003–2008 1998–2008

Fig. 4 Number of commits distributed over number of files for all projects (32,043 commits)

metacity nautilus-cd-burner sound-juicer yelp

window manager package provides a method to write files to CD/DVD burner CD ripper help viewer

Here, we first calculate a number of files involved in all commits, and then use a distribution to classifying them into different categories. Committers group files in one commit wither it has been an added, modified or deleted. Our unit of change, the commit, has different number of files that are committed together. For our ten projects this range is from 1 to 323 files. Most of the times committer change two files and modifying more than 30 files is rare. Figure 4 shows the number of commits distributed over number of files. According to these numbers, the distribution of our commit size is limited to four regions. In the case of our data, the number of files per commit can each

Cohesion Between Size of Commit and Type of Commit Table 2 Top 20 frequent term over all projects

237

Term

Count

Data File Update Bug Remove Fix Doc Configure Add New Include Help Main Use Action Make Move Change Code Build

3,771 2,471 2,309 2,072 1,834 1,631 1,498 1,387 1,377 1,268 1,070 996 939 923 851 760 676 604 561 523

be distributed over these four regions. We classify the size of each commit based on these regions as small, medium, large, and exception. In exception region we have only two files in a most commits are in this region. As we can see in Fig. 4 the distribution skew to the right. Most of the commits are in the exception or small categories and the other regions are not considerable and do happen with a non-trivial frequency. The largest commits are often those that touch every file.

4.2 Commit Activity Commit size was described in a previous subscription, now we want to know what type of activity or changes usually happen for each category. For this purpose we extract the most frequent words used in commits’ messages over a ten projects. At first we consider the top most 30 frequent words, and then we find the top most frequent words-sets for each category of all projects. For each project, we collect all commits’ messages and save them in a text file. Then by using MonoConc tool we extract most frequent terms from a text file. At the end we combine all ten projects and take the top most 30 frequent terms. Below table shows the list of the top 20 words of ten projects Table 2. After building most frequent terms, we try to identify term combinations like ‘file-fix’, ‘bug-fix’ and etc. over all projects. Table 3 has a list of term combinations that are ordered by their frequency. As we can see in table, for small and

238

M. Marzban et al.

Table 3 Frequent terms and number of commits using terms (distribution) for all projects over each categories Exception Small Medium Large Term

Count

Term

Count

Term

Count

Term

Count

Fix-bug Make-file Fix-file Add-file Add-bug Use-bug Update file Move-bug Fix-data Remove-bug Make-bug New-bug Update-bug Fix-code New-file

708 261 234 216 178 176 142 128 126 120 118 108 106 104 98

Make-file Fix-bug Update-file Fix-file Move-file Add-bug Use-bug Move-bug Make-bug Change-file Remove-bug New-file Configure-file New-bug Move-code

541 454 292 276 275 223 215 214 191 176 169 155 146 140 128

Make-file Add-file Use-file New-file Move-file Remove-file Configure-file Add-bug Update-file New-bug Add-data Make-data Add-code Fix-bug Use-code

481 465 428 318 250 217 200 19O 186 183 159 141 129 121 117

Add-file Make-file New-file Move-file Use-file Remove-file Update-file Configure-file Add-data Make-data Add-code Change-file Fix-bug New-code New-data

191 181 154 133 125 110 106 100 93 66 60 59 57 56 56

exception categories most activities are belong to ‘fixing or updating files or bug’, but the type of activities of medium and large regions are ‘make, add, new or move files’. The list of frequent term combination indicates that small and large categories have different trend, in small categories most activities are related to bugs (fixing bugs or file) but in large commits adding new files or data is more common. It seems that types of changes correspond to size categories have a correlation with commit messages. However, there is at least some evidence of this phenomenon.

5 Conclusion This paper work on the version histories of ten open source projects to uncover a correlation between type and size of commit source code to subversion system (SVN, CVS). The goal is to characterize what a typical commits with respect to the number of files committed together. Actually commit messages are used along with the characteristic. The result data indicate that a large amount of commits has small or exception size (1–4 files are involved in a commit). Furthermore, we analyze vocabulary associated with the commit messages. The vocabulary data shows interesting sets of terms used across different categories of commits. The activity such as adding files or data is belong to large categories but small categories have fixing bug or file activities. It seems that commit data can provide an insight into the purposes of the changes.

Cohesion Between Size of Commit and Type of Commit

239

References 1. Kagdi H, Collard ML, Maletic JI (2007) A survey and taxonomy of approaches for mining software repositories in the context of software evolution. J Softw Maint Evolut Res Pr 19(2):77–131 (March/April) 2. Santiago D, Liliana T (2008) Database specification deliverable ID: D3.1, Version: 2.0, Oct 10 3. Arafat O, Riehle D (2009) The commit size distribution of open source software. HICSS. pp 1–8 4. Purushothaman R, Perry DE (2005) Toward understanding the rhetoric of small source code changes. IEEE Trans Software Eng 31(6):511–526 5. Hindle A, Godfrey MW, Holt RC (2007) Release pattern discovery via partitioning: methodology and case study. ACM Special Interest Group on Software Engineering, Washington, pp 19–27 6. Alali A, Kagdi H, Maletic J (2008) What’s a typical commit? A characterization of open source software repositories. ICPC. IEEE, Amsterdam, pp 182–191 7. Scacchi W (2007) Free/open source software development: recent research results and emerging opportunities. ESEC/SIGSOFT FSE (Companion), pp 459–468 8. Hindle A, Germán DM, Holt RC (2008) What do large commits tell us: a taxonomical study of large commits. MSR. ACM Press, New York, pp 99–108 9. Romain Robbes (2007) Mining a change-based software repository. MSR. p 15 10. Ohloh,Inc.Ohloh API. See http://www.ohloh.net/api

Software Defect Prediction Using Transitive Dependencies on Software Dependency Graph Javad Kamyabi, Farzaneh Maleki and Ashkan Sami

Abstract In software production process, quality assurance resources are limited by time and cost. In order to achieve high quality, managers need to detect the defect prone parts of code and allocate the resources to them. So far, researchers have used some methods such as complexity metrics, design metrics and network measures for software defect prediction. Although these methods are somewhat efficient, still there isn’t a global method for it. In this paper we have presented two new definitions which are dependency tree and circular dependency. Dependency tree covers all of the direct and indirect dependencies circular dependency evaluates all of the cyclic chains of dependency for each component. What has been differed our work from the previous related works is that we have presented that using transitive dependencies is efficient in defect prediction.













Keywords Defect Prediction Dependency graph Dependency tree Circular dependency

J. Kamyabi (&)
F. Maleki Department of E-Learning, Shiraz University, Shiraz, Iran e-mail: [email protected] F. Maleki e-mail: [email protected] A. Sami Department of Engineering, Shiraz University, Shiraz, Iran e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_23, Ó Springer Science+Business Media B.V. 2012

241

242

J. Kamyabi et al.

1 Introduction One of the most challenging presented issues in software quality assurance is software defect debug. According to the announcement of National Institute of Standards and Technology in 2002, software defects caused a 60 billion damage to USA economy [1]. Quality assurance is limited by time and cost; therefore managers try to allocate the resources to those parts of program code which are most likely to contain defects in comparison to the other parts. Many efforts have been done to help software project managers in order to test softwares and it has been always tried to determine the defect prone code parts by presenting various metrics of defect prediction. These metrics have been somewhat efficient. A group of these metrics determine software defects by measuring the code characteristics such as McCabe complexity metric [2–4]. These metrics are acceptable by many researchers and they can be easily extracted from the code. Another group of the metrics helps software teams in defect prediction by using the dependency between code components such as modules, classes and files. Zimmerman researched on network metrics in 2008 and showed metrics based on dependency have twice recall as much as the recalls of complexity metrics [5]. In this paper we have tried to present two hypothesizes for defect prediction based on the dependencies between the code components. The level of dependencies that we worked on is file level. The two metrics in this paper we have working on uses both of direct and transitive dependencies between the components for identifying those parts of the program which contain defects. The obtained metric on different versions of Eclipse and Apache Tomcat will be studied. The outline of the paper is Studying the previous literatures (part 2), Analyzing the main motivation of this paper (part 3), Data collecting (part 4), Analyzing the obtained result according to the metrics (part 5), Conclusion (part6).

2 Related Works The previous researches in defect prediction by data mining can be categorized in three groups: complexity metrics, historical data and dependencies. Binkley and Scotch presented coupling dependency metrics. Their researches indicated that these metrics are successful in improving software designing quality [6]. Alberg and Olson used complexity metric for predicting those modules that create run time failures. Their predictor models could identify 20% of code Paper Preparation modules that contained 47% of the total defects [7]. Nagappan discovered some metrics for predicting post released defects by researching on 5 Microsoft projects and presented a report about how post released defect can be created based on the history [8]. Bal presented that churned codes and the dependency to churned codes could cause post released defects [9].

Software Defect Prediction Using Transitive Dependencies

243

Hodpel could predict whether if a module contains defect or not by using the combination of complexity metric and historical data [10]. Ostrand studied on 2 huge software systems based on historical data up to 17 releases for predicting the files with the highest defect density. The defect predictor model identified 20% of the total files of the 2 software systems as the files that are containing defect and could identified 71 and 92% of the total defects [11]. Gravis demonstrated that metrics based on historical data perform better than complexity metrics by using a very huge system with long life time [12]. As it is obvious from the result of the researches based on the historical data, metrics that are based on historical data perform better than complexity metrics. But there are some problems in using the metrics based on historical data. The long term record of program defects may not exist. Henry and Kafura presented an issue as information follow in 1998, they presented a new metric named complexity metric using fan-in and fan-out and demonstrated that program designed by high rate of fan-in and fan-out have an inappropriate design [13]. Clarke and Pogdulsky in 1990 presented a formal model. The connection between 2 parts of the code can be discovered from the program source code based on this model [14]. Schroter worked on an issued named actual import dependencies in 2006. He also considered the kind of dependency in addition to counting fan-ins and fan-outs [15]. Microsoft team researches in 2007 demonstrated that using code churns and detecting the dependencies to churned codes can predict the post release defects successfully [16]. In 2008 Zimmermann and Nagappan had an experiment on defect detection using dependency, and they showed that the recall in their experiment is twice as much as the recall in Complexity metrics [5]. In 2009 Türhan demonstrated that defect prediction by dependency graph is much more successful than defect prediction by complexity metrics in big programs but in short program there is not much difference [17].

3 Motivation We can show the dependencies by construing the entire software as a low level graph. In this paper we will consider each fan-out as a dependency. According to the software engineering principals a high average of dependency for each component and also circular dependencies between pieces of code are known as weakness in software design. So in the simplest case we can use fan-out’s and directed circular dependencies to specify the defect proneness of pieces of code. You can see the simplest type of circular dependency in Fig. 1. As you see if Node A is depended to node B and node B is also depended to the node A, we have a simple circular dependency. So the first thing we are going to analyze is the relation between simple circular dependencies and defect proneness of nodes. As you see we may have two intermediate nodes that make a transitive circular dependency. And we can have even more intermediate nodes that make more

244

J. Kamyabi et al.

Fig. 1 Direct circular dependency

complex circular dependencies. The main questions we want to answer are whether we can use transitive dependencies and also transitive circular dependencies to predict defective nodes or not. Beneath you can see the hypothesizes we have worked on in this paper: H1 We can use simple direct dependencies in dependency graphs for software defect prediction and specify the quality of software H2 Transitive dependencies are related to defect proneness of software components like directed dependencies H3 We can increase the precision of our experiment by transitive dependencies due to the involvement of more nodes in a circular dependency‘s cycle. H4 We can use simple circular dependencies to focus on high probable defect prone nodes for limited debug time H5 We can use transitive circular dependencies to focus on high probable defect prone nodes for limited debug time H6 We can increase the precision of our experiment by transitive circular dependencies due to the involvement of more nodes in a circular dependency‘s cycle.

4 Data Collection In this experiment we have used two versions of the Eclipse project and one version of Apache Tomcat as our datasets. At the first step we have extracted the dependency graph of these programs by the Class Dependency Analyzer tool (CDA). The function of this tool is to analyze Java class files in order to learn more about the dependencies between those classes [18]. Then we have written a tool to parse the xml output of CDA (Class Dependency Analyser) to make the matrix of our dependency graphs. And finally we made a matrix of dependencies that presents the neighborhood of nodes in the dependency graph.

4.1 Dependency At first we will describe the dependency. A dependency is a fan-out from one component to another component. And in software dependency graph it‘s been presented as a directed tie to another node.

Software Defect Prediction Using Transitive Dependencies

245

Fig. 2 Dependency between two components

As you see in Fig. 2, there may be a multiplicity in dependencies. In this case we neglect it, and consider it as just a single dependency.

4.2 Dependency Tree Each node in software dependency graph, normally have some fan-outs. So if we start from any node, we have some dependencies to other nodes and those nodes also might have several dependencies to other different nodes so we can draw a tree from this chain of dependencies for each node. We call it, ‘‘Dependency Tree’’. A dependency tree shows all direct and indirect dependencies for each node so we can count the nodes of our dependency tree to evaluate the number of total dependencies of the root of the tree. And then we can evaluate the effect of total transitive dependencies on defect proneness of each node.

4.3 Circular Dependency If we have a circuit in the chain of dependencies, that starts from one node and ends to itself, we have a circular dependency. As you see in Fig. 3, there is a directed tie from node A to D and a directed tie from node D to A. so there is a simple circular dependency for both of nodes A and D. And there is a chain of dependencies from A to D, D to C and then C to A. so there is a circular dependency by the two intermediates D and C for node A. More complex circular dependencies can be occurred by more participant nodes in the cycle of dependencies. In this paper we have worked on circular dependencies with maximum 4 intermediate nodes.

4.4 Analyze We have used the number of transitive dependencies in the two defined concepts in this paper as a selection measure to select defective nodes. We can select different groups of nodes by increasing or decreasing the number of dependencies. The charts below show the behavior of selected nodes (True) and selected defective

246

J. Kamyabi et al.

Fig. 3 An example of circular dependencies between software components

Fig. 4 Behavior of T (Selected nodes) and TP (Selected defective nodes) in circular dependencies and dependency trees in eclipse 2.0

nodes (True Positive) toward each other. We have used ‘‘CD’’ instead of ‘‘Circular Dependency’’ and ‘‘DT’’ instead of ‘‘Dependency Tree’’ in the charts. For example ‘‘CDO1’’ means a circular dependency by 1 intermediate and ‘‘DTL2’’ means Dependency Tree by the depth of 2 (Fig. 4). As you see in all of the charts above, by going forward on the X axis, the number of selected nodes is dropping sharply. Thus, the density of defective nodes increases. So that, the density of defective nodes, is higher than 90% in some places. The obtained behavior in Eclipse 3.0 and Tomcat 6.0.32 is the same as the obtained behavior in the above charts. We have used the function Precision that measures the percentage of detected defective binaries to the total detected binaries. The fewer false positives (incorrectly predicted as defect-prone), the closer the precision to 100. [5]. At last we have used the Chi-square test to find out if our hypothesis separates defective and non-defective nodes significantly or not. We can show the capacity of detected defective components among all selected components based on a measure is called Precision. For example the precision 70% means that seventy percent of nodes you are selected from source code are

Software Defect Prediction Using Transitive Dependencies Table 1 Results of experiment

247

Source code

Analyze

Precision (%)

Chi-square

Eclipse 2

CDO1 CDO2 CDO3 CDO4 DTL1 DTL2 DTL3 CDO1 CDO2 CDO3 CDO4 DTL1 DTL2 DTL3 CDO1 CDO2 CDO3 CDO4 DTL1 DTL2 DTL3

48 70 84 77 52 36 57 49 70 94 90 49 52 57 46 50 60 80 66 71 62

Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant

Eclipse 3

Tomcat 6.0.32

defective. So in this paper we are trying to find a way to increase the Precision, and in the following, the best precision results obtained from this experiment are shown: According to the results of Chi-square test, all the measures based on the two main hypothesizes in this paper, have been separated defective and non-defective nodes significantly. As you see in Table 1 we could target a group of components by a high rate of defective nodes but still we do not have any idea about a specified number for circular dependencies to evaluate, to have the best result in addition in our experiment, by increasing the Precision, Recall is been decreased and increasing the Recall could be our next goal in next experiments. It’s a limited study. We will continue our experiment to higher depths of transitive dependencies in next studies. Also, we have worked on 3 open source projects that are written by java but we are confident that dependency has predictive power for other projects—we will repeat our experiments for more datasets.

5 Conclusion In this experiment 2 types of transitive dependencies have been used in defect prediction as 2 metrics for the first time and we worked on 3 java open source and showed that using transitive dependencies for defect prediction could be effective.

248

J. Kamyabi et al.

Our goal was to increase the chance of defect detection in limited time; we have focused on improving the Precision results in our paper. The results of this empirical study are coming below: Transitive dependencies are effective on quality of software design like direct dependencies. We can use transitive dependencies in dependency tree and/or other forms of transitive dependencies in software defect prediction. Transitive circular dependencies are effective in defect proneness of components, because we have more intermediates participating in circular dependency and issues like code churn or being defective, can increase the probability of defect proneness of participating components. By using transitive dependencies, we can really target a group of software components by high probability of defect proneness. High precision in software defect prediction, can be useful in limited time and cost for software debug process.

References 1. Tassey G (2002) The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology, Gaithersburg 2. Basili VR, Briand LC, Melo VL (1996) A validation of object orient design metrics as quality indicators. IEEE Transactions on Software Engineering 22:75–761 3. Nagappan N, Ball T, Zeller A (2006) Mining metrics to predict component failure. In: International conference on software engineering. China, pp 452–461 4. Subramanyam R, Krishnan MS (2003) Empirical analysis of CK metrics for object-oriented design complexity:implications for software defects. IEEE Trans Software Eng 29:29–310 5. Nagappan N, Zimmermann T (2008) Predicting defects using network analysis on dependency graph. In: The 30th international conference on software engineering (ICSE ‘08). Leipzig, Germany, pp 531–540 6. Binkley AB, Schach SR (1998) Validation of the coupling dependency metric as a predictor of failures and maintenance measures. In: International conference on software engineering. Kyoto, pp 452–455 7. Alberg H, Ohlsson N (1996) Predicting fault-prone software modules in telephone switches. IEEE Trans Software Eng 22:886–894 8. Ball T, Nagappan N, Zeller A (2006) Mining metrics to predict component failures. In: International conference on software engineering. China, pp 452–461 9. Ball T, Nagappan N (2005) Use of relative code churn measures to predict system defect density. In: International conference on software engineering. St. Louis, MO, pp 284–292 10. Allen EB, Aud SJ, Hudepohl JP, Khoshgoftaar TM, Mayrand J (1996) Emerald: software metrics and models on the desktop. IEEE Softw 13:56–60 11. Bell RM, Ostrand T, Weyuker E (2005) Predicting the location and number of faults in large software systems. IEEE Trans Software Eng 31:340–355 12. Graves TL, Karr AF, Marron JS, Siy H (2000) Predicting fault incidence using software change history. IEEE Trans Software Eng, pp 653–661 13. Kafura DG, Henry SM (1998) Software structure metrics based on information flow. IEEE Trans Software Eng 7:510–518 14. Clarke LA, Podgurski A (1990) A formal model of program dependences and its implications for software testing, debugging, and maintenance. IEEE Trans Software Eng 16:965–979

Software Defect Prediction Using Transitive Dependencies

249

15. Schröter A, Zeller A, Zimmermann T (2006( Predicting component failures at design time. In: International symposium on empirical software engineering. Rio de Janeiro, Brazil 16. Ball T, Nagappan N (2007) Using software dependencies and churn metrics to predict field failures: An empirical case study. In: International symposium on empirical engineering and measurement. Madrid, Spain, pp 364–373 17. Bener A, Tosun A, Turhan B (2009) Validation of network measures as indicators of defective modules in software systems. In: 5th International conference on predictor models in software engineering, Vancouver, Canada 18. Duchrow M (2010) May 16 programmer’s friend, class dependency analyzer (CDA) V1.13.1. Retrieved July 11, 2011, from http://www.dependency-analyzer.org/

Test Case Generation for Classes in Objects-Oriented Programming Using Grammatical Evolution Jirawat Chaiareerat, Peraphon Sophatsathit and Chidchanok Lursinsap

Abstract This paper proposes a dynamic test case generation approach for Object-Oriented Programming classes, using evolutionary search to find test cases that would satisfy a branch coverage criteria. Grammatical Evolution (GE) is used to search for a solution in accordance to user-specified grammar, thus making the algorithm more flexible than the traditional genetic programming. Rather than generating test cases directly, source code for an Intermediate Test Script (ITS) is generated from the grammar. It is then evaluated and translated into source code by ITS interpreter. Such a provision makes it easy to produce test cases that have object and literal reference, whereby improve the performance of GE. We’ve tested the proposed method with several java classes from open source projects and yielded high code coverage results.





Keywords Test case generation Code coverage Object-oriented programming Grammatical evolution Intermediate test script





J. Chaiareerat (&)  P. Sophatsathit  C. Lursinsap Department of Mathematics and Computer Science, Faculty of Science, Advanced Virtual and Intelligent Computing (AVIC) Center, Chulalongkorn University, Bangkok, 10330 Thailand e-mail: [email protected] P. Sophatsathit e-mail: [email protected] C. Lursinsap e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_24, Ó Springer Science+Business Media B.V. 2012

251

252

J. Chaiareerat et al.

1 Introduction Test case generation is inherently a vital part of software testing. Because testing all possible test data is infeasible, test case generation therefore uses different criteria to select and generate only a subset of test data that guarantees a high quality of the test set. These criteria are called code coverage. Previous works in test case generation can be classified into two groups, namely, static and dynamic approaches. The static test case generation relies on static analysis of the program source code. They usually apply the same concept as procedural test case generation by using symbolic execution and constrained solving. On the other hand, the dynamic approach models the test case generation as a search problem to find an optimal solution, which gives the highest code coverage. This approach executes the test object iteratively by tuning the test case until a satisfied result is reached. Various optimization techniques are used in dynamic test case generation including Genetic Algorithm [1–3], Simulated Annealing [2, 3], Memetic Algorithm [2, 3] and Strongly-Typed Genetic Programming [4, 5]. This paper proposes a dynamic approach test case generation for an object using GE to find test cases that would satisfy branch coverage criteria. Each test case is translated into Intermediate Test Script (ITS) format based on user-specified grammar. The idea is to find an executable program or program fragment that will yield a good fitness value for the given objective function.

2 Test Case Generation Framework The Test Case Generation Framework consists of 4 modules: Code Instrumentation, Grammar Generator, Search, and Evaluation Module. Referring to Fig. 1, Java source code is instrumented by Code Instrumentation module. This process will insert certain code to the original source code which will be used for collecting branch coverage information during the evaluation of each test case. The instrumented source code is then compiled into a Java class file by java compiler. This compiled Java class will later be fed as an input to the Evaluation Module. The Grammar Generator Module, as stated by its name, will automatically generate grammar for test cases. At this step, users can review the generated grammar and modify it according to their specific grammar. The Search Module then derives these test cases through the defined grammar based on genetic algorithm. This process transforms the test cases into ITS which later are evaluated by the Evaluation Module. The Evaluation Module processes the compiled Java class, together with Intermediate Test Script using its interpreter, to produce branch coverage information. The resulting branch coverage information is then fed back to the Search Module for selecting the best test case in each generation.

Test Case Generation for Classes

Grammar Generator Module

Java Sourcecode

Grammar

Code Instrument Module

Result Testcase

Search Module

Instrumented Sourcecode

Intermediate Test Script

Java Compiler JavaClass file

Fig. 1 Test case generation framework

253

Coverage Information Evaluation Module ITS Interpreter

3 Code Instrumentation Code Instrumentation is the process of inserting certain code to the original source code, wherever there’s a condition check and method declaration. The objective of code insertion is to let the newly inserted code collect information on path and coverage each time they are executed. Apart from collecting the mentioned information, they are also used for calculating branch distance. Branch distance information is used to evaluate how much each branch is closed to being taken. We used Java Parser for parsing and inserting new code to the original java source code automatically. In a situation that two test cases have the same degree of coverage, branch distance is used to determine which one is a better and suitable for potential test case. The value of branch distance represents how much a branch is close to being taken. In this paper, branch distance is calculated, using the formula shown in Table 1. The lower branch distance means the branch is closer to be covered. From Table 1, dist(x) is the branch distance of condition x and k is the smallest possible value of the branch distance (we used 0.0001 in this paper). The branch distance in the above table is then normalized by the following equation distðAÞnormalize ¼ 1  ð1 þ 0:1Þ0distðAÞ :

ð1Þ

254

J. Chaiareerat et al.

Table 1 Branch distance calculation Operation

Branch distance

dist(A == B) Dist(A! = B) Dist(A [= B) Dist(A [ B) Dist(A \= B) Dist(A \ B) Dist(A and B) Dist(A or B)

Abs(A - B) k B-A B-A?k A-B A-B?k Max[dist(A), dist(B)] min[dist(A), dist(B)]

4 Intermediate Test Script and Grammar In this proposed method, each test case is represented in ITS Format. Grammar for the ITS of a class under test is automatically generated from the source code by the Grammar Generator. The proposed framework generates test cases in Intermediate Test Script (ITS) format instead of a java source code. In ITS, each parameter of the method call can be referred to the previous created object or literal value. The advantage of using the same parameter helps improve the search process. Moreover, ITS also reduces the time for compiling the source code, since it can be run directly by ITS interpreter. Grammar of ITS for the class under test is generated from a java source code. Tester can modified the grammar by inserting some heuristics, which can help reduce the search space, making it easier for the Search Module to discover the optimum solution. Using grammar to represent test cases have an advantage in flexibility since various types of parameter such as array, string and object reference can be created by grammar. In this paper, we used grammar in BackusNaur Form (BNF) as an input of GE.

5 Search and Evaluation Grammatical Evolution performs the process of searching for a test case that has the best code coverage for the class under test. In this paper, we used GEVA [6] for our experiment. Traditionally, Genetic Algorithm (GA) is used to find the optimal solutions to a search problem. Genetic algorithm is classified as global search heuristics. Genetic Programming (GP) is for finding computer programs that satisfy user-defined criteria. Grammatical Evolution (GE) [7, 8], on the other hand, is an evolutionary computation technique that can be considered as a grammar-based GP. A program is represented by an ordered-list of integer. GE uses genotype-phenotype mapping process to map an ordered-list of integer to a computer program. With the help of

Test Case Generation for Classes

255

Table 2 Experimenta result (GE) Class name Total Achievable branches branches

Branch coverage (mean)

Branch coverage (%)

Stack StringTokenizer Vector LinkedList BinTree BinomialHeap BrentSolver SecantSolver Complex

10.00 39.00 123.00 122.00 37.00 74.67 28.00 18.86 51.80

100 97.50 96.09 93.84 100.00 85.82 96.55 99.26 95.92

10 40 128 130 37 87 29 19 54

10 39 123 122 37 77 28 19 52

grammar, GE also provides a very flexible way to control an algorithm. The user can define a grammar that biases to produce very specific form of program, or can incorporate domain knowledge of the problem into the underlying grammar. GE, GA, and GP use crossover and mutation operations to modify each individual and reproduce new populations. In this paper, a standard single point crossover; and nodal mutation [9] is used since it gives the best result among the three in our experiment. Fitness function will be calculated based on coverage information collected during the execution of each test case in the population. The following equation is used in the calculation: fitnessðtÞ ¼ ðb  covÞ þ 1  ð1:1Þ0bd

ð2Þ

where b is the total number of branches, cov is the total branch coverage of test case t, and bd is the total branch distance.

6 Experimental Results We used selected java class for our experiment including J2SDK version 1.4.2_12, Java Path Finder [10] version 1.3r1258 and Apache Commons Math version 1.1. The experimental results are shown in Table 2 which compares the proposed algorithm with Acuri [3] and Wappler [5] in Tables 3 and 4 respectively. In the proposed algorithm, GE is executed for 200 generations with the number of population equals to 50. The probability of crossover and mutation of GE is configured to 0.9 and 0.1, respectively. Average performance execution of the algorithm is based on 50 runs. Branch Coverage also includes the number of method calls and try/catch statements. The branch coverage percentage is the ratio of mean branch coverage over total branch. The total branches were not stated in Acuri [3], while the achievable branches and branch coverage were not stated in Wappler [5].

256

J. Chaiareerat et al.

Table 3 Experimenta result (Acuri’s memetic method [3]) Class name Total Achievable Branch coverage branches branches (mean)

Branch coverage (%)

Stack StringTokenizer Vector LinkedList BinTree BinomialHeap BrentSolver SecantSolver Complex

– – – – – – – – –

– – – – – – – – –

10 – 100 84 37 79 – – –

10.00 – 100.00 84.00 37.00 77.66 – – –

Table 4 Experimenta result (Wappler’s EvoUnit [5]) Class name Total Achievable Branch coverage branches branches (mean)

Branch coverage (%)

Stack StringTokenizer Vector LinkedList BinTree BinomialHeap BrentSolver SecantSolver Complex

100 93.70 – 98.30 – – 96.30 100.00 90.20

8 29 – 68 – – 27 17 51

– – – – – – – – –

– – – – – – – – –

The result shows that our algorithm is capable of generating almost all types of test cases by virtue of grammar which was evident by the high coverage outcome.

7 Conclusion We propose a grammar-based test case generation for a class in Object-Oriented by using Grammatical Evolution and Intermediate Test Script. This method also supports various features of Object-Oriented Programming through the grammar. The results confirm that the proposed method can generate high coverage test cases. Moreover, being grammar-based, this technique has great flexibility and advantages in that it is possible to incorporate heuristics and characteristic of the class under test into the grammar to reduce the search space of the problem.

Test Case Generation for Classes

257

References 1. Tonella P (2004) Evolutionary testing of classes. In: Proceedings of the 2004 ACM SIGSOFT international symposium on software testing and analysis (ISSTA’04), ACM, New York, pp 119–128 2. Acuri A, Yao X (2007) A memetic algorithm for test data generation of object-oriented software. In: IEEE congress on evolutionary computation (CEC 2007), pp 2048–2055, IEEE 3. Acuri A, Yao X (2008) Search based software testing of object-oriented containers: information sciences, vol 178, issue 15, pp 3075–3095, Elsevier Science, New York 4. Wappler S, Wegener J (2006) Evolutionary unit testing of object-oriented software using strongly-typed genetic programming. In: Proceedings of the 2006 conference on genetic and evolutionary computation (GECCO’06), ACM, New York, pp 1925–1932 5. Wappler S (2008) Automatic generation of object-oriented unit tests using genetic programming. PhD thesis, Technical University of Berlin 6. O’Neill M, Hemberg E, Gilligan C, Bartley E, McDermott J, Brabazon A (2008) GEVA: grammatical evolution in java. SIGEVOlution 3(2):17–22 ACM New York 7. O’Neill M, Ryan C (2001) Grammatical evolution. IEEE Trans Evol Comput 5(4):349–358 8. O’Neill M, Ryan C, Keijzer M, Cattolico M (2003) Crossover in grammatical evolution. Genetic programming and evolvable machines, vol 4, issue 1, pp 67–03, Academic Publishers, Kluwer 9. Byrne J, O’Neill M, McDermott J, Brabazon A (2009) Structural and nodal mutation in grammatical evolution. In: Proceedings of the 11th annual conference on genetic and evolutionary computation (GECCO), ACM New York, pp 1881–1882 10. Visser W, Pasareanu CS, Khurshid S (2004) Test input generation with java pathFinder. In: Proceedings of 2004 ACM SIGSOFT international symposium on software testing and analysis (ISSTA’04), ACM New York, pp 97–107 11. Buy U, Orso A, Pezze M (2000) Automated testing of classes. In: Proceedings of 2000 ACM SIGSOFT international symposium on software testing and analysis (ISSTA 2000), ACM New York, pp 39–48 12. Sen K, Agha G (2006) CUTE and jCUTE: concolic unit testing and explicit path modelchecking tools. In: 18th international conference on computer aided verification (CAV’06), LNCS vol 4144, pp 419–423, Springer, Berlin 13. Sen K, Marinov D, Agha G, (2005) CUTE: a concolic unit testing engine for C. In: Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, ACM New York, pp 263–272 14. Xie T, Marinov D, Shulte W, Notkin D (2005) Symstra: a framework for generating objectoriented unit tests using symbolic execution. In: Proceedings of the 11th international conference on tools and algorithms for the construction and analysis of systems (TACAS 05), LNCS vol 3440, pp 365–381, Springer, Berlin

An Efficient Access Control Scheme for Online Gaming Server Hye-Young Kim

Abstract Massively multiplayer online games (MMOG) are projected to one of the rapidly growing entertainment services that will gain large popularity in near future. When there is a request from the client, we almost apply dynamic method using Accept( ) of looping method; thus, there could be process of connecting synchronously lot of client in most of Online gaming server engine. However, this kind of method causes online gaming server which need to support and process the clients, longer loading and bottlenecking. Therefore we propose the object polling scheme to minimize the memory fragmentation and the load of the initialization to the client using an AcceptEx( ) and static allocation method for an efficient gaming server of the Online in this paper. We design and implement the gaming server applying to our proposed scheme. Also, we show efficiency of our proposed scheme by performance analysis in this paper. Keywords Gaming server


Object polling
Session

1 Introduction Online game is game of form that many clients connect to server through communication network and play their games in real time [1] Therefore, it must be able to access many players and should be able to guarantee the speed,

This work was supported by 2011 Hongik University Research Fund. H.-Y. Kim (&) School of Games, Hongik University, Seoul, South Korea e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_25, Ó Springer Science+Business Media B.V. 2012

259

260

H.-Y. Kim

scalability, reliability and security to play the game smoothly in online gaming [2]. When there is a request from the client, there could be process of connecting synchronously lot of client in most of online gaming server engine. However, this kind of method causes online gaming server which need to support and process the clients, longer loading and bottlenecking [3, 4]. Therefore we propose the object polling scheme to minimize the memory fragmentation and the load of the initialization to the client using an AcceptEx ( ) and static allocation method for an efficient gaming server of the Online in this paper. We design and implement the gaming server applying to our proposed scheme. Also, we show efficiency of our proposed scheme by performance analysis in this paper. The rest of paper is organized as follow. We explain in detail the background and the proposed scheme for an efficient online gaming server in next chapter. In Sect. 3, we describe our proposed architecture and provided the detailed functions and procedures. We show an effectiveness of proposed scheme through the performance analysis of our scheme and previous online gaming server in Sect. 4. Finally, we conclude the paper in Sect. 5 by presenting conclusion and future research.

2 Proposed Scheme 2.1 Proposed Background The most of online gaming servers necessary the sockets which take charge of communication with clients and they are necessary the buffers for copy of user data when they send/receive the data on asynchronous input/output. However, If the gaming server creates or release the socket and buffer whenever the access request and exit request from the clients access, they could not ignore the load which for a create the socket and an allocate the heap memory. Because there are lots of creation and delete of socket for connection and exit with client object that connect or disconnect to gaming server are happened frequently, they cause of memory fragmentation by the several dynamic memory allocation. It cause of lots of load and bottleneck, and it is important reason of obstruction of performance of online gaming server [5]. Therefore, we use and apply statically allocation scheme to reuse next time assigning socket or memory beforehand statically as need that is expected to online gaming server. So, our proposed scheme integrates resources of memory and so on that session and client who communicates with each client need and use object polling method to design to one object. Reusing socket and buffer same time connection of large quantity and addition delete frequent login server’s load as curtailment efficiency of online gaming server performance raise can.

An Efficient Access Control Scheme for Online Gaming Server

261

2.2 Object Polling Method The online gaming server reuse socket using AcceptEx ( ) TransmitFile ( ) function that are WinSock extension functions for object polling in Windows Operating System. An object polling method that apply socket reusability function divides and manages by map to use when do unicast, multicast, broadcast according to necessity in server being activated after become list and connection for unused list when wait for an opportunity before is activated after connected making beforehand like thread polling that use in IOCP (Input Output Control Protocol). We design the Session Mange class to manage session in data structure. Gaming server inserts and connects listen socket to IOCP list after create session dynamically as quantity which is necessary. After access request by asynchronous version of Accept ( ), gaming server is notified completion, and do asynchronous access request through AcceptEx ( ) that achieve work about connection. At successful connection completion through socket that is linked to IOCP Input Output Control Protocol) in worker thread, gaming server receives address of the client through GetAcceptExSockaddrs ( ) and add connected session to using map after search in unused list of the Session Manager and deletes in unused list. Also, in case of termination is confirmed through worker thread, delete resource about the client because request socket reusability using TransmitFile ( ) so that may can reuse socket that is used to AcceptEx ( ) and search in using map of Session Manager, and function call- AcceptEx ( )—after insert again to unused list and access request asynchronous again. We diagrammatize our proposed scheme in Fig. 1. Object polling methods which manage the object statically to apply in our proposed scheme minimize the happened load of the gaming server whenever clients connect the session.

3 Implementation of Gaming Server Applying Our Proposed Scheme 3.1 Implementation Environment We implement gaming server in Windows XP Operating System using DirectX 9.0c SDK 2008 Octobers applying object polling method for efficient online gaming server in this paper.

3.2 Gaming Server Architecture Session class process asynchronous access request and request through function call of AcceptEx ( ) using Client socket and Listen socket that exist to session

262

H.-Y. Kim

Fig. 1 Proposed scheme

using function call of PrepareAccept ( ). Our proposed scheme use AcceptOn (OVERLAPPED* Ov) to receive address of the client through function call of GetAcceptExSockaddrs ( ) after function call of AcceptEx ( ). Also, CloseSocket ( ) make reuse the socket using function call of the ttransmitFile ( ) after use function call of the shutdown ( ) to terminate and reuse the transmission of the session class. In our paper, we suggest that Session Manager class insert a number of object that have been arranged into a OffSession (List) through dynamic creation. And we implement CreateNewSession ( ) to function call, PrepareAccept ( ), to process asynchronous connection request. Also, we designed ActiveSession (Session* p_Session) to add to OnMap (Map) after we delete Session class that receive by transfer factor in Session class in OffSession (List) 1. Create session dynamically by function call of CreateNewSession( ) 2. Asynchronous access request by function call of AcceptEx( ) through function call of PrePareAccept( ) in created session.

An Efficient Access Control Scheme for Online Gaming Server

3. 4. 5. 7. 8. 9. 10. 13.

14. 15.

263

Inset session which have been created in Session Manager into InActiveList. Attach ListenSocket to IOCP Port. Create Thread, call to Worker Thread, and stand by function call of GQCS( ) Receive session address through GetAcceptExSockaddrs( ) when happen to accept event. Attach IOCP post to returned socket by function call of AcceptEx( ). Delete session in InActiveList which connected by function call of AcceptEx( ), and insert ActiceMap. Process Recv./Send. Session that when detected GQCS ( ) of Worker Thread END Event, and terminate send-receive work through function call of shutdown ( ) and make reactivated through function call TransmitFile( ). Delete session in ActiveMap of Session Manager, insert again in InActiveList. again asynchronous input/output request to Terminated Session through function call of PrePareAccept ( ).

Reuse the object through repeat 1 to 15 For using CreateIOCompletionPort ( ) in Init ( ) in IOCP Handler class, create IOCP port, and used CreateIoCompletionPort ( ) for connection of socket that is passed on IOCP port by socket value in IocpListenSockJoin (SOCKET Sock). Also, after connection completion, implementedd IocpJoin (Session* p_Session) method using CreateIoCompletionPort ( ) serving to receive connected Session by transfer parameter and connect on IOCP port. Acceptor class process function call of Listen ( ) and Bind ( ) by port number that is passed in ListenAndBind (UINT n_port) method and manage Listen socket [6]. We explained by simple pseudo code as follows about important part of online gaming server engine that apply and implement object polling which our proposed scheme. Listen socket connect on IOCP port CreateIoCompletionPort ((HANDLE)m_ListenSock,m_HandleIocp,(ULONG_PTR)0,0) Add to unused list after object create and asynchronous access request function call of AcceptEx( ) for ( int I = 0; I \ number of object polling; i ++) { Session* p_Session = new Session( ) Session- [ AcceptEx( ) //asynchronous access request by function of AcceptEx( ) InActiveList.push_back(p_Session) //add session in unused list }

264

H.-Y. Kim

Fig. 2 Running screen shot of gaming server

When Worker Thread that detect IOCP completion queue makes sure access, delete the session in unused list and add to using map a) Delete session on unused list for (std::list \ Session* [ ::iterator iter = nActiveList.begin( ); iter ! = InActiveList.end( );) { if(*iter == p_Session) { ActiveMap.insert(socket value, Session pointer); Iter = InActiveList.erase(iter); } else iter ++; } b) Add to using map std::map \ UINT,Session* [ :: iterator iter = OnMap.find (p_Session- [ GetSocketValue( )); if(iter ! = OnMap.end ( )) { InActiveList.push_back (p_Session); iter = ActiveMap.erase (iter); } When terminate the access request of session in Worker Thread that detect IOCP completion queue, the gaming server engine delete the session in using map resource of client. And it adds again to unused list after socket reusability request

An Efficient Access Control Scheme for Online Gaming Server

265

Fig. 3 Number of successful access session

Fig. 4 Performance analysis

using function of TransmitFile ( ). Also, again function call AcceptEx( ). The function call of Shutdown ( ) that explain above is as following: shutdown (m_SessionSock, SD_BOTH);

4 Performance Analysis To analysis the performance of online gaming server engine which is applied our proposed scheme in Section Chap. 2 and 3, online gaming server engine were compared with existing game engine about performance speed as increasing rate of the number of session. We use Microsoft Window XP Professional Version 2002 and Service Pack 2 for operating system in our implementation. Also our computer consist of Intel (R) Core (TM) 2 Duo CPU T9300 @ 2.50GHz3.0 GB RAM. Applied methods for the test are easy method of movement, conflict of box and client that is chatting possible. As socket event is detected by EventSelect( ) function on client-side is programmed DirectX9.0C, the packet is received and analyzed. The float value of the coordinates x, y, z, the float value of angle, ViewDis, ID and char type are corresponding with each other. Figure 2 shows the running programs of gaming server.

266

H.-Y. Kim

In the Accept method, the simultaneously accessible maximum numbers of session are 5 allowed the Listen function. So a lot of thread is required to increase the maximum number [7, 8]. However in case of function call AcceptEx( ) of IOCP, the maximum number is the number of pre-generated object with a small amount of thread. In order to perform efficiently large and simultaneous requests, 256 sockets generated in advance put on the unused list temporarily and then if connection requests are allowed, the sockets are inserted additional on using list from unused list. Figure 3 is shown the difference between method of dynamic memory allocation and static memory allocation as amount of sending and receiving packet and the difference between the Accept-looping method using an synchronous connection request of client and AcceptEx( ) method using an asynchronous access request. As the existing gaming server dynamically allocates socket and buffer whenever client access requests, the load of the server to create the socket and the load to allocate heap memory are weighted. The more sessions are increased, the width of the load increasing were significantly. But the load of gaming server with our proposed method is less increased, because socket and memory in Client connection as initial stage of gaming server is statically allocated. The result of mentioned analysis is illustrated in Fig. 4.

5 Conclusion In this paper, we proposed the asynchronous access request method and object polling method for efficient online gaming server. So we could minimize the memory fragmentation of online gaming server engine and the load that is occurred to initialize of session connection of the client object. Also we designed a game engine applied by proposed scheme and implement it. We could prove the efficiency of our proposed scheme through the performance analysis. However we cannot estimate the threshold which is optimized to the static allocation of client, we applied to the 256 sockets and 4 k byte memory to static allocation. Therefore, in the future research, we will design analysis model for the accurate prediction about the session connected the online gaming server and study more efficiency and reliable engine for online gaming server by offering the threshold value to the static allocation of client side. Acknowledgments This work was supported by 2011 Hongik University Research Fund.

References 1. Kim SW (2007) Window network programing, Hanbit Media, Seoul 2. Lee S-W, Kim H-Y (2008) An efficient message management scheme in game engine. J Korea Game Society 8(2)

An Efficient Access Control Scheme for Online Gaming Server

267

3. Duck M, Read R. Data communication and computer network. Prentice Hall, Englewood Cliffs 4. Kurose JF, Ross KW. Computer networking. Addison Welsey, Reading 5. Han D-H (2008) Online game server programming benchmarking. Information Publishing Group, pp 340–377 6. Han D-H (2007) Online game server programming. Information Publishing Group, pp 214–577 7. Nam-Hyoung, Moon H-J, Lee S-H (2007) Design and implementation based on load uniformity between game server. J Korea Inform Commun Soc Vol. 32(3) 8. Chen J, Knutsson B, Wu B, Delap M, Amza C (2007) Locality aware dynamic load management from massively mutiplayer games, PPoPP

Performance Evaluation of a New Scheduling Algorithm in Data Grid Environment N. Mansouri, Gh. Dastghaibyfard and A. Horri

Abstract Data Grid is a geographically distributed environment that deals with large-scale data-intensive problems. The main problems in data grid are job scheduling and data management. Generally, job scheduling in Grid has been studied from the perspective of computational Grid. In Data Grid, effective scheduling policy should consider both computational and data storage resources. In this paper a new job scheduling algorithm, called Combine Scheduling Strategy (CSS) is proposed that considers number of jobs waiting in the queue, location of required data and the computing capacity of sites. Scheduling cannot be effective unless to combine it with replication. Therefore, we have discussed various strategies in scheduling and replica optimization. Simulation results demonstrate that CSS gives better performance compared to the other algorithms. Keywords Data grids

 Data replication  Job scheduling  Simulation

1 Introduction A large number of scientific applications such as high energy physics, climate transition, and earth observation generate huge amounts of data per year. Today, management and efficient use of large distributed resources are important issues in N. Mansouri (&)  Gh. Dastghaibyfard  A. Horri Department of Computer Science and Engineering, College of Engineering, Shiraz University, Shiraz, Iran e-mail: [email protected] Gh. Dastghaibyfard e-mail: [email protected] A. Horri e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_26,  Springer Science+Business Media B.V. 2012

269

270

N. Mansouri et al.

scientific research and commercial application. Grid is a solution for this problem. Grid can be divided as two parts, Computational Grid and Data Grid. Computational Grids are used for computationally intensive applications that require small amounts of data. But, Data Grids deals with the applications that require studying and analyzing massive data sets. The size of data that are requested in Data Grid is from terabytes to petabytes. Effective scheduling of jobs is necessary in such a system to use available resources such as computational, storage and network efficiently. If a job is assigned to a site where all the required data are available, then it would have no data transmission delay or reduced turnaround time. Of course, a scheduler should also consider parameters such CPU workload, features of computational capability, network load, resource utilization and response time. Data replication is another key optimization technique to manage large data by storing data in a wisely manner. The replication strategies decide when to perform replication, which files to replicate and where to store replicas. Storing replica close to the users or grid computation nodes improves response time, fault tolerance and decreases bandwidth consumption. Therefore, appropriate replica management framework is critical in complex systems such as Data Grids that access large amounts of data. Some researchers have evaluated the impact of data replication on the job scheduling performance in the Data Grid [1–3]. In this paper a novel job scheduling algorithm that considers the number of jobs waiting in queue, the location of required data and the computing capacity of sites is proposed. It is called Combine Scheduling Strategy (CSS). The aim of CSS is to reduce the job execution time by reducing job data access time. It is simulated in OptorSim and compared with various combinations of scheduling and replication strategies. The test results show that CSS successfully reduces the mean job execution time. The rest of this paper is as follows: Sect. 2 describes the related works. In Sect. 3, scheduling problem is explained. CSS algorithm is proposed in Sect. 4. Simulation results based on the OptorSim are presented in Sect. 5. Final section contains the main conclusions of this work and future works.

2 Related Work Zhang et al. [4] have investigated the impact of parallel download on job scheduling performance in Data Grid. In their proposed method, data file is divided into fragments and placed on various replica servers. The replica fragments are transferred concurrently from several replica servers when a replica is requested. They have evaluated the performance of the parallel downloading system with the traditional non-parallel downloading systems using different grid scheduling strategies. The results of simulation indicated that it can improve the network bandwidth consumption and speed up the data access. When network bandwidth is relatively low and computing power is relatively high, parallel downloading has been effective.

Performance Evaluation of a New Scheduling Algorithm

271

Mohamed et al. [5] proposed the Close-to-Files (CF) scheduling algorithm that schedules a job on sites with enough processing capacity close to a site where data are present. CF uses an exhaustive algorithm to check all combinations of computing sites and data sites to select a site with the minimum computation and transmission delay. CF has better performance in comparison with Worst-Fit (WF) job placement strategy, which allocates jobs on the sites with the largest number of idle processors. In [1] the authors proposed a framework that supports job scheduling and data replication in Data Grid. Several replication strategies and their performance are evaluated with some scheduling heuristics. The data replication algorithms increase the scheduling performance from the perspective of job computing time. In particular, the combination of shortest turnaround time scheduling heuristic (STT) and centralized dynamic replication indicates prominent performance. In [6] the authors developed the Hierarchical Cluster Scheduling algorithm (HCS) and the Hierarchical Replication Strategy (HRS) to enhance the data access efficiencies in a grid. HCS considers hierarchical scheduling and uses cluster information to decrease search time for a proper computing node. HRS replication algorithm uses the concept of ‘‘network locality’’ as Bandwidth Hierarchy based Replication (BHR) strategy [7]. HRS has two advantages in comparison with BHR: First, BHR checks all sites to find the best replica, while in HRS required replica within the local cluster has high priority. Second, BHR uses popularity of replicas at cluster level but HRS uses at site level. HCS scheduling along with HRS replica strategy improve data access time and the amount of inter-cluster communications in comparison to others scheduling algorithms and replication strategies. Foster and Ranganathan [8, 9], have proposed six distinct replica strategies for a multi-tier data: No Replica, Best Client, Cascading Replication, Plain Caching, Caching plus Cascading Replica and Fast Spread. They also introduced three types of localities, namely: Temporal locality: The files accessed recently are much possible to be requested again shortly. Geographical locality: The files accessed recently by a client are probably to be requested by adjacent clients, too. Spatial locality: The related files to recently accessed file are likely to be requested in the near future. They evaluated these strategies with different data patterns: access pattern with no locality, data access with a small degree of temporal locality and finally data access with a small degree of temporal and geographical locality. The results of simulations indicate that different access pattern needs different replica strategies. Cascading and Fast Spread performed the best in the simulations. They have presented in another work [10] the problem of scheduling job and data movement operations in a distributed ‘‘Data Grid’’ environment to identify both general principles and specific strategy that can be used to improve system utilization and/or response times. They have also proposed framework with four different job scheduling algorithms, as follows: (1) JobRandom: select a site randomly, (2) JobLeastLoaded: select a site where has the least number of jobs waiting to run, (3) JobDataPresent: select a site where has requested data, and (4) JobLocally: run jobs locally. These job scheduling

272

N. Mansouri et al.

strategies are combined with three various replication algorithms: (1) DataDoNothing: there is no replication and data may be fetched from a remote site for a particular job, (2) DataRandom: when popularity of the file exceeds a threshold, a replica is created at a random site, (3) DataLeastLoad: when the threshold for a file exceeds, a replica is placed at the least loaded site. They can enhance performance by scheduling jobs where data is located and using a replication policy that periodically creates new replicas of popular datasets at each site. The results also show that while it is important to consider the impact of replication on the scheduling strategy, it is not always necessary to couple data movement and computation scheduling. In [11] the problem of co-scheduling job dispatching and data replication in large distributed systems in an integrated manner is presented. They used a massively-parallel computation model that contains a collection of heterogeneous independent jobs with no inter job communication. The proposed model has three variables within a job scheduling system: the job order in the scheduler queue, the assignment of jobs to the nodes, and the assignment of data replicas to data stores. Finding the optimal tuple requires an exhaustive search and it is costly because the solution space is very big. The results show that deploying a genetic search algorithm has the potential to achieve better performance than traditional allocation methods. In [12] the authors proposed a two-phase dynamic replication algorithm coupled with two-stage job scheduling to provide an integrated environment for efficient access to data and job scheduling. In the first decides what to replica and the second phase, selects best network link from local node to the nodes containing other replicas of same file. The jobs are scheduled to sites where have maximum availability of required data files and replica creation is based on the access load. The simulation results show that two-level job scheduling and two-phase dynamic replication strategy have better performance in terms of both job execution time and number of replicas created.

3 The Scheduling Problem Data Grid environment is comprised of storage resources (SE’s) which store the data and compute resources (CE’s) that execute the jobs. It is possible that the each resource have both storage and computation capabilities. The step-by-step procedure for scheduling jobs is shown in Fig. 1. The Resource Broker collects information about the available computational resources and the datasets through a resource information service and the Data Catalog respectively. Then, it uses a scheduling policy to calculate the cost of running a job on a group of candidate sites i.e. the scheduler determines where to dispatch a job based on the cost of the computational resource and data transfer time. The applications or jobs typically need large datasets for their execution; so the main factor in execution time is the total access cost of the required data. The

Performance Evaluation of a New Scheduling Algorithm

273

Fig. 1 The scheduling procedure in Data Grid

job is dispatched to the site with the minimum estimated cost by the scheduler. Job processing results are transferred to the Resource Broker or another storage resource. This process is repeated until all the jobs are scheduled.

4 The Proposed Scheduling Algorithm The proposed Combined Scheduling Strategy (CSS), first checks if the requested files are available in the local storage, then job will be executed otherwise; files should be transferred and replicated. CSS considers the number of jobs waiting in the queue, size of requested data files and size of computing elements (CE) of the site. Let Ji ¼ fF1 ; F2 ; . . .; Fm g be the m required files for job i. Let FSIZEi,j be size of requested data files that are not available in site j for Job i. X FSIZEi; j ¼ size of all unavailable files Fk for Ji ð1Þ Loadj ¼ Qj = Sj

ð2Þ

274

N. Mansouri et al.

where Qj and Sj are the number of jobs waiting in queue and size of CE’s of site j, respectively. These factors have an important role in job scheduling. So sum of them along with proper weights (w1, w2) for each factor yields the combined cost of executing Job i in site j as follows: CombinedCosti; j ¼ w1  FSIZEi; j þ w2  Loadj

ð3Þ

This function can be tailored, because it is defined as a weighted combination of two former metrics. Next combined cost for each Job is computed and the job will be assigned to the site with minimum combined cost i.e. CombinedCostmin ¼ MINnj¼1 CombinedCosti; j ; where n is the number of sites:

5 Experiments In this section first simulation tool and configuration are explained and then simulation results are given.

5.1 Simulation Tool CSS is implemented and evaluated in OptorSim, a data grid simulator developed by the EU Data Grid project, as shown in Fig. 2 [13, 14]. The OptorSim has several parts: • Computing Element (CE) represents computational resource in Data grid. • Storage Element (SE) represents data resource in Data grid. • Resource Broker (RB) gets the user’s job and allocates each job to proper site based on the selected scheduling policy. • Replica Manager (RM) at each site controls data transferring and provides a mechanism for accessing the Replica Catalog. • Replica Optimizer (RO) within RM implements the replication algorithm shown in Fig. 2b. When a file is requested by a job, RO would locate the best replica in terms of the file’s Logical File Name (LFN) and determine whether it should be replication, or create a temporary cache of the file locally.

5.2 Configuration For the experiments, we used the same testbed configuration in our previous work [15]. The Grid topology has 3 regions and on average three LAN’s in every region (Fig. 3). Table 1 specifies the simulation parameters used in our study. CERN

Performance Evaluation of a New Scheduling Algorithm

275

Fig. 2 a OptorSim architecture. b An expanded illustration of grid site

(where the data are originally produced) has a high capacity of 300 GB, and the storage capacity of all other sites is 30 GB. There are 6 job types, and each job type on average requires 16 files (each is 1 GB) for execution.

5.3 Simulation Results and Discussion In this experiment a combination of various replication and scheduling algorithms are evaluated. Four scheduling strategies have been considered, as follows: (a) Random scheduler selects a computing node to execute specific job randomly. (b) Shortest Queue scheduler calculates all of the queue length of computing nodes and selects one that has the least number of jobs waiting in the queue. (c) Access Cost scheduler assigns the job to computing element where the file has the lowest access cost (cost to get all files needed for executing job). (d) Queue Access Cost scheduler selects computing element where has the smallest sum of the access cost for the job and the access costs of all jobs waiting in the queue. More details about these schedulers are given in [16]. Also five replication algorithms have been used for evaluation, namely: (a) Least Frequently Used (LFU) always replicates files to local storage system. If available space in local storage is not enough, the replica which has been accessed the least number of times in the past is deleted. (b) Least Recently Used (LRU) always replicates files to local storage system. If the local storage space is full, replica that has the oldest last access time is deleted to make space for new replicas.

276

N. Mansouri et al.

Fig. 3 Grid topology in the simulation

Table 1 Simulation parameters

Topology parameters

Values

Number of regions Storage space at each site (GB) Inter LAN bandwidth (Mpbs) Intra LAN bandwidth (Mpbs) Intra region bandwidth (Mpbs)

3 30 1,000 100 10

Grid job parameter

Value

Number of jobs Number of jobs types Number of file access per jobs Size of single file (GB) Total size of files (GB) Job delay (ms)

2,000 6 16 1 97 2,500

(c) Bandwidth Hierarchy based Replication (BHR) [7] decreases the data access time by maximizing network-level locality and avoiding network congestions. Sites are divides into several regions. Network bandwidth between the regions is lower than the bandwidth within the regions. So, if the required file is placed in the same region, its fetching time will be less. Also, if the new replica is already duplicated in other site in the same region, the optimizer in BHR will terminate. (d) Modified BHR [17] is an extension of BHR strategy. The difference between Modified BHR and BHR is Modified BHR tries to replicate files in a site where has accessed the file for the most time based on the assumption that it may also be required in future.

Performance Evaluation of a New Scheduling Algorithm

277

Fig. 4 Mean job execution time for various job scheduling and replication algorithms

(e) 3-Level Hierarchical Algorithm (3LHA) [15] is replication algorithm for a 3-level hierarchy structure. It considered a hierarchical network structure that has three levels. First level are Regions that are connected through internet i.e. have low bandwidth. Second level comprises local area network’s (LAN) within each region that have moderately higher bandwidth comparing to the first level. Finally the third level are the nodes (sites) within each the LAN’s, that are connected to each other with a high bandwidth. 3LHA among the candidate replicas selects the one that has the highest bandwidth to the requested file. Similarly, use the same technique for file deletion. Figure 4 shows the mean job execution time for 2000 jobs using the above mentioned scheduling and replication strategies. 3LHA has better performance than BHR since it considers the differences between intra-LAN and inter-LAN communication. Also, if the available storage for replication is not enough, 3LHA will only replicate those files that are not available in the local LAN. So it will not delete those file that have a high cost of transfer i.e. bandwidth is an important factor for deleting and this leads to a better performance comparing with LRU and LFU methods. Modified BHR improves the mean job execution time by storing the replica in the most frequently accessed site. Figure 4 shows that a suitable choice of data replication algorithm can enhance Grid performance. In Random scheduling the mean job execution time obviously increases because it doesn’t consider effective factors. It probably selects CE that has large number of jobs in the queue and it is possible to be idle in other CEs or must spend a lot of transmission time for getting required files. In Shortest Job Queue Scheduling each CE receive approximately the same number of jobs. If CE’s have low network bandwidth, then file transfer time will be high and overall job execution time will increase. Access Cost Scheduling selects a CE based on its access cost. CE’s with lower access cost may receive large number of jobs to execute.

278

N. Mansouri et al.

Fig. 5 Mean job time based on varying number of jobs

Fig. 6 Mean job execution time based on varying size of files

So, overall performance is decreased. The Queue Access Cost considers not only shortest job queue but also access cost. Therefore, the Queue Access Cost decrease total job execution time. Finally, CSS is the best job scheduling algorithms for the large number of jobs because it schedules jobs close to the data whilst ensuring sites with high network connectivity are not overloaded and sites with poor connectivity are not left idle. Figure 5 displays the mean job time based on changing number of jobs that use sequential access pattern. As number of jobs increases the performance of CSS increases in comparison with other methods. It is similar to a real grid environment where a lot of jobs should be executed. Figure 6 shows the effect of file size on the execution time for 5 algorithms. CSS has less job execution time in comparison with other strategies especially when size of the files is increased. It is necessary to note that in data-intensive applications, the locations of data required by the job affects the performance

Performance Evaluation of a New Scheduling Algorithm

279

significantly. This performance will be more visible in the real data grid environments where dataset sizes reach to many terabytes.

6 Conclusion and Future Work Replication and scheduling problem has been investigated both separately and jointly by researchers. Effective scheduling of jobs in a dynamic Data Grid environment is complex since it deals with huge amount of data. In this paper, a novel Combined Scheduling Strategy (CSS) is proposed that considers number of jobs waiting in queue, the location of required data for the job and the computing capacity of sites. To evaluate the efficiency of the job scheduling policy, OptorSim is used to simulate a real data. Results are shown for various scheduling and replica optimization policies. It is evident that the choice of scheduling algorithm has a significant impact on system performance. CSS takes less job execution time than other strategies especially when number of jobs or size of the files or both increases. For future work, we plan to evaluate CSS with more scheduling and replication algorithms as well as investigating more complicated scheduling algorithms and replica placement strategies to improve the overall system performance.

References 1. Tang M, Lee BS, Tang X, Yeo C (2006) The impact of data replication on job scheduling performance in the data grid. Fut Gener Comput Syst 22:254–268 2. Rangmathan K, Foster I (2003) Simulation studies of computation and data scheduling algorithms for data grid. J Grid Comput 1:53–62 3. Takefusa A, Tatebe O, Matsuoka S, Morita Y (2003) Performance analysis of scheduling and replication algorithms on grid datafarm architecture for high-energy physics applications. In: Proceedings of the 12th IEEE international symposium on high performance distributed computing (HPDC’03) 4. Zhang J, Lee B, Tang X, Yeo C (2008) Impact of parallel download on job scheduling in data grid environment. Seventh international conference on grid and cooperative computing 5. Mohamed HH, Epema DHJ (2004) An evaluation of the close-to-files processor and data co-allocation policy in multiclusters. International conference on cluster computing, IEEE Society Press, San Diego, pp 287–298 6. Chang R, Chang J, Lin S (2007) Job scheduling and data replication on data grids. Fut Gener Comput Syst 23:846–860 7. Park S, Kim J, Ko Y, Yoon W (2004) Dynamic data replication strategy based on internet hierarchy BHR. Lecture notes in computer science publisher, vol 3033. Springer, Heidelberg, pp 838–846 8. Foster I, Ranganathan K (2001) Design and evaluation of dynamic replication strategies for high performance data grids. In: Proceedings of international conference on computing in high energy and nuclear physics, Beijing

280

N. Mansouri et al.

9. Foster I, Ranganathan K (2002) Identifying dynamic replication strategies for high performance data grids. In: Proceedings of 3rd IEEE/ACM international workshop on grid computing. Lecture Notes on Computer Science, vol 2242, Denver, pp 75–86 10. Foster I, Ranganathan K (2002) Decoupling computation and data scheduling in distributed data-intensive applications. In: Proceedings of the 11th IEEE international symposium on high performance distributed computing, HPDC-11, IEEE. CS Press, Edinburgh, pp 352–358 11. Vazhkudai S (2003) Enabling the co-allocation of grid data transfers. In: Proceedings of the fourth international workshop on grid computing 12. Suri PK, Singh M (2009) JS2DR2: An effective two-level job scheduling algorithm and twophase dynamic replication strategy for data grid. International conference on advances in computing, control, and telecommunication technologies 13. OptorSim—a replica optimiser simulation 14. http://grid-data-management.web.cern.ch/grid-datamanagement/optimisation/optor/ 15. Horri A, Sepahvand R, Dastghaibyfard Gh (2008) A hierarchical scheduling and replication strategy. Int J Comput Sci Netw Secur 8 16. Cameron DG, Carvajal-Schiaffino R, Paul Millar A, Nicholson C, Stockinger K, Zini F (2003) Evaluating scheduling and replica optimisation strategies in OptorSim. 4th international workshop on grid computing (Grid2003), Phoenix, Arizona, IEEE Computer Society Press 17. Sashi K, Selvadoss Thanamani A (2011) Dynamic replication in a data grid using a modified BHR region based algorithm. Fut Gener Comput Syst (Elsevier) 27:202–210

A Novel Dynamic Replication Algorithm in Hierarchical Data Grid N. Mansouri and Gh. Dastghaibyfard

Abstract Data grid provides massive aggregated computing resources and distributed storage space to deal with data-intensive applications. Due to limitation of available resources in grid as well as production the large number of workloads, efficient use of the grid resources becomes important challenge. Replication provides decreased bandwidth consumption, improved fault tolerance, and enhanced scalability. Although replication is a key technique, but the problem of selecting proper locations for placing replicas i.e. best replica placement in Data Grid has not been widely studied, yet. In this paper a modified three-layer hierarchical replication algorithm is proposed for dynamic creation of replicas, replica selection and especially replica placement which selects best location. The simulation and experiment results show the proposed replication strategy outperforms over current strategies about 14%. contain at least 70 and at most 150 words. It should be set in nine-point font size and should be inset 1.0 cm from the right and left margins. There should be two blank (10-point) lines before and after the abstract. This document is in the required format. Keywords Data Grid


Replication
Simulation

N. Mansouri (&)
Gh. Dastghaibyfard Department of Computer Science and Engineering, College of Engineering, Shiraz University, Shiraz, Iran e-mail: [email protected] Gh. Dastghaibyfard e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_27,  Springer Science+Business Media B.V. 2012

281

282

N. Mansouri and Gh. Dastghaibyfard

Fig. 1 Grid layer architecture

1 Introduction Data intensive applications such as high-energy physics, bioinformatics, climate modeling, earthquake engineering. which process and produce huge amounts of data, are the building blocks in Data Grid. The big challenge is how to manage these amounts of data that require high capacity mass storage systems; high bandwidth networks and finally how to disseminate efficiently such data among users that are distributed over a wide geographic area. Data Grids aggregate a collection of distributed resources placed in different parts of the world to enable users to share data and resources [1–3]. Figure 1 shows the components of a Data Grid in a layered architecture [4, 5]. The grid fabric layer involves distributed computational resources, storage resources, and instruments that are connected by high bandwidth. The connectivity layer contains protocols used to query resources in the grid fabric layer and to control transferring data between them. Data Grid service layer consists of several services such as replication, data discovery and resource brokering. The application layer contains the user applications that work within a virtual organization environment. The size of data that are requested in Data Grid is from terabytes to petabytes. Meanwhile, even though the memory and storage size of new computers are ever increasing, they are still not keeping up with the request of storing large number of data. The major challenge is a decision problem i.e. how many replicas should be

A Novel Dynamic Replication Algorithm

283

created and where replicas should be stored. Hence methods needed to create replicas that increase availability without using unnecessary storage and bandwidth. There are three key issues in all the data replication algorithms as follows: • Replica selection: process of selecting replica among other copies that are spread across the grid. • Replica placement: process of selecting a grid site to place the replica. • Replica management: the process of creating or deleting replicas in Data Grid. Generally, replication algorithms are either static or dynamic. In static approaches the created replica will exist in the same place till user deletes it manually or its duration is expired. On the other hand, dynamic strategies create and delete replicas according to the changes in grid environments, i.e. users’ file access pattern. Almost all data replication studied in Data Grid environments have focused on creating and deleting replicas methods [6–11], but best replica placement has not drawn much attention of researchers. Best replica placement is an important issue and it is considered in this paper, since it plays an important role for obtaining maximum benefit from replication as well as reducing storage cost and mean job execution time. In this work a novel data replication strategy, called Modified three-Layer Hierarchical Replication (M3LHR) algorithm is proposed. M3LHR extends proposed algorithm in [12] and stores each replica in an appropriate site i.e. best site in the requested region that has the highest number of access for that particular replica. The paper is organized as follows. Section 2 gives an overview of some related work on data replication strategies. In Sect. 3 a M3LHR algorithm is proposed. Section 4 shows the simulations results. Finally, conclusions and future research works are presented in Sect. 5.

2 Related Work Recently, modeling Data Grid environments and simulating different data replication strategies as well as basic file replication protocols [13] has drawn researchers attention. In [14] the authors have presented two dynamic replica strategies, Simple Bottom Up and Aggregate Bottom Up, for the multi-tiered grid. The main point of these strategies is to place the replicas as close as possible to the clients. Files are replicated from down levels to up levels according to their popularity. Both strategies decrease the mean response time of data access in comparison with the static replication algorithm in a multi-tiered grid. Fast Spread is a replication algorithms that works well for random request patterns [15]. It places replica at each node along its path to the best client. If one of these nodes does not have enough free storage for the new replica, a group of replicas may be deleted without considering such deleted files may be more

284

N. Mansouri and Gh. Dastghaibyfard

important than the new replica. To overcome this problem Bsoul et al. [16] extended Fast Spread strategy by proposing a dynamic threshold that determines the replica should be stored at each node or not. The results showed that the proposed strategy achieved better performance compared with Fast Spread. Zhao et al. [17] proposed a dynamic optimal replication strategy (DORS) to overcome the problem of limited storage capacity in each node. They presented a replica replacement strategy based on the access history, file’s size and the network condition. They also empirically presented a threshold to determine whether one file should be replicated in other nodes or not. They achieved better performance in comparison with LFU and LRU strategies. In [18], Khanli et al. presented a dynamic replication strategy for a multi-tier data grid, i.e. predictive hierarchical fast spread (PHFS) which is an extended version of fast spread. PHFS predicts future requests according to spatial locality, and pre-replicates them in various layers of the multi-tier data grid to improve storage resources usage. They showed PHFS strategy is appropriate for applications in which the clients use a context for duration of time, similar to the scientific applications that users work on a project. Park et al. [19] presented a Bandwidth Hierarchy based Replication (BHR) which decreases the data access time by maximizing network-level locality and avoiding network congestions. They divided the sites into several regions. Network bandwidth between the regions is lower than the bandwidth within the regions. So, if the required file is placed in the same region, its fetching time will be less. BHR strategy has a deficiency, replicated files are placed in all the requested sites not the appropriate sites. In [20] the authors introduced a concept called fragmented replicas. When replication takes place, instead of copying the whole replica, it just stores partial contents that is needed locally. It can significantly save the storage space for not storing unnecessary data. They also presented a block mapping procedure to set the distribution of blocks in every available server for next replica retrieval. A client can retrieve a fragmented replica quickly by employing the block mapping procedure. The results of simulation indicated that the co-allocation schemes enhance download efficiency in a fragmented replication system. Another replication method, called Branch replication schema (BRS) have been presented in [21]. BRS provides more scalability than serial solutions through the parallel replication of a file in several sites. In BRS each replica is comprised of a disparate set of subreplicas organized using a hierarchical tree topology where subreplicas of a replica do not overlap. To improve the scalability and performances for both read and write operation, they used parallel approach [22]. The simulation results showed that BRS improves data access performance for files with different sizes and provides an efficient way for updating data replicas. Foster and Ranganathan [15], have proposed six distinct replica strategies No Replica, Best Client, Cascading Replication, Plain Caching, Caching plus Cascading Replica and Fast Spread) for multi-tier data. They also introduced three types of localities, namely:

A Novel Dynamic Replication Algorithm

285

• Temporal locality: The files accessed recently are much possible to be requested again shortly. • Geographical locality: The files accessed recently by a client are probably to be requested by adjacent clients, too. • Spatial locality: The related files to recently accessed file are likely to be requested in the near future. These strategies evaluated with different data patterns: first, access pattern with no locality. Second, data access with a small degree of temporal locality and finally data access with a small degree of temporal and geographical locality. The results of simulations indicate that different access pattern needs different replica strategies. Cascading and Fast Spread performed the best in the simulations. Also, the authors combined different scheduling and replication strategies. In a data grid reducing job’s waiting time in queue and execution time depends on where to assign job for execution and where to get the required data files. Therefore, scheduling jobs at proper sites and getting replicas from proper sites are important factors from user’s point of view. We leave the job scheduling problem [23], which studies how to dispatch jobs into Grid sites for execution, and it’s coupling with data replication, as our future research. Some researchers have studied the relationship between data replication and job scheduling [24–26].

3 The Proposed Method In this section, first network structure is described, and then a novel M3LHR algorithm is proposed.

3.1 Network Structure The grid topology of simulated platform is given in Fig. 2. Which has three levels similar to what is given in [12] i.e. regions’, LAN’s within each region and nodes within each LAN.

3.2 Modified Three-Layer Hierarchical Replication Algorithm When a job is allocated to local scheduler, before job execution the replica manager should transfer all the required files that are not available. So, the data replication enhances the job scheduling performance by decreasing job turnaround time. M3LHR has three parts:

286

N. Mansouri and Gh. Dastghaibyfard

Fig. 2 Grid topology in the simulation

Replica Selection: For all requested files of job i, Replica Manager first tries to select the unavailable requested file if possible within the local LAN next within the local region. Otherwise it generates a list of replicas that are available in other regions. Then from this list it selects the replica that has the lowest transfer time to requested site. Replica Placement: According to the temporal and geographical locality the replica is placed in the best site (BSE). To select the BSE, M3LHR creates a sorted list (by number of replica access) of all SE’s that requested that particular replica in the region. Now the replica will be placed in the first SE of the above sorted list, if more than one SE is candidate for BSE, one can be selected randomly. Therefore, replica is not placed in all the requested sites. Hence, storage cost as well as mean job execution time can be decreased. Assume list 1 shows the sorted list created for replica R, now M3LHR selects site S7 from LAN3 which is shown in Fig. 3. Replica Management: If enough storage space exists in the BSE, the selected file is replicated. Otherwise if the file is available in the local LAN, then the file is accessed remotely. Now, if enough space for replication does not exist and requested file is not available in the same LAN, one or more files should be deleted using the following rules: • Generate a least recently used (LRU) sorted list of replicas that are both available in the site as well as the local LAN. Now start deleting files from the above list till space is available for replica. • If space is still insufficient, then repeat previous step for each LAN in current region randomly. • If space is still insufficient, generate a LRU sorted list of the remaining files in the site and start deleting files from the above list till space is available for replica. Fig. 4 describes M3LHR strategy.

A Novel Dynamic Replication Algorithm

287

Fig. 3 Replica selection and replica placement strategies

4 Experiments 4.1 Simulation Tool M3LHR is evaluated using OptorSim [27], a simulator for Data Grids, which is s developed by the European Data Grid (EDG) project. OptorSim has several important components such as computing element (CE), storage element (SE), resource broker (RB), replica manager (RM), and replica optimizer (RO). Computing elements and storage elements are used to execute grid jobs and store the files respectively. Additional details about OptorSim are available in the literatures [28, 29].

4.2 Configuration With OptorSim, it is possible to simulate any grid topology and replication strategy. So OptorSim code has been modified to implement the hierarchical structure, since it uses a flat network structure. It is assumed the network has three

288

N. Mansouri and Gh. Dastghaibyfard

Fig. 4 M3LHR algorithm

regions and on average two LAN’s in every region. The storage capacity of the master site is 250 GB, and the storage capacity of all other sites is 30 GB. Bandwidth in each level is given in Table 1. The number of storage elements and the number of computing elements are set 11 and 10, respectively. Table 2 specifies the simulation parameters and their values used in our study. There are six job types, and each job type on average requires 16 files (each is 2 GB) for execution. To simplify the requirements, we assumed that the data is read-only.

A Novel Dynamic Replication Algorithm

289

Table 1 Bandwidth configuration Parameter

Value (Mpbs)

Inter LAN bandwidth (level3) Intra LAN bandwidth (level 2) Intra Region bandwidth (level 1)

1000 100 10

Table 2 General job configuration Parameter

Value

Number of jobs Number of jobs types Number of file access per jobs Size of single file (GB) Total size of files (GB) Job delay (ms) Maximum Queue Size

500 6 16 2 200 2500 200

4.3 Simulation Results and Discussion The proposed M3LHR algorithm is compared with four replication algorithms namely, No Replication, Least Frequently Used (LFU), Least Recently Used (LRU), and BHR. In No Replication strategy files are accessed remotely. When storage is full, LRU deletes least recently accessed files and LFU deletes least frequency accessed files. The BHR algorithm stores the replicas in a site that has a high bandwidth and replicates those files that are likely to be requested soon within the region. Figure 5 shows the mean job execution time for the sequential access pattern. Obviously, the No Replication strategy has the worst performance as all the files requested by jobs have to be transferred from CERN. In this simulation LRU and LFU have almost the same execution time. BHR improves data access time by avoiding network congestions. M3LHR mean job execution time is about 14% faster than BHR since it considers the differences between intra-LAN and interLAN communication. If the available storage for replication is not enough M3LHR will not delete those file that have a high transferring time. It also improves the mean job execution time by storing the replica in the most frequently accessed site of the requested region. Valuable information can be gained by monitoring storage resources usage. Since resource cost is proportional to the resource used, so minimizing storage usage is a must. Figure 6 shows the storage usage which is the percentage of available spaces that are used. No Replication strategy has best storage since it gets files from CERN. LFU and LRU are always replicate when a request is made, hence they use a lot of space. BHR strategy performs better than the previous three strategies since it keeps at most one copy of file in the region. The proposed M3LHR strategy has minimum storage usage among the current algorithms because it place replicas in the appropriate site so reduces unnecessary replication.

290

N. Mansouri and Gh. Dastghaibyfard

Fig. 5 Mean job execution time for various replication algorithms

Fig. 6 Storage resources usage for various replication algorithms

Fig. 7 Mean job execution time based on varying size of files

Figure 7 shows the mean job time for varying file sizes for the four algorithms. Since the storage size is fixed, increasing in file size will decrease the average number of copies of each file, so placing replicas in the suitable sites significantly increases performance. Here M3LHR outperforms the other methods as file size

A Novel Dynamic Replication Algorithm

291

increases since it prevents from creating unnecessary replicas. It is expected that even in the real data grid environments the difference will be even more considerable as dataset sizes reach to many terabytes.

5 Conclusion and Future Work Data replication is a frequently used technique that can enhance the data availability and fault tolerance. Since a grid environment is dynamic, network latency and user behavior may change. To address these issues and storage capacity limitation, a well-designed dynamic replication strategy is necessary in data grid. In this paper a dynamic data replication strategy, called M3LHR for a three level hierarchical structure is proposed. M3LHR deletes those file that are exist in local LAN (i. e. only files with minimum time for transferring) when free space is not enough for the new replica. It also stores the replicas in the best site where the file has been accessed for the most time instead of storing files in many sites. Experimental results of M3LHR with OptorSim simulator show that it has less job execution time in comparison with other strategies especially when the grid sites have comparatively small storage size. Also, that performance increases as file size increases, since the proposed method decreases unnecessary replication by placing replica in the best site. To the best of our knowledge, in the real Data Grid environments these advantages will be even more significant as dataset sizes reach to many terabytes. In future work, M3LHR can be combined with a proper scheduling to improve performance. Data transferring between different grid sites is time consuming and consequently scheduling jobs to the appropriate sites is necessary. Searching for advanced replica replacement methods certainly enhances replication strategies. Acknowledgments The authors would like to thank Iran Telecommunication Research Center (www.itrc.ac.ir) for their financial support.

References 1. Figueira S, Trieu T (2008) Data replication and the storage capacity of data grids. Springer, Berlin, Heidelberg, pp 567–575 2. Lamehamedi H, Szymanski B, Shentu Z, Deelman E (2002) Data replication strategies in grid environments.In: Proceedings of the 5th international conference on algorithms and architectures for parallel processing, pp 378–383 3. Cameron D, Millar A, Nicholson C, Carvajal-Schiaffino R, Stockinger K, Zini F (2004) Analysis of scheduling and replica optimisation strategies for data grids using OptorSim. J Grid Comput 2:57–69 4. Foster I, Kesselman C, Tuecke S (2001) The anatomy of the Grid: enabling scalable virtual organizations. Int J High Perform Comput Appl 15:200–222

292

N. Mansouri and Gh. Dastghaibyfard

5. Baker M, Buyya R, Laforenza D (2002) Grids and grid technologies for wide-area distributed computing. Softw Pract Exper 32:1437–1466 6. Chervenak A, Schuler R, Kesselman C, Koranda S, Moe B (2005) Wide area data replication for scientific collaborations. In: Proceedings of the 6th international workshop on grid computing, pp 1–8 7. David WB (2003) Evaluation of an economy-based file replication strategy for a data grid. International workshop on agent based cluster and grid computing, pp 120–126 8. Deris M, Abawaji J, Suzuri H (2004) An efficient replicated data access approach for largescale distributed systems.In: IEEE international symposium on cluster computing and the grid 9. Ranganathan K, Iamnitchi A, Foster I (2002) Improving data availability through dynamic model-driven replication in large peer-to-peer communities. In 2nd IEEE/ACM international symposium on cluster computing and the grid, pp 376–381 10. Chang R, Chang J, Lin S (2007) Job scheduling and data replication on data grids. Futur Gener Comput Syst, Elsevier, 23: 846–860 11. Abdi S, Pedram H, Mohamadi S (2010) The impact of data replication on job scheduling performance in hierarchical data grid. Int J Appl Graph Theory in Wireless Ad hoc Netw Sensor Netw (GRAPH-HOC) 2 12. Horri A, Sepahvand R, Dastghaibyfard Gh (2008) A hierarchical scheduling and replication strategy. Int J Comput Sci Netw Secur, vol 8 13. Stockinger H, Samar A, Allcock B, Foster I, Holtman K, Tierney B (2002) File and object replication in data grids. J Clust Comput 5:305–314 14. Tang M, Lee BS, Yao CK, Tang XY (2005) Dynamic replication algorithm for the multi tier data grid. Futur Gener Comput Syst, Elsevier 21: 775–790 15. Foster I, Ranganathan K (2002) Identifying dynamic replication strategies for high performance data grids.In: Proceedings of 3rd IEEE/ACM international workshop on grid computing, Lecture notes on computer science, vol 2242, pp 75–86 16. Bsoul M, Al-Khasawneh A, Kilani Y, Obeidat I (2010) A threshold-based dynamic data replication strategy. J Supercomput 17. Zhao W, Xu X, Wang Z, Zhang Y, He S (2010) A dynamic optimal replication strategy in data grid environment.In: International conference on internet technology and applications 18. Khanli LM, Isazadeh A, Shishavan TN (2010) PHFS: a dynamic replication method, to decrease access latency in the multi-tier data grid. Futur Gener Comput Syst, Elsevier, vol 27, pp 233–244 19. Park S, Kim J, Ko Y, Yoon W (2004) Dynamic data replication strategy based on internet hierarchy BHR. Lecture notes in computer science, Springer, Heidelberg, vol 3033, pp 838–846 20. Chang R, Chen P (2007) Complete and fragmented replica selection and retrieval in Data Grids. Futur Gener Comput Syst, Elsevier, vol 23, pp 536–546 21. Pérez JM, Carballeira F, Carretero J, Calderón A, Fernández J (2009) Branch replication scheme: a new model for data replication in large scale data grids. Futur Gener Comput Syst, Elsevier, vol 26, pp 12–20 22. Carballeira F, Carretero J, Calderon A, Garcia JD, Sanchez LM (2007) A global and parallel file systems for grids. Futur Gener Comput Syst 23:116–122 23. Venugopal S, Buyya R (2008) An scp-based heuristic approach for scheduling distributed data-intensive applications on global grids. J Parallel Distributed Comput 68:471–487 24. Chakrabarti A, Sengupta S (2008) Scalable and distributed mechanisms for integrated scheduling and replication in data grids. Proceedings of 10th International Conference on Distributed Computing and Networking (ICDCN) 25. Chervenak A, Deelman E, Livny M, Su M, Schuler R, Bharathi S, Mehta G, Vahi K (2007) Data placement for scientific applications in distributed environments. Proceedings of IEEE/ ACM international conference on grid computing 26. Dang NN, Lim SB (2007) Combination of replication and scheduling in data grids. Int J Comput Sci Netw Secur 7:304–308

A Novel Dynamic Replication Algorithm

293

27. OptorSim, http://edg-wp2.web.cern.ch/ 28. Cameron D, Carvajal-Schiaffino R, Millar P, Nicholson C, Stockinger K, Zini F (2003) Evaluating scheduling and replica optimization strategies in OptorSim. Forth international workshop on grid computing, Phoenix, USA 29. Bell W, Cameron D, Carvajal-Schiaffino R, Millar P, Nicholson C, Stockinger K, Zini F (2004) OptorSim v1.0 installation and user guide

Intelligent Application Migration within a Self-Provisioned Hybrid Cloud Environment Wei-Jen Wang, Yo-Ming Lo, Shao-Jui Chen and Yue-Shan Chang

Abstract A typical hybrid cloud connects a private data center to a portion of a public cloud. It can potentially scale up the computing pool of a private cloud on demand. This work presents a novel adaptive computing framework that enables intelligent automatic application migration between a private data center and a public cloud. In the case that the workload of the private data center is too high, the proposed framework can create more computing resources in the public cloud automatically, and then move applications to the public cloud in order to reduce the workload of the private data center. The proposed framework employs the multi-agent technique to manage all resources, monitor system behavior, and negotiate all actions in the hybrid cloud. We have built a prototype of a hybrid cloud environment to demonstrate that the proposed framework is able to perform intelligent application migration and self-provisioning. Keywords Application migration based intelligence


Cloud computing
Hybrid cloud
Agent-

1 Introduction The emerging cloud computing technology [1–3] aims to deliver flexible, ondemand computing resources as services to different users. According to the NIST’s definition of cloud [4], a cloud is a share pool of configurable computing W.-J. Wang (&)
Y.-M. Lo
S.-J. Chen Department of Computer Science and Information Engineering, National Central University, 300 Jhongda Rd., Jhongli, Taoyuan, Taiwan Y.-S. Chang Department of Computer Science and Information Engineering, National Taipei University, 151 University Rd., Sanhsia, New Taipei, Taiwan

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_28,  Springer Science+Business Media B.V. 2012

295

296

W.-J. Wang et al.

Fig. 1 The idea of the proposed framework

resources that can be rapidly provisioned and released. Recently, the deployment model for hybrid clouds [5–8] is becoming more popular in both the academia and the industry. A typical hybrid cloud [9] connects a private datacenter to a portion of a public cloud through a virtual private network (VPN). It adopts the concept of Virtual Private Cloud (VPC) [8–10], which is a user-defined group of virtual resources on a public cloud. In 2010, Amazon announced its virtual private cloud [8] service that can provision a private, isolated cloud on Amazon Web Services (AWS) Cloud, where the users can launch computing resources in a user-defined virtual network. In Taiwan, the cloud computing center of Industrial Technology and Research Institute (ITRI) [11] also launched a hybrid cloud project to support the VPC service in their public cloud. One benefit of using VPC is that, the VPC model can scale up the computing pool of a private cloud on demand. When the workload of a private cloud is high, the hybrid cloud environment can dynamically create several virtual machine instances in the public cloud to form a VPC and then move some workload to the VPC. As a result, the quality of service in a hybrid cloud can be ensured by resource provisioning and service migration. To enable intelligent, dynamic application migration in a hybrid cloud, one may adopt the agent technology [12– 15] for system monitoring, self-provisioning, computation coordination, and automatic load balancing. Ian Foster et al. [12] have pointed out that agent technology can enhance the ability of problem solving of grid. We believe it can enhance the cloud computing environment as well. For example, a multi-agent intelligent system can be installed on a hybrid cloud. Some agent is responsible for system monitoring, while some decides how and when to perform system reconfiguration in a hybrid cloud. This research aims to provide an intelligent, agent-based framework that supports dynamic application migration in a hybrid cloud, in particular dynamic application migration between the public cloud and the private cloud. Figure 1 shows the concept of proposed framework where there are two major components. The first component is a smart multi-agent system that detects the status of the hybrid cloud and manages the resources in the hybrid cloud. We used the Java Agent Development framework (JADE) [16] to implement this component. The

Intelligent Application Migration

297

second component is a hybrid cloud environment where a set of mobile applications are running. The proposed framework is able to adjust the workload by moving applications automatically. When existing computing resources are not enough to serve all applications, the proposed framework will notify the public cloud to create more computing resources automatically. We have built a prototype of the proposed framework. In the future, we plan to implement the proposed framework on a larger hybrid cloud environment that connects a small private data center in National Central University and a virtual private cloud in the ITRI public cloud [11].

2 Related Work Agent-based systems [12–18] have been widely studied in the past decade. The fundamental computing element in an agent-based system is the agent, which acts as an independent, identical, and autonomous process with the ability to communicate with other agents [16]. Different agents may perform different tasks, and they can work together to accomplish a complex task in a distributed environment. Some agent-based systems such as JADE [14, 16] and SALSA [19] support dynamic agent migration. The mobility feature gives more flexibility and more efficiency to programmers and system administrators [13]. Many software engineers and researchers focused their research on either the study of agent-based systems [12–14, 16–19] or the study of hybrid cloud models [5, 6, 8–10]. However, the topic of automatic service and application migration in a cloud is relatively scarce in the literature. Zhang et al. [7] proposed a workload factoring service to make the best use of public cloud services along with data centers. They used a fast algorithm to detect frequent data items, and enabled factoring incoming requests on volume and data content upon an environment where data popularity may change in different time slots. Amoretti et al. [15] presented a framework that supports code mobility, and used a pull strategy for service migration. Kaisler and Money [20] proposed a model to examine feasibility of service migration in a cloud computing environment by considering security, integration, and service implementation. Hao et al. [21] considered the problem of service selection and migration in clouds. They developed a framework to facilitate service migration, and designed a cost model and a genetic algorithm to determine the tradeoffs on service selection and migration.

3 The Proposed Framework This section will present the proposed framework, consisting of a smart multiagent system called the federated broker, and a hybrid cloud environment that hosts a set of mobile applications. In our hybrid cloud computing model, we

298

W.-J. Wang et al.

assume that the resources in the private data center must be fully utilized before any resources in the public cloud are used. This assumption comes from the constraint that the users must pay for using the public cloud resources and people do not want to waste money for no reason. We also assume that applications are always submitted to the private data center. The applications then wait in a queue for execution. An application is executed in the private data center directly if the private data center is capable of running the application. Otherwise, it may be transferred to the public cloud for execution.

3.1 Federated Broker The federated broker, mostly implemented in JADE [16], is a smart multi-agent system for system monitoring and reconfiguration decision. The federated broker consists of three major components—the system monitoring agent, the service migration agent, and the reconfiguration decision agent. Figure 2 shows the three major components of the federated broker and the interactions among these components and the hybrid cloud environment. Each cloud of the hybrid cloud environment has a cloud administration agent in it to interact with the federated broker. It gathers the information of physical machines and applications, and sends the data to the system monitoring agent of the federated broker. The system monitoring agent then processes the primitive data and saves the digested data in a profiling database. The reconfiguration decision agent periodically retrieves data from the system monitoring agent and the profiling database, and then uses the information to decide whether the system requires reconfiguration. If the reconfiguration decision agent decides to perform reconfiguration, it will notify the service migration agent to move some applications from the private data center to the public cloud. Virtual machines in the public cloud will be created if the computing pool in the public cloud is not enough. The three major components of the federated broker are described below: • System Monitoring Agent (SyMA): the agent receives data from cloud administration agents, such as the identity of an application, the status of an application, and the workload information of the hardware. Then it filters the data and puts historical data in a profiling database. It does not make any decision for system reconfiguration. Some fresh data such as the current state of the private data center are sent to the reconfiguration decision agent directly. • Reconfiguration Decision Agent (RDA): the agent is supposed to use the current state of the private center and the historic data in the profiling database to predict the future workload of the private data center. The prediction result and a pre-defined reconfiguration policy are used together to decide how to adjust the workload ratio of the private data center and the public cloud. When the workload of the private data center is high, the RDA will create more virtual machine instances in the public cloud and move applications to the public cloud.

Intelligent Application Migration

299

Fig. 2 The federated broker and its interactions with the hybrid cloud

It also periodically evaluates the state of the public cloud, and may terminate some virtual machine instances in the public cloud if they are not used. • Service Migration Agent (SeMA): while the agent receives a reconfiguration decision message, it will send commands to the private data center and the public cloud to ask for application migration. If the public cloud does not have enough virtual machines to host incoming applications, the agent will check the reconfiguration policy and then create more virtual machines on the public cloud if necessary and legal. The agent does not decide which application should move to the public cloud. Instead, it tells the manager of the private data center to move a certain percentage of workload to the public cloud.

3.2 Hybrid Cloud Environment The hybrid cloud environment consists of two set of computing resources. One is the private data center, and the other is the public cloud that provisions resources on demand. There are three major components in the hybrid cloud environment. Figure 3 shows how the three components in the hybrid cloud environment interact with users and the federated broker. The details of them are described below: • Local Monitoring Agents (LMA): each machine of the private data center has a local monitoring agent in it. The LMA periodically transmits the observed machine status to the private cloud administration agent, which will then organize the data and send the data to the system monitoring agent of the

300

W.-J. Wang et al.

Fig. 3 The federated broker and its interactions with the hybrid cloud

federated broker. Note that the public cloud does not have any LMA in it. Similar functionality should be provided through the API of the public cloud. • Cloud Administration Agent (CAA): both the public cloud and the private data center of the hybrid cloud environment have their own cloud administration agent. Each CAA is a central manager of its computing pool. It is responsible to gather system information and to send the information to the federated broker. It is also responsible to receive commands from the federated broker and to execute the commands for the federated broker. • Application Management Agent (AMA): the AMA is designed to manage applications in the private data center or the public cloud. To execute an application, the user has to submit the application to the AMA in the private data center. The application then waits in a queue. As soon as the application is selected for execution, it will be registered in the private CAA and then be executed in the private data center. When the AMA receives a message for application reconfiguration from the federated broker, it will ask the selected application to save its state. The state along with the program of the application will be sent to the public CCA by the private CCA. Then, the migrated application is resumed by the public CCA.

4 Migration Policies and Preliminary Results The current implementation of the proposed framework uses average CPU load and the number of applications as the criteria to judge whether a reconfiguration is required. To execute an application in the proposed framework, a pre-define

Intelligent Application Migration

301

number of virtual machines for the application must be provided. The information will be used to judge whether the resources are enough to serve a newly incoming application. The framework will create enough virtual machines in the public cloud to host the application. Applications that are submitted to the AMA of the private data center are put in a waiting queue first. A waiting application is executed immediately if there are enough resources in the private data center. Otherwise, the private CAA notifies the federated broker for the situation, and then the federated broker contacts the public CAA to create more resources for the waiting applications. When the resources are ready, the public CAA will notify the federated broker, which will then ask the private CAA to migrate some applications to the public cloud. Finally, some waiting applications are moved to the public CAA for execution to reduce the workload in the private data center. We have built a prototype consisting of one server (2 quad-core CPUs and 8 GB RAM) with four VMs as the private data center and another server (each with 2 quad-core CPUs and 16 GB RAM) as the public cloud. The maximum number of VMs that can be created in the public cloud is set to be 5. We designed an experiment to evaluate the prototype. In the experiment, we submitted 5, 10, 15, and 20 applications to the prototype. Each application is a distributed application that demands 2*3 machines to calculate a maximum likelihood estimation problem with execution time of 30*50 s. In addition, we used three different policies to see how the proposed framework works in the experiment. The first policy, no migration, only executes applications on the private data center. This policy represents the traditional computing model that only uses a private data center for computation. The second policy, migration based on the number of applications, only moves applications to the public cloud while the private data center is overloading. The number of applications to run on the public cloud is set to 50% of the number of total applications. The third policy, migration based on the execution time of applications, moves applications to the public cloud while the private data center is overloading as well. The proposed framework tries to re-arrange 50% of the total remaining expected execution time of applications in the public cloud. Figure 4 demonstrates the result of the experiment. It shows that the proposed framework with intelligent application migration can effectively reduce the total execution time of the applications. This can be attributed to that the proposed framework automatically creates as many virtual machines in the public cloud as the number of machines in the private data center, and then moves applications to the public cloud. The result also shows that using execution time for decision-making is more effective. Note that the proposed framework using the third policy creates the same amount of virtual machines in the public cloud as that using the second policy. The third policy is more efficient because the applications in the private data center and the applications in the public cloud almost finish at the same time.

302

W.-J. Wang et al.

Fig. 4 The result of executing different number of applications using different policies

5 Conclusions and Future Work In this paper, we presented a novel computing framework for hybrid clouds based on multi-agent technology. The framework is able to create virtual machine instances and to move applications to the public cloud automatically in the case that the workload is too high in the private data center. We implemented the proposed framework in a prototype of a hybrid cloud environment, and conducted an experiment using three different migration policies to evaluate the proposed framework. The preliminary result demonstrated that the framework is able to handle a short-term workload peak in the private data center by moving the workload to the public cloud. In the future, we plan to improve the automatic migration policy and the self-provisioning mechanism by considering different public cloud pricing models and the issues of quality of service. We also plan to build a larger hybrid cloud environment to evaluate the proposed framework. Acknowledgments This work was partially supported by the Nation Science Council of Republic of China under Grant No. 100-2218-E-008-005- and by Industrial Technology Research Institute of Republic of China under Grant No. B0-A0007.

References 1. Pallis G (2010) Cloud computing: the new frontier of internet computing. IEEE Internet Comput 14(5):70–73 2. Armbrust M, Fox A, Griffith R, Joseph AD, Katz RH, Konwinski A, Lee G, Patterson DA, Rabkin A, Stoica I, Zaharia M (2009) Above the cloud: a Berkeley view of cloud computing. Electrical Engineering and Computer Sciences, University of California at Berkeley, 10 Feb 2009

Intelligent Application Migration

303

3. Forter I, Zhao Y, Raicu I, Lu S (2008) Cloud computing and grid computing 360-degree compared. Grid computing environment workshop 4. Mell P, Grance T (2009) The NIST definition of cloud computing, version 15, NIST. Retrieved Jan 2010, from http://csrc.nist.gov/groups/SNS/cloud-computing/ 5. Sotomayor B, Montero RS, Llorente IM, Foster I (2009) Virtual infrastructure management in private and hybrid clouds. IEEE Internet Comput 13(5):14–22 6. Van den Bossche R, Vanmechelen K, Broeckhove J (2010) Cost-optimal scheduling in hybrid IaaS clouds for deadline constrained workloads. In: 2010 IEEE 3rd international conference on cloud computing (CLOUD), pp 228–235 7. Zhang H, Jiang G, Yoshihira K, Chen H, Saxena A (2009) Intelligent workload factoring for a hybrid cloud computing model. In: Proceedings of the 2009 congress on services—I, pp 701–708 8. Amazon Virtual Private Cloud (VPC), retrieved from http://aws.amazon.com/vpc/ 9. Nick JM, Cohen D, Kaliski BS Jr (2010) Key enabling technologies for virtual private clouds. In: Furht B, Escalante A (eds) Handbook of cloud computing. Springer Science+Business Media, LLC, New York, pp 47–63 10. Wood T, Shenoy P, Gerber A, Ramakrishnan KK, Van der Merwe J (2009) The case for enterprise-ready virtual private clouds. In: Proceedings of HotCloud’09 workshop on hot topics in cloud computing, San Diego, CA, USA, June 2009 11. Cloud Computing Center for Mobile Applications, retrieved from http://www.itri.org.tw/chi/ ccma/ 12. Foster I, Jennings NR, Kesselman C (2004) Brain meets brawn: why grid and agents need each other, AAMAS’04, 19–23 July 2004, New York, NY, USA 13. Lange DB, Oshima M (1999) Seven good reasons for mobile agents. Commun ACM 42:88–89 14. Spanoudakis N, Moraitis P (2006) The Gaia2JADE process for multi-agent systems development. Appl Artif Intel J 20(2):251–273 15. Amoretti M, Laghi MC, Tassoni F, Zanichelli F (2010) Service migration within the cloud: code mobility in SP2A. In: 2010 international conference on high performance computing and simulation, pp 196–202, June 2010 16. Java Agent Development Framework (JADE), retrieved from http://jade.tilab.com/ 17. Olfati-Saber R (2006) Flocking for multi-agent dynamic systems: algorithms and theory. IEEE Trans Auto Cont 51(3):401–420 Mar 18. Jung JJ (2009) Consensus-based evaluation framework for cooperative information retrieval systems. Knowl Inf Syst 18(2):199–211 19. Varela CA, Agha G (2001) Programming dynamically reconfigurable open systems with SALSA (OOPSLA 2001 ACM Conference on Object-Oriented Systems, Languages and Applications). ACM SIGPLAN Not 36:20–34 20. Kaisler S, Money WH (2011) Service migration in a cloud architecture. In: 44th Hawaii international conference on system sciences (HICSS), pp 1–10 21. Hao W, Yen I-L, Thuraisingham B (2009) Dynamic service and data migration in the clouds. In:33rd annual IEEE international computer software and applications conference, pp 134–139

Multi-Band OFDM and OFDM Simulation Software Using MATLABÒ Graphical User Interface Roslina Mohamad, Wan Mohd Hafeez, Wan Mohd Salleh and Nuzli Mohamad Anas

Abstract This paper presents the conventional and multiband Orthogonal Frequency Division Multiplexing (OFDM) simulation software named as Signal Modulation Simulator (SiMiSIM). A simulation system with Graphical User Interface (GUI) is built to interface user friendlier than contemporary interface or command line interface. Three basic frequency modulation techniques are given as options which are Binary Phase Shift Keying (BPSK), Quadrature Phase-Shift Keying (QPSK) and Quadrature Amplitude Modulation (QAM). Each of the signals has to face Forward Error Correction (FEC) encoder to enhance the signal robustness at the receiver end. The simulation is able to simulate Bit Error Rate (BER) and Power Spectral Density (PSD) for research purpose as well as educational purpose.





Keywords Multiband OFDM Additive White Gaussian Noise Bit Error Rate Forward Error Correction



1 Introduction In early parallel transmission systems, a few non-overlapping sub-channels share the whole frequency band as shown in Fig. 1. Apparently the existence of guard band between two adjacent sub-channels is to provide non-overlapping subR. Mohamad (&)  W. M. Hafeez (&)  W. M. Salleh  N. M. Anas (&) Faculty of Electrical Engineering, Universiti Teknologi MARA Malaysia, 40450 Shah Alam, Selangor, Malaysia e-mail: [email protected] W. M. Hafeez e-mail: [email protected] N. M. Anas e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_29,  Springer Science+Business Media B.V. 2012

305

306

R. Mohamad et al.

Fig. 1 Conventional nonoverlapping multi-carrier modulation [2]

Fig. 2 Overlapping multicarrier modulation [2]

channels as to eliminate the possible interference among adjacent sub-channels, which is known as Inter-Carrier Interference (ICI). This guard band constitutes a waste of spectrum. However in mid-1960s, spectral efficiency was improved by overlapping the sub-channels as shown in Fig. 2 had saved up to 50% of the spectrum used which was developed using OFDM technology. OFDM is not only a frequency multiplexing technique that mandates orthogonally among subchannels but also a special case for multi-carrier modulation. Moreover, OFDM can be regarded as either multiplexing technique or modulation scheme [1]. As compared in Figs. 1 and 2 respectively, apparently that bandwidth can be saved and be used for other sub-channels. In addition, another advantage of OFDM is multiple orthogonal carriers are transmitted simultaneously. By transmitting several symbols in parallel, symbol duration is increased proportionately, which reduces the effects of Inter Symbol Interference (ISI) caused by a dispersive Rayleigh-fading environment [2]. This paper objective is to describe the simulation functioning software called Signal Modulation Simulator, in short SiMiSIM, which developed using MATLAB Graphical User Interface Development Environment (GUIDE) to provide better easy access. SiMiSIM cater not only for OFDM transmission but also simulate the Multiband OFDM system. Various results concerning to the system performances are calculated such as the BER and other relevant information such as PSD and the signals constellation. Hence, it allows user to display multiple graph concurrently for easier comparisons among available modulation options which include PSK and QAM. The rest of this paper is organized as follow: A literature review of the project is explained in Sect. 2, emphasizes on both conventional and multiband OFDM system together with the introduction of SiMiSIM. Then, in Sect. 3, described system model and explained each module involved in baseband signal processing. Thorough explanation includes the data randomizer, random interleaver, channel coding and signal modulation used in this simulation. Section 4 discussed on methodology towards this simulation tool called SiMiSIM and briefed on the simulation flows. Section 5 discussed the result obtained from several examples and continued in the next section with the conclusion and future work recommendation on this project.

Multi-Band OFDM and OFDM Simulation Software

307

2 Literature Review The motivation towards this simulation tool development, so-called SiMiSIM, is to ease the calculation of complex equation involved in bit error rate (BER) alongside the power spectral density (PSD) using GUI manner. Despite of conventional command line interface (CLI) approach, users are able to display various system performance results concurrently.

2.1 Conventional OFDM Orthogonal Frequency Division Multiplexing (OFDM) is a well-known technique to mitigate inter-symbol interference (ISI) due to multipath effects. For highly dispersive channels, OFDM is more efficient at capturing multipath energy provides higher spectral efficiency and inherent resilience to narrowband radio frequency (RF) interference. It also has excellent robustness in multipath environment while preserving the orthogonally among subcarriers. The concept is to divide a bandwidth into several parallel streams, one for each sub-carrier. Each sub-carrier is modulated with a conventional modulation scheme at a lower rate.

2.2 Multiband OFDM The principle idea of multiband approaches is to divide the Ultra-Wideband (UWB) frequency band from 3.1 to 10.6 GHz into multiple smaller frequency bands or sub bands and uses multiple carrier frequencies to transmit the information. Each sub band has a bandwidth greater than 500 MHz to comply with the Federal Communication Commission (FCC) definition of UWB signal [3]. Multiband OFDM spectrum in UWB channel is shown in Fig. 3. Apparently it had been divided to S sub band and each sub band may have N subcarriers each.

2.3 Signal Modulation Simulator Signal Modulation Simulator (SiMiSIM) is a tool to simulate the performance of variants coding and modulation techniques in the context of OFDM transmission, both conventional and multiband OFDM system. SiMiSIM gives easy access and attractive user interfacing built using Graphical User Interface Development Environments (GUIDE) tool in MATLAB, where users are able to program the software themselves provided in m-file script format. Furthermore, ones can

308

R. Mohamad et al.

Fig. 3 Overlapping multicarrier modulation [2]

visualize the system performance in terms of BER, PSD and also the signal constellation graphically at both ends.

3 System Model This section discussed the transceiver module used in constructing SiMiSIM which includes the data randomizer, Forward Error Correction (FEC), bit-interleaving, signal modulation and also OFDM transmission. Each module is thoroughly explained in short while. Figure 4 depicted the system model where each module is representing in block diagram concatenate. Noted that, Additive White Gaussian Noise (AWGN) and Saleh–Valenzuela (S–V) has been used representing the channel modeling. Signal source enters the randomizer to have more evenly distributed power density and also to avoid too high peak power that may lead to distortion. Also in this process, it breaks up any adjoining long strings of zeros or ones and brings movement to the data stream [4]. The data are scrambled to convert the data bit sequence into a pseudo random sequence which is free from long repeated strings. Polynomial generator of pseudo random binary sequence (PRBS) used in data randomizer as shown in (1) [3]. g ¼ 1 þ D14 þ D15

ð1Þ

In (1), D represents a single bit delay. xn in PRBS is generated as in (2) and the scrambled data bit stream is then obtained using (3) [3] as derived below. xn ¼ xn14  xn15

ð2Þ

s n ¼ bn  x n

ð3Þ

In information theory, FEC used as an error control system of data transmission, whereby systematically generated redundant data are added to its original messages. It is to increase the robustness of the signal due to the compression process and also makes the signal vulnerable to channel noise and interference [4, 5]. In

Multi-Band OFDM and OFDM Simulation Software

309

Fig. 4 Transceiver architecture of OFDM

SiMiSIM, Convolutional encoder is used concatenated with Reed Solomon (RS) code to improve the BER and for more accurate decoding at the receiver [3, 4]. Reed Solomon (RS) code is non-binary cyclic codes with symbols made up of multiple bit sequences [6]. The codes are very useful for burst error correction where errors occur in a large sequence. RS code with symbols made up of m bit sequences where m corresponds to elements of the Galois Field, GF(qm) and is any positive integer greater than 2 [4]. The parameters of RS codes are as the following [4]: • Code length n ¼ 2m  1

ð4Þ

• Number of parity check symbols n  k ¼ 2t

ð5Þ

K ¼ 2m  1

ð6Þ

Dmin ¼ n  k þ 1

ð7Þ

• Number of data symbols

• Minimum distance

Random interleaving is used to rearrange data sequence using a fixed random permutation order. It enhances the error correcting capability of coding by constructing a long block code from small memory convolutional codes. By doing so, errors typically occurs in bursts rather than uniformly distributed in long codes can approach the Shannon capacity limit. In multi-carrier transmission, cyclic prefix of an exact copy of a segment of the OFDM symbol is located toward the symbol end [1] as to mitigate the ISI effects. Despite of redundancy it can effectively be avoided at the cost of power loss and bandwidth expansion but this inserting of cyclic prefix guard interval before each block of parallel data symbols will decreases the spectral efficiency of the OFDM system [7, 8].

310

R. Mohamad et al.

Fig. 5 SiMiSIM operations

Fig. 6 SiMiSIM after the simulation is completed

4 Methodology Initially, the baseband signal processing described in previous section is designed using MATLAB m-file prior converting to GUI based software. Figure 5 depicted the architecture and software flows. It involves displaying the transmitting power spectrum either both in time and frequency domain along with the

Multi-Band OFDM and OFDM Simulation Software

311

Fig. 7 PSD in time domain

Fig. 8 PSD in frequency domain

signals constellation. Towards the end, the error rate analysis is visualized using semi-log plotting format in range of given signal-to-noise power ratio. In this simulation, either AWGN only or S–V channel modeling may be chosen as to simulate the realistic data transmission. Note that conventional or multiband OFDM transmission also can be simulated with various coding and modulation schemes. Options include both binary and quaternary PSK or multi-amplitude QAM of 16 and 64-level for the signal modulations while code rate of 1/2, 3/4 and 5/6 can be chosen for the convolutional encoder. Users also able to determine the length of cycle prefix and number of symbols involved. Ones would be able to compare various results obtained in the simulation with theoretical performance. BER plots of each type of modulation as well as their signal constellations can be easily visualized for academic and research purposes.

312

R. Mohamad et al.

Fig. 9 16-QAM bit error rate

Fig. 10 16-QAM scatter plot

Those visualizing figures are displayed side by side simultaneously in the Graph Preview section which includes the power spectrum of respective signal in both frequency and time domains. Figure 6 shows a snapshot of the SiMiSIM after a simulation is completed.

5 Results and Discussion Performance of 16-QAM with 3/4 Convolutional Encoder is used to discuss in this section. Guard period of rate 1/32 with 100 OFDM symbols is used throughout discussion. Hence in Figs. 7 and 8, the power spectrum density in both time and

Multi-Band OFDM and OFDM Simulation Software

313

frequency domain is shown respectively. A multi-carrier modulation with different power is transmitted within the OFDM signal. While in Fig. 9, shown the BER calculated in SiMiSIM using semiology representation which follows the waterfall-like behavior for Shannon theory. Scatter plot of the 16-QAM transmission with AWGN corrupted signal is shown in Fig. 10. As a 16-QAM modulation is used, thus 4 constellations exist at each of quartet of the graph and making the total of the constellations is 16.

6 Conclusion SiMiSIM is a tool to simulate baseband signal processing involved in conventional and multiband OFDM systems in a graphical user interface manner. This tool able to simulates the error performance and other visualize figures relate to the signal processing which includes the signal constellation and the power spectrum. Easy access and attractive interface provided makes users able adapt and learn in short time. Furthermore, results obtained from simulations can be compared to theoretical or any other previous work for easy understanding. We would like to recommend of the future works of implementing the conventional and multiband OFDM system on a DSP hardware platform in order to gauge real-time system performances. Acknowledgments The authors would like to thank Ministry of Higher Education, Malaysia (MOHE) for supporting this research through grant number 600-RMI/ST/FRGS 5/3/Fst (120/2010) and 600-RMI/ST/FRGS 5/3/Fst(164/2010).

References 1. Chiueh T-D, Tsai P-Y (2007) OFDM baseband receiver design for wireless communications. Wiley, Singapore 2. Ghavami M et al (2007) Ultra wideband signals and systems in communication engineering, 2nd edn. Wiley, West Sussex 3. Siriwongpairat WP, Liu KJR (2008) Ultra-wideband communications systems multiband OFDM approach. Wiley, New Jersey 4. Vengadasalam JAL (2007) Performance analysis of digital television systems. Master of Engineering (Telecommunication), Faculty of Engineering, University Malaya 5. Mohamad R, Anas NM (2010) Performance analysis of convolutional interleaver on TMS320C6711 digital signal processing kit. In: Proceedings of the 2010 international conference on computer applications and industrial electronics, 5–8 December 2010 6. Sklar B (2001) Digital communications fundamental and applications, 2nd edn. Prentice-Hall, Upper Saddle River 7. Kattoush AH et al (2009) The performance of multiwavelets based OFDM systems under different channel conditions. Dig Sig Proc 20:472–482 8. Batra A et al (2004) Multi-band OFDM: a new approach for UWB. Presented at the 2004 IEEE international symposium on circuits and systems, Sheraton Vancouver Wall Centre Hotel, Vancouver, Canada

A Comparison on PSO Variants in Calculating HEPWM Switching Angles for a Cascaded H-Bridge Multilevel Inverter Norkharziana Mohd Nayan, Naziha Ahmad Azli and Shahrin Md. Ayob

Abstract This paper presents a comparison study on the application of various PSO algorithms in calculating the Pulse Width Modulation (PWM) switching angles of a cascaded H-bridge multilevel inverter. It aims to investigate the behavior of the PSO algorithm in three different variants. The results are compared to determine the most suitable algorithm for optimizing the multilevel inverter output voltage through Harmonic Elimination PWM (HEPWM) switching technique.





Keywords Particle Swarm Optimization Harmonic Elimination PWM Inverter

1 Introduction Multilevel inverter plays an important role in medium to high power conversion application. The applications include active power filters, static var compensator, unified power flow controller (UPFC), electric vehicles, and industrial motor drives [1–4]. Harmonic Elimination Pulse Width Modulation (HEPWM) offers N. M. Nayan (&) School of Electrical System Engineering, Universiti Malaysia Perlis, 02000 Kuala Perlis, Perlis, Malaysia e-mail: [email protected] N. A. Azli  S. Md. Ayob Department of Energy Conversion (ENCON), Faculty of Electrical Engineering, Universiti Teknologi Malaysia, 81310 UTM Skudai, Johor, Malaysia e-mail: [email protected] S. Md. Ayob e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_30,  Springer Science+Business Media B.V. 2012

315

316

N. M. Nayan et al.

elimination of dominant lower order harmonics and filters higher residual frequencies in a multilevel inverter output voltage. In general, this method produces a good harmonic performance for the inverter output. The elimination of the unwanted harmonics is obtained via the substitution of the nonlinear transcendental equations with a set of systems of linear equations. Recent research on power converter control using PSO shows that the optimization algorithm can be successfully used to control a multilevel inverter [5, 6]. Moreover, PSO algorithm is easy to implement and has been empirically shown to perform well on many optimization problems. Involvement of the PSO algorithm has improved the overall system performance, especially on the aspect of speed processing and the quality of the output power. Multilevel inverters with HEPWM switching controls are known to inherent characteristics such as nonlinearity, unavailability of a precise model and excessive complexity that make them suited for PSO control. Thus, it is expected to be able to reduce the computational burden associated with the solutions of the non-linear transcendental equations relevant to the harmonic elimination problem. The known problem of HEPWM switching technique for inverters in general is computing the switching angles as it involves the solving of transcendental equations. The problem gets worse when the technique is applied to a multilevel inverter topology. In order to solve this problem, a solution using PSO method is proposed in this work. The advantage of this method is its capability in solving the non-linear equations associated with HEPWM with fast generation of the switching angles due to faster convergence when searching for the solutions. The calculated switching angles are also accurate to any desired value of modulation index, which makes the angle resolution higher. The produced output voltage is expected to be of high-quality with precision in the PWM switching angles while the unwanted harmonics are eliminated. Furthermore, via PSO method, the memory capacity can be reduced due to its online computational capability, which makes the system more flexible and interactive. In addition, using this method the elimination of selected harmonics could minimize the THD and motor losses [7]. Thus, the filtering process can be omitted from the system. Since the inception of PSO in 1995 [8], several researchers have modified the algorithm in order to make it adaptable to their research application and perform faster than the original version. Common variants of the PSO algorithm are standard PSO, Inertia Weight PSO, Constriction Factor PSO and also Mutative PSO [9, 10]. This work aims to investigate the behavior of the PSO algorithm in three different variants. The algorithms are used to compute the multilevel inverter switching angles. The results are compared to determine an optimized algorithm for optimizing switching control performance of multilevel inverter implementation. It is expected that a suitable PSO algorithm that will produce a high resolution of optimized PWM switching angles and low THD voltage for a multilevel inverter can be achieved.

A Comparison on PSO Variants

317

1

0 ααα 1 2 3

360

αN

ωt

180

Fig. 1 Harmonic elimination PWM waveform

2 Harmonic Elimination PWM A typical output voltage waveform of a HEPWM is shown in Fig. 1 contains harmonics and the output function f(t) can be expressed in Fourier’s series as: 1 X ðan sin nan þ bn cos nan Þ ð1Þ f ðtÞ ¼ n¼1

Due to the quarter wave symmetry of the output voltage, the even harmonics are absent and only odd harmonic are present [11]. The amplitude of the nth harmonic an is expressed only with the first quadrant switching angles a1, a2,…, aN #  " N X 4 k ð1Þ cos nak Þ ð2Þ 1þ2 an ¼ np k¼1 and the solution must satisfy: 0\a1 \a2 \    \aN \

p 2

ð3Þ

For any odd harmonics, Equation (2) can be expanded up to the kth term. Where N is the number of variables corresponding to switching angles a1 through aN of the first quadrant. In order to eliminate the selected harmonic, an is assigned the desired value for fundamental component and equated to zero for the harmonics to be eliminated. Nonlinear transcendental equations are formed. a1 through aN are calculated by solving this equation. It is evident that (N-1) harmonic can be eliminated with N number of switching angles. #  " N X 4 k ð1Þ cos ak Þ ¼ M 1þ2 a1 ¼ np k¼1 #  " N X 4 k ð4Þ 1þ2 ð1Þ cos 5ak Þ ¼ 0 a5 ¼ 5p k¼1 #  " N X 4 k ð1Þ cos nak Þ ¼ 0 1þ2 an ¼ np k¼1

318

N. M. Nayan et al.

where M is the amplitude of the fundamental component. The main issue with regard to the HEPWM method is computation the switching angles. Several methods for computing optimal PWM switching angles includes Newton–Raphson, Walsh function, Resultant Theory, Genetic Algorithm (GA), curve fitting techniques and PSO [4, 9, 16–18]. Newton–Raphson method is widely used in this area, but it is not appropriate for a large number of switching angles if good initial guesses are not available and may end in local optima. A second approach is Walsh’s functions commonly used to solve a linear equation. An alternative technique based on GA is very effective in solving optimal PWM switching angle. Unfortunately this method is complicated and has a mutation probability. The Resultant Theory’s method has been used to solve non-linear transcendental harmonic elimination equations by converting them into polynomial equations. However, this method is unattractive and has a limitation when the number of inverter level increases as the degrees of the polynomial will also increase that may lead to numerical difficulty and substantial computational burden. Curve fitting techniques present a solution for the optimal PWM switching angle’s trajectories with polynomial equations. However, the problem with curve fitting techniques arises when a large look-up table needs to be stored on the control system memory. In addition the sampling interval for the PWM switching angle is low when digitally implemented. Neural network control, fuzzy logic and PSO are parts of the latest trends on computing optimal PWM switching angles. Neural network and fuzzy logic control are quite complicated since the parameters need repetitive training before being implemented on the system and this may become a time-consuming task. PSO has similar characteristics with GA, but it is easy to implement as it does not have genetic operators such as cross over and mutation which may drag to longer processing time.

3 PSO: The Theory and its Variants PSO is relatively another type of an optimization algorithm originated from Kennedy and Eberhart [12]. Theory of this stochastic optimization procedure is based on the movement and intelligence of swarms, which has the ability to solve optimization problems with social interactions. Research conducted by Al-Othman [4] on the harmonic elimination problem of a cascaded H-bridge inverter with nonequal DC sources proves that PSO have reduced the computational burden associated with the solution of the non-linear transcendental equations. The main reason for choosing PSO in this work is that PSO is very simple in concept, easy to implement and computationally efficient. In power converter areas, PSO mainly involves inverter control, AC/AC choppers, photovoltaic controller and motor drives systems [9, 11, 16, 21]. It is proven that PSO algorithm is very efficient relative to other evolutionary computation techniques in order to solve optimization problems.

A Comparison on PSO Variants

319

3.1 PSO: The Theory PSO is inspired by social behaviour of birds flocking or fish schooling. PSO conducts the searching process using a population of particles. Each particle is a potential solution to the problem under investigation. Each particle in a given population adjusts its position by flying in multi-dimensional search space until an unchanging position of the fittest particle is encountered. The concept; at each time step changing the velocity and position of each particle toward its pbest and gbest is according to Equations (5) and (6) vid ðt þ 1Þ ¼ ½w  vid ðtÞ þ ½c1  r1  ðpid ðtÞ  xid ðtÞÞ þ ½c2  r2  ðpid ðtÞ  xid ðtÞÞ

ð5Þ

xid ðt þ 1Þ ¼ xid ðtÞ þ vid ðt þ 1Þ

ð6Þ

For a target problem which has n-dimensions; xi = (xi1, xi2, …, xin)T; position vector of the ith particle Vi = (vi1, vi2, …, vin)T; velocity vector of the ith particle Pi = (pi1, pi2, …, pin)T; best position of each particle/pbest Pg = index of best particle among all particle in the population/gbest w = inertia weight c1 and c2 = acceleration constant r1 and r2 = random number in a range of 01 Pi is the best fitness so far that particles have achieved. The first part of Equation (5) is the inertia weight part whereas w represents the degree of the momentum of the particles. The second part is the cognition part, which represents the independent behaviour of the particle itself. The rest parts of the equation represent the social part, which represents the collaboration among the particles. c1 and c2 represent the weighting of the cognition and social parts that pull each particle towards pbest and gbest positions. Variable xi is the solution found by each particle. The goodness of the solution is evaluated based on its fitness value.

3.2 The PSO Variants Since its inception in 1995, several researchers have modified the algorithm in order to make it adaptable to their research application and perform faster than the original version. The common variants of the PSO algorithm are as follows: i) The original PSO or standard PSO—An explorative PSO algorithm. Unfortunately, it may suffer from late convergence as it goes closer to the maximum iterations. The equation for the standard PSO is given by Equation (7);

320

N. M. Nayan et al.

Vid ¼ Vid þ C1 ðpBest  Xid Þ  rand1 þ C2 ðgBest  Xid Þ  rand2

ð7Þ

ii) Inertia weight PSO—Good for focus but lacks in solution quality. The equations for the standard PSO are given by Equations (5) and (6). The inertia weight is usually decreased linearly from 0.9 to 0.4 to improve convergence. iii) Constriction factor PSO—best for focus and solution quality, better performance in continuous value. Unfortunately lacks in discrete and binary problems. Vid ¼ xðVid þ C1 ðpBest  Xid Þ  rand1 þ C2 ðgBest  Xid Þ  rand2 Þ

2 v ¼  pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; 2  u  u2  4u

u ¼ c1 þ c2; u [ 4

ð8Þ

ð9Þ

iv) Mutative PSO—Rapid convergence but may be premature. This algorithm kills off non-performing particles and replaces them with mutated version of gbest.

4 Simulation The simulation is done by implementing a PSO algorithm on calculating the PWM switching angles of the power switches in a multilevel inverter to produce the its output voltage. Through PSO algorithm, optimum PWM switching angles of the multilevel inverter can be determined. With that, a high processing speed and resolution of the generated switching angles can be established. A single-phase cascaded H-bridge multilevel inverter PSO algorithm is applied to this system. The PSO control algorithms are designed and simulated using MATLAB/Simulink. In addition, the training of the PSO control algorithm on a representative set of input and targets pairs have been done using the same simulation software.

4.1 The Software Development The development and implementation stages of the PSO algorithm on the system are shown in Fig. 2. It begins with the determination of the number of angles per quarter cycle, N for the HEPWM equations. With that, Equation (4) is used to calculate the switching angles by adapting the equation into an m-file programming command with PSO algorithm. Subsequently, this program is trained to determine the correct weighting parameter solutions for the HEPWM switching angles through a training simulation. Then, the algorithm is integrated with the multilevel inverter simulation system to verify the control parameters. After

A Comparison on PSO Variants

HEPWM

PSO

DSP

321

Gate Driver

Power Switch

Fig. 2 The system block diagram

Table 1 Comparison of the PSO variants Standard PSO Best fitness Last iteration Reference angle a1 = 15.2985 a2 = 20.7779 a3 = 30.8777 a4 = 41.5097 a5 = 47.0563 a6 = 62.2443 a7 = 64.3129 a8 = 89.8477

8.324 9 10-6 350 15 20.1 30 38 57 63 64 90

Inertia weight PSO

Constriction factor PSO

1.1102 9 10-16 302 Calculated angle 15 21 30 42 48 57 64 87.6362

0 126 15 21 30 38 49.2042 57 64 90

achieving the optimized PWM switching angles, another program in C is developed to generate the pulse widths for the HEPWM. In order to drive the power switch on the inverter’s side, the pulse generation algorithm is programmed on a DSP to produce the PWM pulses.

5 Results In order to select the suitable PSO algorithm for the PWM switching angles implementation, several simulations using various types of PSO algorithm as discussed in the previous section have been done. Computation of the PWM switching angles for eight angles per quarter cycle (N = 8 and) amplitude modulation, ap1 M = 1 is made using Equation (4). The PSO equation parameters are: Swarm size = 50, Dimension = 8, Maximum iteration = 1000. Table 1 presents the comparison made on the PSO algorithm variants based on the computed angle value, best fitness and number of maximum iteration with the switching angles obtained from Newton–Raphson’s method [13] used as the reference angle.

322

N. M. Nayan et al.

6 Discussions The simulation result shows that implementation of the standard PSO to calculate the PWM switching angles of the multilevel inverter suffers from late convergence and immature value. It can be seen that the calculated angles do not converge to stable values and they are also inaccurate. Meanwhile, from the Inertia Weight PSO best fitness, there is a significant difference on maximum iteration compared to the standard PSO. The Inertia Weight PSO converges faster than the Standard PSO. The computed angle values are much closer to the reference angles but not very accurate. Constriction Factor PSO has the fastest convergence performance based on its final iteration to achieve best fitness and have stable angle values. The calculated angles are closer to the reference angles compared to Inertia Weight PSO and standard PSO. Therefore, it is concluded that the Constriction Factor PSO is the most suitable algorithm for implementing in a multilevel inverter system. However, in order to achieve more optimal results further training should be done by changing the PSO parameters. Apart from that, the objective function of the PSO algorithm can also be changed according to the programmer’s creativity as it may give different way of iteration and best fitness value. Nevertheless, Standard PSO and Inertia weight PSO can as well be used to give an optimal result only if the processing time is not the work constraint.

7 Conclusions In this paper, three types of PSO algorithms namely standard PSO, Inertia Weight PSO and Constriction Factor PSO have been used to calculate the PWM switching angles of a multilevel inverter. A brief summary on HEPWM and PSO has been given. Based on the simulation result, PSO algorithm can be successfully used to calculate the HEPWM switching angles of a multilevel inverter. At the same time, the selected harmonics in the multilevel inverter output voltage are eliminated with low THD percentage. Implementation of the PSO based system on a multilevel inverter switching control has produced a high accuracy of switching angles generation with high quality output voltage waveform. From the simulation with three types of PSO algorithm, it is found that PSO implementation performances according to the selected application give a distinct characteristic. Implementation of Standard PSO suffers from late convergence and immature value. While with Inertia weight PSO the computed results are good for focus, but lacks in solution quality. Whereas, with Constriction PSO algorithm the computed angle values is best for focus and solution quality. Simulation results also show that Constriction Factor PSO is the potential method in producing an optimize pulse generation for the multilevel inverter. It is expected that with the implementation of the PSO algorithm, a high processing speed system with less complexity in the control part is produced.

A Comparison on PSO Variants

323

References 1. Tolbert LM, Member S, Chiasson JN, Peng FZ (2003) Modulation index regulation of a multilevel inverter for static var compensation East, pp. 1–6 2. Lai J, Member S, Peng FZ (1996) Multilevel converters—A new breed of power converters, vol 32(3), pp 509–517 3. Cunnyngham T (2001) Cascade multilevel inverters for large hybrid-electric vehicle applications with variant DC sources. University of Tennessee, Knoxville 4. Al-Othman AK, Abdelhamid TH (2009) Elimination of harmonics in multilevel inverters with non-equal dc sources using PSO. Energy Convers Manag 50(3):756–764 5. Ray RN, Chatterjee D, Goswami SK (2009) An application of PSO technique for harmonic elimination in a PWM inverter §. Appl Soft Comput 9:1315–1320 6. Kaviani AK, Fathi SH, Farokhnia N, Ardakani AJ (2009) PSO, an effective tool for harmonics elimination and optimization in multi-level inverters. In: 2009 4th IEEE conference on industrial electronics and applications, pp 2902–2907 7. Kouzou A, Saadi S, Mahmoudi MOO, Boucherit MSS (2009) Particle swarm optimization applied for the improvement of the PWM AC/AC choppers voltage. In: 2009 compatability and power electronics, pp 146–152 8. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95— International conference on neural networks, pp 1942–1948 9. Eberhart R, Shi Y (2000) Comparing inertia weights and constriction factors in particle swarm optimization. System, no. 7, pp 84–88 10. Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: 1998 IEEE international conference on evolutionary computation proceedings. IEEE world congress on computational intelligence (Cat. No.98TH8360), vol 49(5), pp 69–73, May 11. Holmes DG, Lipo TA (2003) Pulse width modulation for power converters—Principles and practice. Wiley, USA, p 724 12. Hagh MT, Taghizadeh H, Razi K (2009) Harmonic minimization in multilevel inverters using modified species-based particle swarm optimization, October, vol 24(10), pp 2259– 2267 13. Azli NA (2001) Online optimal pulsewidth modulation (PWM) multilevel control of a modular structured multilevel inverter (MSMI) for high power AC power supply applications, Universiti Teknologi Malaysia

A Hybrid Artificial Potential Field: Genetic Algorithm Approach to Mobile Robot Path Planning in Dynamic Environments Yanping Liu and Kamal K. Bharadwaj

Abstract In this paper, a hybrid Artificial Potential Field—Genetic Algorithm approach is developed and implemented for mobile robot path planning in dynamic environments. The hybrid approach first uses Grid Method where the mobile robot environment is represented by orderly numbered grids, each of which represents a location in the environment. Then, it applies Genetic Algorithm (GA), a global planner, to find an optimal path according to the current environment. The GA proposed here uses an evolutionary population initialization and genetic operators, which make the evolutionary process converge very efficiently. Finally, a new Artificial Potential Field method, a local planner, is applied to follow the path obtained by GA from one intermediate node to next intermediate node avoiding the obstacles. Experimental results clearly illustrate that the proposed hybrid approach works well on large scale dynamic environments. Keywords Artificial Potential Field Dynamic Path Planning



Genetic Algorithm



Mobile Robot



Y. Liu (&)  K. K. Bharadwaj School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi, 110067 India e-mail: [email protected] K. K. Bharadwaj e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_31,  Springer Science+Business Media B.V. 2012

325

326

Y. Liu and K. K. Bharadwaj

1 Introduction In the control of mobile robot, path planning is one of the key issues. The robot path planning problem can be typically described as follows: given a robot and a description of its working environment, plan a collision free path between two specified locations that satisfies certain optimization criteria [1]. Path planning research of mobile robot in static environment has reached mature stage, but the path planning in dynamic environment is a big challenge. In practice, it is often no complete knowledge about the environment. Having a detailed map with all the obstacles marked seems to be unrealistic for most of the situations. Whether the robot path planning belongs to which category, the path should meet the following conditions [2]: • The path should be collision-free; • The path should be as short as possible and search time should be as little as possible; • The path should be as smooth as possible. Many algorithms for path planning have been studied and developed over the past few years. The main methods of path planning for mobile robot can be divided into two categories—Artificial Potential Field (APF) approaches and Artificial Intelligence (AI) approaches. The main AI-based approaches for robot path planning are Genetic Algorithm (GA), Fuzzy Logic Control (FLC) and Artificial Neural Network (ANN) [3]. Recently, many researchers have applied the hybrid approaches, such as Fuzzy-Genetic approach, on path planning. This paper develops and implements a hybrid Artificial Potential Field— Genetic Algorithm approach to mobile robot path planning in dynamic environments. The approach first uses Grid method to represent the environment. Then, it applies Genetic Algorithm, a global planner, to find an approximate optimal path according to the current environment. At last, Artificial Potential Field method, a local planner, is applied to follow the path from one intermediate node to next intermediate node avoiding the obstacles.

2 Hybrid Approach Based on Artificial Potential Field and Genetic Algorithm 2.1 Overview The mobile robot environment is represented by orderly numbered grids, each of which represents a location in the environment. The boundary of obstacles is formed by their actual boundary plus minimum safety distance considering the size of the mobile robot, which makes it possible to treat the mobile robot a point in the

A Hybrid Artificial Potential Field

327

Fig. 1 The outline of hybrid approach

Goal GA (Global Planner)

Start

APF (Local planner& Obstacles Avoidance)

environment. In a large scale environment, suppose the environment is 10 9 10 meters with a robot size 10 9 10 cm. We can model the environment into 100 9 100 grids, and treat the robot as a point in the environment. Figure 1 shows the outline of the hybrid approach. In this hybrid approach, a Genetic Algorithm is proposed first. It uses a simple effective path representation that combines grids and coordinates representations. This approach makes it possible to have one number for each gene and to use integer numbers instead of real numbers in chromosomes. As we know, the time cost of Genetic Algorithm increases exponentially when the dimension of configuration space increases. So the Genetic Algorithm will not work properly on the environments with large scale dimension. The suggested path length should be constrained to be less than 20. After applying Genetic Algorithm, a global optimal or near-optimal path has been got according to the current environment. However, as the environment is dynamic, some obstacles are moving, we can not follow the fixed path obtained by GA simply. How to follow the path and avoid the obstacles? It is not efficient to apply GA again when the environment is changed. Therefore, a local planner is proposed. We need an effective local path planner that follows the path obtained by global planner and avoid the moving obstacles properly. Since the predefined path is obtained before the movement of robot. Artificial Potential Field, a local planner, is proposed to follow the path from one intermediate node to next intermediate node avoiding the obstacles.

2.2 Genetic Algorithm (GA) as Global Planner GA first searches the optimal path or near-optimal path based on the current environment. In this paper, GA uses a chromosome with variable length. As the dimension of the environment is very large, the chromosome size (the number of nodes included in the path) should not be as long as the environment size, otherwise GA will not work efficiently. Therefore, the proposed chromosome size is less than 20. The following will discuss the chromosome, fitness function and genetic operators in details. Representation and Initial Population. A chromosome represents a path as a sequence of nodes, where each node indicates a grid number representing a location in the environment. The first node is the start and the last node represents the

328

Y. Liu and K. K. Bharadwaj

Fig. 2 A sample chromosome

goal. A feasible path is a collision free path, i.e. no nodes fall on any obstacles. The length of a chromosome is variable and between 2 and maximum length 20. As an example, Fig. 2 shows a sample chromosome. The initial population can be generated randomly, however, by experiment, it is shown that there are many infeasible paths in random initial population, which makes the evolutionary process converge very slowly. In some complicated environment which contains many obstacles, it is usually not applicable to get a feasible path. In this paper, the generation of initial population should satisfy one condition—all of the individuals should be feasible path. By doing this, the initialization will be slower than random initialization, but the evolutionary process is improved a lot. The simulation result will show that. Evaluation. The design of fitness function is one of the key issues in GA evolution. In this paper, the fitness function not only considers the length of the path, but also considers the smoothness, since the smoother path will make the mobile robot move more easily. As mentioned previously, all the paths generated initially are feasible, hence we do not need consider the path security. The fitness function is composed of two sub-functions: a. Sub-function of Path Length Fit1 ¼

N X

di

ð1Þ

i¼1

Where N is the number of line segment of a path, di is the Euclidean distance of the two nodes forming the line segment. b. Sub-function of Smoothness Fit2 ¼

N 1 X

gðli ; liþ1 Þ

ð2Þ

i¼1

Wheregðli ; liþ1 Þdenotes the inclination of two adjacent line segments li and liþ1 : To make sure the value is non-negative, we can define gðli ; liþ1 Þ ¼ cosðli ; liþ1 Þ þ 1. The fitness function can be given as Fit ¼ w1  Fit1 þ w2  Fit2

ð3Þ

A Hybrid Artificial Potential Field

329

Where wi ði ¼ 1; 2Þ respectively stands for the weighted values of length, smoothness in the fitness function. According to the fitness function, we can see that the fitter individual will have smaller fitness value, which indicates the path is shorter and smoother. Reproduction and Genetic Operators. In this approach a Tournament Selection method is used as a selection strategy. N individuals are selected randomly and the fitter individual will be selected as the first parent. N is the selection pressure according to the size of population. The same process is repeated to select the second parent. After the selection process, four operators are used to evolve the selected parents. Each operator is controlled by its probability. These operators are: a. Crossover This operator combines two selected paths (parents) to generate two offspring. Compared with traditional crossover operation, which randomly selects a mating intermediate node on each parent, the crossover operation proposed here is under one condition: the mating intermediate nodes selected from two parents should assure that two offspring are feasible. The mating intermediate nodes are randomly selected several times. If these two nodes can not be found, the crossover operation for these two parents is skipped. b. Mutation There are three types of mutation—Replacement, Addition and Deletion. The position is randomly selected for Replacement, Addition and Deletion several times. It should make sure that after Replacement, Addition and Deletion, the new path is feasible. Otherwise, this mutation operation will be skipped.

2.3 Artificial Potential Field (APF) as Local Planner In this paper, a new potential field method is proposed for path planning of a mobile robot in a dynamic environment where the goal and obstacles are moving. The attractive potential is defined as a function of the relative position and velocity of the goal with respect to the robot. The repulsive potential is also defined as the relative position and velocity of the robot with respect to the obstacles. Accordingly, the virtual force is defined as the negative gradient of the potential in terms of both position and velocity rather than position only. Attractive Potential Function. The attractive potential field function is presented as follows [4]  m  n Uatt ðp; vÞ ¼ ap pg ðtÞ  pr ðtÞ þav vg ðtÞ  vr ðtÞ

ð4Þ

Where pr ðtÞ and pg ðtÞ denote the positions of the robot and the goal at time t, respectively; pr ¼ ½x y zT in a three-dimensional space or pr ¼ ½x yT in a twothe velocities of the robot and the goal at dimensional space; vrðtÞ and vg ðtÞ denote  time t, respectively; pg ðtÞ  pr ðtÞ is the Euclidean distance between the robot

330

Y. Liu and K. K. Bharadwaj

  and the goal at time t; vg ðtÞ  vr ðtÞ is the magnitude of the relative velocity between the goal and the robot at time t; ap and av are scalar positive parameters; m and n are positive constants which satisfy m; n [ 1. The new attractive potential Uatt ðp; vÞ is a function of both the position p and velocity v of the robot. Therefore, we shall define the corresponding virtual attractive force as the negative gradient of the attractive potential in terms of both position and velocity. When pr 6¼ pg and vr 6¼ vg , we will have Fatt ðp; vÞ ¼ Fatt1 ðpÞ þ Fatt2 ðvÞ

ð5Þ

 m1 Fatt1 ðpÞ¼map pg ðtÞ  pr ðtÞ nP RG

ð6Þ

 n1 Fatt2 ðvÞ¼nav vg ðtÞ  vr ðtÞ nV RG

ð7Þ

Where

With nP RG being the unit vector pointing from the robot to the goal and nV RG being the unit vector denoting the relative velocity direction of the goal with respect to the robot. The attractive force Fatt consists of two components: the first component, Fatt1 ðpÞ; pulls the robot to the goal and shortens the distance between them, the second component, Fatt2 ðvÞ; drives the robot to move at the same velocity of the goal. Repulsive Potential Function. Similarly, we can get the repulsive potential Urep ðp; vÞ as a function of both the position p and velocity v of the robot with respect to the obstacles. The repulsive force function used here is referred in [4]. After the calculation of the attractive and repulsive forces, the total virtual force can be obtained by Ftotal ¼ Fatt þ Frep

ð8Þ

For the case where there are multiple obstacles, the repulsive force is given by Frep ¼

no X

Frep i

ð9Þ

i¼1

Where no is the number of obstacles and Frep i is the repulsive force generated by the ith obstacle. The total virtual force Ftotal will be used for local path planning. When employing the potential functions for dynamic path planning, local minimum problems do exist and should be taken care of. To solve the problem, the simplest method is to keep the robot move according to the total virtual force as usual and wait for the obstacles to change their positions. Since the environment is highly dynamic where some obstacles are moving, the situations where the configuration of the obstacles keeps static are rare. Therefore, the waiting method is often adopted.

A Hybrid Artificial Potential Field

331

Fig. 3 One typical run of path planning in the simulation environment: (a) The best initial path (fitness: 86.73); (b) The best solution in generation 56 (fitness: 71.81); (c) The best solution in generation 62 (fitness 50.28); (d) The near-optimal path in generation 132 (fitness: 46.04)

3 Simulation In this simulation, the environment is of size 50 9 33 grids. The start is grid number 64 and the goal is grid number 1,304. The population size is 80, crossover rate is 0.8, mutation rate is 0.03 and the maximum generation is kept 200. In the fitness function, w1 ; w2 are chosen as 0.6 and 0.4, respectively. The proposed Genetic Algorithm can easily deal with the obstacles and obtain a near-optimal path. Figure 3 shows the evolution process. The best solution in the initial population is shown in Fig. 3a, b shows the best solution after 56 generations. The genetic algorithm continues to evolve better solution (Fig. 3c) until the maximum generation is reached. A near-optimal path is obtained after 132 generations as shown in Fig. 3d). The near-optimal path is obtained by GA after 132 generations. Then the APF approach is used to follow the optimal path and avoid the obstacles. As a local planning, every time the robot chooses one intermediate node as the start and its next intermediate node as the goal. It moves from one intermediate node to next intermediate node repeatedly till the robot reaches the global goal. The robot tries to reach the desired goal which attracts it. When it moves towards the obstacles, the repulsive force prevents it from colliding with the obstacles.

332

Y. Liu and K. K. Bharadwaj

4 Conclusion and Future Work The novelty of this work is the combination of a global planner and a local planner for robot path planning in dynamic environments with moving obstacles. GA is employed as global planner to obtain a global optimization based on the current environment while the APF is used as local planner that is good at obstacles avoidance while following the optimal path. The hybrid approach is able to quickly determine the optimal feasible path for robot in dynamic environments. In this work, we have developed and implemented a hybrid Artificial Potential Field and Genetic Algorithm approach to deal with path planning problem for mobile robots in dynamic environments. Compared with the genetic based approach, the hybrid approach provides a better performance in processing time, and that is indeed crucial for a robot to quickly respond to avoid collision with the dynamic obstacles. Though the hybrid Artificial Potential Field and Genetic Algorithm approach solves the issue of obtaining the optimal path without collision with obstacles in dynamic environments, some further extensions need to be considered as future work: a. The environment in this simulation is described in a two-dimensional surface. For modeling a more realistic environment, a three-dimensional modeling needs to be considered. b. The hybrid approach is applied on a single robot currently. Extension of our proposed schema to handle on multi-robot system is a more interesting future research work.

References 1. Prahlad V, Kay Chen T, Mingliang W (2000) Evolutionary artificial potential fields and their application in real time robot path planning. In: Proceeding of the IEEE congress on evolutionary computation, pp 256–263 2. Pu S, Yujie C (2010) Dynamic path planning for mobile robot based on genetic algorithm in unknown environment. In: 2010 Chinese control and decision conference, pp 4325–4329 3. Qixin C, Yanwen H, Jingliang Z (2006) An evolutionary artificial potential field algorithm for dynamic path planning of mobile robot. In: Proceeding of IEEE international conference on intelligent robots and systems, pp 3331–3336 4. Ge SS, Cui YJ (2000) New potential functions for mobile robot path planning. In: IEEE transactions on robotics and automation, pp 615–620 5. Jianping T, Simon XY (2003) Genetic algorithm based path planning for a mobile robot. In: Proceeding of IEEE international conference on robotics & automation. IEEE Press, Canberra, pp 1221–1226 6. Yong Z, Lin Z, Xiaohua Z (2008) Mobile Robot path planning base on the hybrid genetic algorithm in unknown environment. In: Eighth international conference on intelligent systems design and application, pp 661–665

A Hybrid Artificial Potential Field

333

7. Khatib O (1985) Real-time obstacle avoidance for manipulators and mobile robots. In: Proceeding of IEEE international conference on robotics and automation, pp 500–505 8. Koren Y, Borenstein J (1991) Potential Field methods and their inherent limitations for mobile robot navigation. In: Proceeding of IEEE international conference on robotics and automation, pp 1398–1404 9. Scott M (2004) An introduction to genetic algorithms. J Comput Sci Coll 20:115–123 10. Wang Y, Sillitoe P, Mulvaney J (2007) Mobile robot path planning in dynamic environments. In: IEEE international conference on robotics and automation, pp 71–76 11. Lbszlo S, Annamiria K (2003) Autonomous navigation in a known dynamic environment. In: The 12th IEEE international conference on fuzzy systems, vol 1, pp 266–271 12. Tom MM (1997) Machine learning. McGraw-Hill, 13. Xio J, Michalewicz Z, Zhang L, Trojanowski K (1997) Adaptive evolutionary planner/ navigator for mobile robots. In: IEEE transactions on evolutionary computation, vol 1, pp 18–28 14. Ahmed E, A.Abdullah H, Shawki A (2004) Genetic algorithm for dynamic path planning. In: CCECE 2004—CCGEI 2004, Niagara Falls 15. Yanrong H, Simon XY (2004) A knowledge based genetic algorithm for path planning of a mobile robot. In: Proceeding of the 2004 IEEE international conference on robotics & automation, pp 4350–4355

Energy-Constrained Scheduling for Weakly-Hard Real-Time Tasks on Multiprocessors Yeonhwa Kong and Hyeonjoong Cho

Abstract In this paper, we consider energy constrained scheduling for weakly hard real-time constrained tasks running on multiprocessors. Rather than simply saving as much energy as possible, we consider energy as hard constraint under which the system remains functional and delivers an acceptable performance at least during the prescribed mission time. We propose a class of Energy-constrained Multiprocessor Real-Time Scheduling algorithms for (m, k)-firm deadline constrained tasks (EMRTS-MK). We prove that EMRTS-MKs provide assurance in terms of quality of multimedia services under a certain condition. We also evaluate the proposed algorithms in several experiments, which quantitatively show that EMRTS-MKs achieve the aforementioned scheduling objectives. Keywords Real-time systems

 (m, k)-firm deadline  Multicore  DVS

1 Introduction Several known advantages of multiprocessors, e.g., low power consumption, low cost per performance, etc. have recently made them pervade into portable computing and communication devices. These technological trends increasingly offer better platforms in which users can experience a variety of services including

Y. Kong (&)  H. Cho Department of Computer and Information Science, Korea University, Seoul, South Korea e-mail: [email protected] H. Cho e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_32,  Springer Science+Business Media B.V. 2012

335

336

Y. Kong and H. Cho

multimedia streaming. One unique feature of multimedia services is that they allow occasional deadline misses to a certain acceptable level, differently from traditional hard real-time services that do not allow any deadline misses. Thus, to guarantee the quality required by multimedia services while utilizing their reduced strictness of timeliness requirement, sophisticated real-time scheduling is crucial particularly on multiprocessors. Moreover, portable devices are battery-powered, and therefore energy efficiency is another significant design factor to consider. Rather than simply saving as much energy as possible in portable devices, we consider the problem of ensuring that the system with finite energy budget remains functional and delivers an acceptable performance at least during the prescribed mission time, where energy turns to be a hard constraint. In this paper, we aim at designing an energy-constrained scheduling algorithm for weakly-hard real-time constrained tasks running on multiprocessors, which is the first research effort to the best of our knowledge. Weakly-hard real-time systems. Real-time systems are classified into hard and soft depending on the degree of deadline satisfaction. In contrast to hard realtime systems, which do not allow any deadline miss, tolerance to deadline misses of soft real-time systems has traditionally been expressed as a maximum allowable loss percentage. However, this probabilistic specification is not sufficiently clear to represent the quality of service of soft real-time systems. For example, a video stream can be represented to tolerate a 10% loss rate. When a video stream has 100 frames, a 10% loss rate can be obtained either by first the 10 frames’ deadline misses out of 100 frames or by one deadline miss per every 10 frames. Even if two systems accomplish the same loss rate, their qualities of service are different with respect to the number of deadline misses within a given time period. Thus Hamaoui et al. [1] proposed the (m, k)-firm deadline constraint, where at least m deadlines should be met in every consecutive k deadlines. A violation of the given (m, k)-firm deadline is called dynamic failure, and the number of dynamic failure occurrences is utilized as a metric to evaluate performance of the systems. Later, Bernat and Burns proposed a concept of weakly hard real-time constraint that can tolerate a specified degree of missed deadlines, which encompasses several alternative (m, k)-firm deadline models [2]. Real-time scheduling for multiprocessors. Real-time scheduling algorithms for multiprocessors can be divided into two approaches, partitioned and global. Partitioned scheduling algorithms restrict task migration between processors after allocating each task to a processor for running. Partitioned scheduling is known to be an NP-hard problem. On the other hand, global scheduling algorithms allow task migration, which has made several optimal algorithms in this category appear, e.g., P-Fair [3], EKG [4], LNREF [5]. Our EMRTS-MKs are designed based on PFair to inherit its timeliness guarantee under under-load situations. DVS for weakly hard real-time system. Several attempts for using the DVS technique to reduce energy consumption for (m, k)-firm deadline task scheduling have been proposed. To guarantee the (m, k)-firm deadline, Ramanathan et al. [6] proposed a strategy to statically determine two types of jobs: mandatory and optional job. The mandatory jobs are the jobs that must meet their deadline to

Energy-Constrained Scheduling

337

satisfy the (m, k)-firm deadline constraints, while the optional jobs can be executed to further improve the quality of service or simply skipped to save energy consumption. Later, Niu and Quan designed a static/dynamic algorithm for weakly hard real-time systems [7]. They statically determined the mandatory job but dynamically updated it during run-time. An energy constrained algorithm for weakly hard real-time system was proposed by Alenawy and Aydin [8]. They defined the energy-constrained scheduling problem for weakly–hard real-time tasks on a single processor. However, to the best of our knowledge, studies on energy efficiency of multiprocessor real-time scheduling algorithms for (m, k)-firm constrained tasks do not exist. Contribution. We consider energy constrained scheduling for weakly hard realtime constrained tasks running on multiprocessors. Rather than simply saving as much energy as possible, we consider energy as a hard constraint under which the system remains functional and delivers an acceptable performance at least during the prescribed mission time. Our scheduling objectives are to (1) meet a prescribed mission time as much as possible; (2) minimize the number of dynamic failure occurrences to satisfy the demand on quality of multimedia services and; (3) satisfy as many deadlines as possible to maximize the quality of multimedia services. To achieve these goals, we design a class of Energy-constrained Multiprocessor RealTime Scheduling for (m, k)-firm deadline constrained tasks (EMRTS-MK). In this paper, we show that EMRTS-MKs provide guaranteed performance and energy constrained scheduling for weakly hard real-time tasks on multiprocessors.

2 System Model 2.1 Task Model We consider a periodic real-time task set T = {T1,….TN} where N is the number of tasks. Each task is independent and characterized by Ti = (Ci, Pi, mi, ki), where Ci is Ti’s execution time, Pi is its period, and mi and ki are its (m, k)-firm deadline parameters. A set of N periodic tasks are executed on M homogenous processors P = {P1,…, PM}. When Ti is subject to (m, k)-firm deadline constraint, its distance, Disti, represents the minimum number of consecutive deadline misses to fall into a dynamic failure. With respect to the distance, we classify tasks into urgent and non-urgent. The Disti of urgent task Ti is one while that of non-urgent task Ti is greater than one. As Ramanathan et al. proposed in [6], we consider two types of jobs: mandatory and optional. We use the concept of (m, k)-pattern. Definition 1 (Quan and Hu [9]) The (m, k)-pattern of task Ti, denoted by Q i ¼ fpi0 ; pi1 ; . . .. . .piðki1Þ g; which satisfies the following: (i) Tij,is a mandatory kP i 1 pij ¼ mi : job if pij = 1 and optional if pij = 0, and (ii) j¼0

338

Y. Kong and H. Cho

The deeply-red pattern is one of (m, k)-patterns that was proposed by Koren et al. [10]. According to this scheme, a job Tij, the jth job of task Ti, is determined to be mandatory if 0 B j mod ki \ mi.

2.2 Power and Energy Consumption Model We consider the real-time tasks running on battery-powered multiprocessors, where energy recharge is not possible at runtime. To describe system behaviors related to energy, we use the following terminologies. • Emax: maximum energy capacity of the system. • Emin: energy threshold below which safe operation of the system is not guaranteed. • Ebudget (t): total amount of energy that the system can consume for running from time t. When the battery is fully charged, Ebudget(t) is defined as Emax-Emin. • Etotal(t, S0 ): total amount of energy required to run all jobs of tasks within a time interval [t, X] at processor speed S0 , where t is the current time and X is its mission time. • Elimit(t, S0 ): total amount of energy required to run tasks within a time interval [t, X] at processor speed S0 to schedule all mandatory jobs. The power consumption of CMOS-based processors consists of dynamic power consumption Pdynamic and stand-by mode power consumption Pstd. Pdynamic is a function of the processor speed s, as denoted by g(s), where g(s)  S3 as explained in [11]. We assume that all processors share the same clock signal and therefore all processors’ speeds are uniformly determined [10, 12]. We also assume that the processors support continuous frequency scale so that the normalized processor speed is in the range of (0, 1).

3 Problem Definition In this paper, we consider a scheduling problem for (m, k)-firm deadline constrained real-time tasks running on multiprocessors. It is distinct from [8] since we additionally consider two important design factors, i.e., multiprocessor environment and computational overload. The scheduling problem can be defined as How can all tasks running on battery-powered multiprocessors remain functional during their mission time (0, X) while satisfying their (m, k)-firm deadline constraints and, if necessary, satisfying as many deadlines as possible in both computational under-load and over-load cases?. Here the system is under-loaded if the total utilization demand of tasks, defined as the sum of Ci/Pi of all Ti, is less than or equal to M. Otherwise, the system is overloaded. This problem can be more

Energy-Constrained Scheduling

339

Table 1 Scheduling objectives Ebudget C Etotal

Ebudget C ELimit

Ebudget \ ELimit

Under-load

Overload

MTS: 100% DFR: 0% DSR: 100% MTS: maximizing DFR: 0% DSR: maximizing DSR: maximizing DFR: minimizing

DSR: maximizing DFR: minimizing

specifically described with respect to the target system’s energy budget and computational load as in Table 1. In Table 1, each cell is filled with the corresponding scheduling objectives. We use three metrics in order to quantitatively evaluate system performance, mission time satisfaction (MTS), dynamic failure ratio (DFR) defined as the number of dynamic failure occurrences per total number of job releases, and deadline satisfaction ratio (DSR) defined as the number of deadline satisfactions per total number of job releases.

4 Proposed Algorithm 4.1 Speed Determination With a single processor, it is known that the normalized processor speed that is equal to total utilization of the periodic real-time task set minimizes its energy consumption without missing any deadlines [11]. For multiprocessors, on the other hand, it is complicated to determine a proper speed. In [13], Funaoka et al. introduced max{Umax, U/M} as an optimal uniform frequency for multiprocessors. U is the total utilization, M is the number of processors, and Umax is the maximum value of individual task utilizations. However, this speed determination requires modification to be applied directly to the quantum-based scheduling algorithms, e.g., P-Fair, that makes their scheduling decisions every time quantum. In the quantum-based scheduling, all task parameters including Ci and Pi are defined as multiples of the time quantum, i.e., defined as integer values. Therefore, after applying the speed determination above, task parameters may become real numbers, which makes a task finish earlier than the next time quantum boundary. Example 1 Suppose that a task’s execution time is given as one. When the normalized processor speed is reduced to 0.8, the execution time accordingly changes from 1 to 1/0.8 and thus the task occupies two time quanta although its extended execution time is smaller than two. If real-time constraints permit, task execution

340

Y. Kong and H. Cho

Fig. 1 An example of relation between processor speed and occupied time quanta

time can be more extended up to the second time quantum’s boundary, which implies lower processor speed and reduced energy consumption. The latter indicates that in order to obtain a proper processor speed in quantum-based scheduling, we should consider the number of time quanta that tasks occupy at a given processor speed (Fig. 1). Definition 2 S0 is the minimum processor speed at which all tasks meet their deadlines under P-Fair scheduling algorithms. Our processor speed determination algorithm is described in Algorithm 1. A set of tasks where each task Ti is characterized by (Ci, Pi, mi, ki) is the input of this algorithm and a determined speed S0 is its output. If the processor speed changes, then the parameters of the task also change to (Ci0 , Pi, mi, ki). Example 2 Consider three tasks constrained by their (m, k)-firm deadlines as shown in Table 2. The number of processors is two and the mission time is given as 12 time quanta. For simplicity, the rests that are less than 10-3 are rounded. Algorithm 1 first determines a tentative S0 for the given task set. As the total utilization is 1.083 in the given task example, the processor speed is determined to be 0.54. When Stemp is 0.54, the time quanta occupied by each task is C10 = d1=0:54e = 2, C20 = d2=0:54e = 4, and C30 = d1=0:54e = 2, respectively. And the total utilization is 2.166. It is greater than the number of processors and therefore 0.54 for Stemp is not feasible. Then, Algorithm 1 attempts to find a task having the least difference between its occupied time quanta and its actual extended execution time. In the example, T2 has the least difference (Cleast = C2), so we reduce T2’s occupied time quanta from 4 to 3(S0 = 0.667). Consequently, S0 is computed as max{0.667,0.5} = 0.667. Algorithm 1. Calculate Speed (Q) P Stemp=calculate ( CPii )/M for(i=0;i\N;i++) Ci0 =Ci/S Ptemp Utemp= (Ci/Stemp)/Pi while(Utemp[M){ for(i=0;i\N;i++){   if( Ci0  Ci0 ! ¼ 0Þf   if ð Ci0  1Þ \  0  - 1)) C’least-( Cleast

Energy-Constrained Scheduling

341

Table 2 A task set T1 T2 T3

Ci

Pi

Ui

mi

ki

1 2 1

2 6 4

0.5 0.333 0.25

2 1 2

3 2 3

{ least = i;} } }  0   1Þ; Stemp ¼ Cleast =ð Cleast Utemp=0; for(i=0;i\N;i++){ Ci0 =Ci/Stemp   Utemp ¼ Utemp þ Ci0 =Pi; }} if(Stemp[1) Stemp=1; S0 = max{Stemp, Umax}

4.2 Static DVS Algorithm We first propose static EMRTS-MK running tasks at a fixed processor speed. Once processor speed is determined when the system starts, and it never changes at runtime. To determine processor speed, the static EMRTS-MK invokes Algorithm 1 once before the system starts. Algorithm 2 is a hierarchical scheduling algorithm consisting of both high-level scheduling and low-level scheduling. High-level scheduling is invoked at every boundary of each task in order to select and dispatch tasks to low-level scheduling. On the other hand, low-level scheduling is invoked at every time quantum to schedule the tasks selected by the high-level scheduling. Thus, whenever highlevel scheduling is invoked, low-level scheduling is also subsequently invoked. We use a P-Fair scheduling as the low-level scheduling. Algorithm 2 has three queues for scheduling tasks. QA is a queue containing tasks admitted for low-level scheduling. QU is a queue containing non-admitted and urgent tasks. QNU is a queue containing non-admitted and non-urgent tasks. EA: sum of energy required to run the tasks in QA UA: sum of utilization of tasks in an admit queue UTi: individual utilization of a task Ti ETi: energy required to run Ti

342

Y. Kong and H. Cho

In Algorithm 2, getTask() obtains the task at its boundary. updateDist(Ti) updates Ti’s distance. Sort(Q, A) sorts all tasks in queue Q in the order of A. move(Ti, Q) pops and inserts Ti into queue Q. Algorithm 2. Static EMRTS-MK 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36:

At start CalculateSpeed() At boundary of each task Tk, a present time t Tk=getTask(); updateDist(Tk); if Tk is urgent then insert(Tk, QU) else insert(Tk, QNU) calculate ETotal calculate ELimit calculate EA if(Etotal B Ebudget) { if(U B M){ Sort(QU, LeastUtilizationFirst) Sort(QNU, LeastDistanceFirst) for(Ti[ QU)do if (UA ? UTi B M){ move(Ti[ QU, QA) UA = UA ? UTi} else break for(Ti [ QNU)do if (UA ? UTi B M){ move(Ti[ QNU, QA) UA = UA ? UTi;} } } else if(ELimit B Ebudget){ Sort(QNU, LeastEDFirst) Sort(QU, LeastEDFirst) for(Ti[ QU)do if(UA ? UTi B M){ move(Ti[ QU, QA) UA = UA ? UTi } if(QU ==NULL){ for(Ti[ QNU){ if(ELimit+ETemp B Ebudget &&

UA ? UTi B M){ 37: move(Ti[ QNU, QA)

Energy-Constrained Scheduling

38: 39: 40: 41: 42: 43: 44: 45: 46: 47:

343

ETemp = ETemp ? ETi UA = UA ? UTi} else break } } else if(ELimit [ Ebudget){ Sort(QU, LeastEDFirst) Etemp = EA; for(Ti[ QU)do if(ETemp B Ebudget &&

UTi ? UA B M){ 48: 49: 50: 51: 52: 53: 54:

move(Ti[ QU, QA) ETemp = ETi ? Etemp UA = UA ? UTi} else break } At every quantum Pfair(QA)

P Theorem 1 Algorithm 2 satisfies all tasks’ deadlines if Ebudget C Etotal and Ci/ Pi B M(0 B i\N), where M is the number of processors and N is the number of tasks. Proof At a time instant of each boundary of a task, the task is inserted into either QU or QNU. Since the energy budget is larger than or equal to Etotal (not Elimit), it goes to the first if branch. If the total utilization is less than or equal to M, then all tasks are admitted to low-level scheduling. The low-level scheduling, P-fair, is optimal in the sense that it satisfies all tasks’ deadlines when the total utilization is less than or equal to M. Therefore, all tasks meet their deadlines and consequently no dynamic failure occurs. In Algorithm 2, Energy Density (ED) is used to sort urgent tasks in QU and non-urgent tasks in QNU in the order of the least ED first. Definition 3 (AlEnawy and Aydin [8]) The energy density EDi of a task Ti is the weighted ratio of energy consumption Ei of the mandatory instances of the task Ti divided by the number of dynamic failures DFmax that this task can cause during i . the mission time, that is, EDi = Ei/wiDFmax i The wi is a weight indicating the relative impact of Ti’s dynamic failures on the overall system performance. To ensure that important tasks are given priority in allocating the energy needed to meet their constraints, the designer can assign large weights to those tasks. The wi is a weight indicating the relative impact of Ti’s dynamic failures on the overall system performance [8].

344

Y. Kong and H. Cho

Fig. 2 EMRTS. (a) Tasks run by static EMRTS-MK when Ebudget(0) equals Elimit(0). (b) Tasks run by dynamic EMRTS-MK when Ebudget(0) is 70% of Elimit(0)

Fig. 3 DSR and DFR when the total utilization is 2.5

4.3 Dynamic DVS Algorithm The static EMRTS-MK runs at fixed processor speed S0 that is determined as the minimum speed. However, if Ebudget is lower than Etotal, then we cannot run all tasks due to the limited energy and instead, Algorithm 2 selects tasks to execute. The fact that only the selected tasks run implies that we can determine new processor speed just to run those tasks. The new processor speed, denoted by S*, is lower than S0 , and it consumes less energy. To determine S*, dynamic EMRTSMK invokes a dynamic speed determination algorithm, a simple modification of Algorithm 1, for tasks in QA at every boundary of each task before running Algorithm 2. Thus, dynamic EMRTS-MK can dynamically adjust the processor

Energy-Constrained Scheduling

345

Fig. 4 DSR and DFR when the total utilization is 4.5

speed at every boundary of each task. In the modification of Algorithm 1, the parameter Ci is replaced with Ri where Ri is Ti’s remaining execution time. Example 3 Consider the three (m, k)-firm deadline constrained tasks shown in Table 1. The mission time is also given as 12 time quanta and M is 2. In Example 2, S0 is determined as 0.667. In Fig. 2, the grey rectangles are selected jobs. Example 3 shows how EMRTS-MKs schedule tasks. In Fig. 2a, the static EMRTS-MK schedules urgent tasks setting S0 to 0.667 and S0 does not change at runtime. In Fig. 2b, we see that the processor speed changes at every boundary.

5 Experiment Result We evaluated the performance of our proposed algorithms using the OMNeT++ simulator. Two metrics DSR and DFR were used for the evaluation. We first randomly generated real-time tasks constrained by (2, 3)-firm deadlines. For setting task periods, we consider the task’s parameters used in [14] that show an example of mobile multimedia application parameters. One of our target applications is the mobile multimedia service, and thus we set the task periods to 30, 25, or 400. The mission time X is given as 4 times of the hyper-period of all tasks. Our experimental setting including (m, k)-firm deadline parameters and mission time follows that of [8]. We assume the number of processors is four. We varied the total utilization within the range of (2.5, 4.5), which covers both underloaded and overload situations. Ebudget is set to range from 20 to 100% of Elimit at time 0. For each experimental result, we repeated the same experiments 10 times to obtain the average value. Figure 3 shows when the system is under-loaded, static and dynamic EMRTSMKs decrease DSR and increase DFR as the ratio of Ebudget to Elimit decreases. Figure 3 also shows that the dynamic EMRTS-MK outperforms static EMRTS-

346

Y. Kong and H. Cho

MK in terms of both DSR and DFR. Especially, the dynamic EMRTS-MK incurs no dynamic failure even with 80% of Elimit to Ebudget. It implies that when the system is under-loaded, EMRTS-MK can satisfy the minimum quality of multimedia service even under a little shortage of energy by dynamically adjusting the processor speed. Figure 4 shows EMRTS-MK’s graceful performance degradation even in an overload situation. In the case of an overload, if the energy budget is sufficient to execute a mandatory job, then the dynamic algorithm achieves the same performance as the static algorithm. Since the system schedules the maximum number of tasks, the system cannot decrease the processor speed. We consider energy constrained scheduling for weakly hard real-time constrained tasks running on multiprocessors. Rather than simply saving as much energy as possible, we consider energy as a hard constraint under which the system remains functional and deliver an acceptable performance during the least prescribed mission time We prove that EMRTS-MKs provide assurance in quality of multimedia services under a certain condition. We also evaluate the proposed algorithms in several experiments, which quantitatively show EMRTS-MKs achieve the aforementioned scheduling objectives.

6 Conclusion Aiming at multimedia services on mobile devices, we consider energy constrained scheduling for (m, k)-firm deadline constrained tasks running on multiprocessors. We propose static and dynamic EMRTS-MKs to maximize the quality of multimedia services under both energy and weakly-hard real-time constraints. We prove that EMRTS-MKs provide assurance in the quality of multimedia services when the system with sufficient energy is under-loaded. Our experimental evaluation validates that EMRTS-MKs achieve our scheduling objectives. Acknowledgments This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MEST) (No. 2011-0011534).

References 1. Hamdaoui M, Ramanathan P (1995) A dynamic priority assignment technique for streams with (m,k)-firm deadlines. IEEE Trans Comput 44(12):1443–1451 2. Bernat G, Burns A, Llamosi A (2001) Weakly hard real-time systems. IEEE Trans Comput 50(4):308–321 Apr 3. Baruah SK, Cohen NK, Plaxton CG, Varvel DA (1996) Proportionate progress: a notion of fairness in resource allocation. Algorithmica 15(6):600–625 4. Andersson B, Tovar E (2006) Multiprocessor scheduling with few preemptions. In: Proceedings of the 12th IEEE international conference on embedded and real-time computing systems and applications, pp 322–334, Aug 2006

Energy-Constrained Scheduling

347

5. Cho H, Ravindran B, Jensen ED (2006) An optimal real-time scheduling algorithm for multiprocessors. In: Proceedings of the 27th RTSS, IEEE Computer Society, pp 101–110 6. Ramanathan P (1999) Overload management in real-time control applications using (m,k)firm guarantee. IEEE Trans Paral And Dist Sys 10(6):549–559 Jun 7. Niu L, Quan G (2005) A hybrid static/dynamic DVS scheduling for real-time systems with(m,k)-guarantee. In: Proceedings of the 26th RTSS 2005, pp 356–365, Miami, Florida, Dec 8. Alenawy TA, Aydin H (2005) Energy-constrained scheduling for weakly-hard real-time systems. In: Proceedings of the 26th IEEE RTSS’05, pp 376–385 9. Quan G, Hu X (2000) Enhanced fixed-priority scheduling with (m,k)-firm guaranteed. In: RTSS, pp 79–88 10. Koren G, Shasha D (1995) Skip-over: algorithm and complexity for overloaded systems that allow skips, In: RTSS 11. Aydin H, Melhem R, Mosse D, Alvarez PM (2004) Power-aware scheduling for periodic real-time tasks. IEEE Trans Comput 53(5):584–600 May 12. Magkils G, Semeraro G, Albonesi DH, Dropsho SG, Dwarkadas S, Scott ML (2003) Dynamic frequency and voltage scaling for multiple-clock-domain microprocessor. IEEE Micro 23(6):62–68 13. Funaoka K, Kato S, Yamasaki N (2008) Energy-efficient optimal real-time scheduling on multiprocessors. In: Proceedings of the 11th IEEE international symposium on object/ component/service-oriented real-time distributed 14. Yuan W, Nahrstedt K (2003) Energy-efficient soft real-time cpu scheduling for mobile multimedia systems. In: SOSP, October

A New Software Process Model: The Insect Model Mahmood Vahedi Moghaddam and Mehdi Zand Akbari

Abstract Process models are common topic in the software engineering world. While there are many of these models, just one of them is used very commonly in the real cases which is classical model. In this article a new model that is called ‘‘Insect model’’ is presented which has new suggestions for use in real operational environments.




Analysis Keywords Process models Planning Implementation Test










Design




Coding




Development




1 Introduction The existence of various amount of software process models, seem to bring no further increase in software quality and costs. An evidence of this is the common use of one of the most primitive process models: ‘‘the water fall model’’. This is caused by the simplicity and the low budget requirement of that waterfall model. But this model has great disadvantages which make it an unwilling choice for software projects. This leads us to use more dynamic and flexible models like DSDM. But flexibility and dynamic behavior of these kinds of models make some problems in

M. V. Moghaddam (&)
M. Z. Akbari Department of Computer Engineering Science and Technology, University of Mazandaran, Babol, Iran e-mail: [email protected] M. Z. Akbari e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_33, Ó Springer Science+Business Media B.V. 2012

349

350

M. V. Moghaddam and M. Z. Akbari

feasibility phase. Experienced software project managers claim the difficulty of Cost evaluation and effort estimation for these kinds of models. The other problem with existing process models is the way that we adopt our software to the end user and costumer needs. For instance, as Hughes says in his ‘‘software project management’’ book about XP process model: ‘‘test cases and expected results are devised before the design take place. To do this, the developer and user must agree exactly what should happen given specific circumstances’’, it is obvious that our users are not such professional to help us to create precise test cases. Insect model is away to think and plan more realistic. It insists on having a complete knowledge about end-users needs before starting the implementation phase. This strategy leads to (a) Risk rates reduction (b) Much easier cost evaluation and effort estimation (c) Lower production costs caused by having very small amount of code changes (almost no code changes).

2 What is a Software Process Model? Software process models often represent a networked sequence of activities, objects, transformations, and events that embody strategies for accomplishing software evolution. Such models can be used to develop more precise and formalized descriptions of software life cycle activities. Their power emerges from their utilization of a sufficiently rich notation, syntax, or semantics, often suitable for computational processing. Software process networks can be viewed as representing multiple interconnected task chains [1, 2]. Task chains represent a non-linear sequence of actions that structure and transform available computational objects (resources) into intermediate or finished products. Non-linearity implies that the sequence of actions may be non-deterministic, iterative, accommodate multiple/parallel alternatives, as well as partially ordered to account for incremental progress. Task actions in turn can be viewed a non-linear sequences of primitive actions which denote atomic units of computing work, such as a user’s selection of a command or menu entry using a mouse or keyboard. Winograd and others have referred to these units of cooperative work between people and computers as ‘‘structured discourses of work’’ [3], while task chains have become popularized under the name of ‘‘workflow’’ [4]. Task chains can be employed to characterize either prescriptive or descriptive action sequences. Prescriptive task chains are idealized plans of what actions should be accomplished, and in what order. For example, a task chain for the activity of object-oriented software design might include the following task actions:

A New Software Process Model

351

Develop an informal narrative specification of the system. Identify the objects and their attributes. Identify the operations on the objects. Identify the interfaces between objects, attributes, or operations. Implement the operations. Clearly, this sequence of actions could entail multiple iterations and non-procedural primitive action invocations in the course of incrementally progressing toward an object-oriented software design. Task chains join or split into other task chains resulting in an overall production network or web [1]. The production web represents the ‘‘organizational production system’’ that transforms raw computational, cognitive, and other organizational resources into assembled, integrated and usable software systems. The production lattice therefore structures how a software system is developed, used, and maintained. However, prescriptive task chains and actions cannot be formally guaranteed to anticipate all possible circumstances or idiosyncratic foul-ups that can emerge in the real world of software development [5, 6]. Thus, any software production web will in some way realize only an approximate or incomplete description of software development. Articulation work is a kind of unanticipated task that is performed when a planned task chain is inadequate or breaks down. It is work that represents an open-ended non-deterministic sequence of actions taken to restore progress on the disarticulated task chain, or else to shift the flow of productive work onto some other task chain [7, 8, 9, 10]. Thus, descriptive task chains are employed to characterize the observed course of events and situations that emerge when people try to follow a planned task sequence. Articulation work in the context of software evolution includes actions people take that entail either their accommodation to the contingent or anomalous behavior of a software system, or negotiation with others who may be able to affect a system modification or otherwise alter current circumstances (Bendifallah 1987, Grinter 1996, Mi 1990, Mi 1996, Scacchi and Mi 1997). This notion of articulation work has also been referred to as software process dynamism.

3 New Insect Model This model has three main steps. In the first step that can be consider as a head of the model, we have project evaluation with the foundation user agreement. In this step project manager must has several meetings with end-users and also costumers for having their opinion and using it in the feasibility study of the project. As illustrated in Fig. 1 pay attentions that after finishing this step, it is not possible to come back to this part again. If, for any reason, you decided to go backward in the model, it would be considered as a new project.

352

M. V. Moghaddam and M. Z. Akbari

Fig. 1 Insect model Table 1 Comparing different factors that have effect on software process model Models Speed of Sharing Cost Effort Total development Stakeholders estimation estimation Insect model The linear sequential model Incremental model The spiral model Win–win spiral model Dynamic systems development method (DSDM) Extreme programming (XP)

2 2 1 2 3 1

1 3 2 2 1 1

2 3 2 3 3 3

2 3 3 3 3 3

7 11 8 10 10 8

2

1

3

3

9

The dynamic step that can use the floatability of the project effort estimation is the middle part. In this step first the requirements of the user must be determined and according to them the analysis and design of the project must be occurred. These two part (analysis and design) must have an internal test for confirmation of the user requirements. After that, it is the time for creating demo to show to the

A New Software Process Model

353

end-user.So with constructing demo codes (which is mostly just a user interface) we can have the ideas, new requirements and view point of non-software-professional end users. This loop will be continued until the user confirms that his requirements are completely involved in the project. The state of ‘‘having a compete vision on the project goals’’ should also be confirmed by the analysis group and project manager. Then the project members can start the coding phase with free mind of having any serious code changes. And after coding it is the time of final test and V&V phase and at last the maintenance. As mentioned before, this will reduce the project budget requirement dramatically. Notice that in this model, although we have a dynamic behavior, but it is fully predictable and under control, so the harmful face of dynamicity, which usually lets the whole project down (because of unpredictability of costs, efforts and delivery time), is completely vanished.

4 Compare with Other Models There are some factors that effectively influence on performance of a process model. So comparing these factors can lead us to best model. Some of these factors are Speed of development, sharing stakeholders in duration of project, and some project management factors as cost estimation and effort estimation. We have get a priority between number one and three to each of these factors for a special process model according of our experience and studies. It is obvious that a model which has the lowest total priority is a good suggestion to take. But notice that according to different teams and different managers these priorities may have change a little. According to above results it seems that insect model is a best and intermediate model in various factors and is a good suggestion to take (Table 1).

5 Conclusion In this article, we produced a new process model that seems to have an undercontrol dynamic behavior that leads to a better software project management method, which is much easier to implement and manage. Because in this model we have just low cost changes, it introduces a high level of efficiency.

References 1. Kling R, Scacchi W (1982) The web of computing: computer technology as social organization, ‘‘advances in computers’’, vol 21. Academic Press, New York, pp 1–90 2. Garg PK, Scacchi W (1989) ISHYS: design of an intelligent software hypertext environment. IEEE Expert 4(3):52–63

354

M. V. Moghaddam and M. Z. Akbari

3. Winograd T, Flores F (1986) Understanding computers and cognition: a new foundation for design. Ablex Publishers, Lexington 4. Bolcer GA, Taylor RN (1998) Advanced workflow management technologies. Softw Process Improv Pract 4(3):125–171 5. Bendifallah S, Scacchi W (1989) Work structures and shifts: an empirical analysis of software specification teamwork. In: Proceedings of 11th international conference on software engineering, IEEE Computer Society, pp 260–270 6. Mi P, Scacchi W (1990) A knowledge base environment for modeling and simulating software engineering processes. IEEE Trans Knowl Data Eng 2(3):283–294 7. Bendifallah S, Scacchi W (1987) Understanding Software Maintenance Work. IEEE Trans Softw Eng 13(3):311–323 8. Grinter R (1996) Supporting articulation work using software configuration management. J Comput Support Coop Work 5:447–465 9. Mi P, Scacchi W (1996) A meta-model for formulating knowledge-based models of software development. Decis Support Syst 17(4):313–330 10. Scacchi W, Mi P (1997) Process life cycle engineering: a knowledge-based approach and environment. Intell Syst Account Finance Manag 6(1):83–107

Improving Recommendations in Tag-Based Systems with Spectral Clustering of Tag Neighbors Rong Pan, Guandong Xu and Peter Dolog

Abstract Tag as a useful metadata reflects the collaborative and conceptual features of documents in social collaborative annotation systems. In this paper, we propose a collaborative approach for expanding tag neighbors and investigate the spectral clustering algorithm to filter out noisy tag neighbors in order to get appropriate recommendation for users. The preliminary experiments have been conducted on MovieLens dataset to compare our proposed approach with the traditional collaborative filtering recommendation approach and naive tag neighbors expansion approach in terms of precision, and the result demonstrates that our approach could considerably improve the performance of recommendations. keywords Tag neighbors Social tagging

 Recommender system  Spectral clustering 

This research is supported by EU FP7 ICT project M-Eco: Medical Ecosystem Personalised Event-based Surveillance (No.247829). R. Pan (&)  P. Dolog Department of Computer Science, Aalborg University, Aalborg, Denmark e-mail: [email protected] P. Dolog e-mail: [email protected] G. Xu Centre for Applied Informatics, Victoria University, Melbourne, Australia e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_34, Ó Springer Science+Business Media B.V. 2012

355

356

R. Pan et al.

1 Introduction The success of Web 2.0 applications has faced a new era for sharing and organizing documents in online social communities. A large number of social tagging sites, like Delicious, CiteUlike, Digg, or Flickr, has sprung up in a short period. Tag, as one kind of specific lexical information that is user-generated metadata with uncontrolled vocabulary, plays a crucial role in such social collaborative tagging systems. With the help of tagging data, user annotation preference and document topical tendency are substantially coded into the profiles of users or documents for further web data management. However, one serious problem in social tagging systems is the sparseness of tagging data. Users typically like to annotate a small fraction of the shared documents only that they are really interested in, leaving most of the other documents with incomplete annotations. As the tags are of syntactic nature, in a free style and do not reflect sufficient semantics, the problems of redundancy, ambiguity and less semantics of tags are often incurred in all kinds of social tagging systems [8, 9]. Due to the problems of less semantics and sparse quality of tagging data, the tag-based recommender systems which rely simply on the lexical similarity of tags alone are likely to neglect the retrieval of some closely related documents. Therefore, our motivation in this paper is to expand the tag vector expression of user and document by employing the statistical learning in tagging data. One technical solution we present in this paper is the expansion of tag expression with co-occurred tag neighbors, with which we are able to facilitate the organization of information documents in search and navigation. However, there is another problem emerging: not all the social tagging systems proposed so far maintain high quality and quantity of tag data, even the tag elements in tag expression can be expanded, such tag expression also contains lots of inappropriate tags, we particularly call them as ‘‘noisy tags’’. Thus another challenge is how to expand the tag expression in an appropriate way and filter out the noisy tags to enhance the recommendations. We propose a new recommendation approach by combining the tag neighbor expansion and noisy tag filtering techniques, where the former is realized by the collaborative approach and the latter is manipulated by the spectral clustering algorithm. With the enhanced tag expression by appropriate tag neighbors, we remeasure the similarity between the expanded user profile and document profile to improve the tag-based recommendations. The main contributions made in this paper are: 1. The collaborative tag neighbor expansion approach via co-occurrence analysis. 2. The filtering of noisy tag neighbors by utilizing the clustering algorithm. 3. The improved tag-based recommendations by enhancing the quality and quantity of tag neighbors. The rest of the paper is organized as follows: Sect. 2 presents the related work in the field of tag neighbors and tag clustering. In Sect. 3, we describe the

Improving Recommendations in Tag-Based Systems

357

preliminaries for the data model, user profile and document profile. Section 4 discusses the detailed process of extending the tag neighbors. The experiment design, evaluation measures and the comparison of the results are in Sect. 5 We conclude the paper in Sect. 6.

2 Related Work Tags have been recently studied in the context of recommender systems due to various reasons. Recommendations of relevant documents should be based on the sufficient occurrences for similar signals expressed by tags. We review the related literatures from the perspectives of tag expansion and tag clustering. Tag Expansion in Recommendation: Bayyapu and Dolog in [1] try to solve the problems of sparse data and low quality of tags from related domains. They suggest using tag neighbors for tag expression expansion. However the tag neighbors are based on the content of documents. We propose another approach to extend the tag set for the user profile by collaborative filtering approach. In [2], an interesting approach was proposed to model the documents in social tagging systems as a document graph. The relevance of tag propagated along edges of the documents graph is determined via a scoring scheme, with which the tag prediction was carried out. Tag Clustering: The purpose of tag clustering is the ability of aggregating tags into topic domains. In [4] topic relevant partitions are created by clustering documents rather than tags. By clustering of documents, it improves recommendation by distinguishing between alternative meanings of a query. While in [3], clusters of documents are shown to improve recommendation by categorizing the documents into topic domains.

3 Preliminaries 3.1 Folksonomy The folksonomy is a three-dimensional data model of social tagging behaviors of users on various documents. It reveals the mutual relationships between these threefold entities, i.e. user, document and tag. A folksonomy F according to [5] is a tuple F = (U, T, D, A), where U is a set of users, T is a set of tags, D is a set of web documents, and A  U  T  D is a set of annotations. The activity in folksonomy is tijk  fðui ; dj ; tk Þ : ui  U; dj  D; tk  Tg; where U ¼ fU1 ; U2 ;    ; UM g is the set of users, D ¼ fD1 ; D2 ;    ; DN g is the set of documents, and T ¼ fT1 ; T2 ;    ; TK g is the set of tags. tijk ¼ 1 if there is an annotation ðui ; dj ; tk Þ; otherwise tijk ¼ 0:

358

R. Pan et al.

3.2 User Profile and Document Profile The constructed folksonomy data model is actually a three-dimensional array (or called tripartite hyper-graph). In real applications, we often decompose the tagging data into two two-dimensional matrices, i.e., user profile and document profile. User profile can be used to store the descriptive tags of the user’s characteristics and preferences. The document profile is represented by the tags generated by the group of users tagging the documents. In the context of social tagging systems, the user profiles and document profiles thus are expected to be represented by the representative tags. Therefore the process of user and document profiling is to capture the significant tags from a large volume of tagging data in a social collaborative environment. We will utilize them based on our previous work [7].

3.3 Similarity Measure for Tags The similarity is quantity that reflects the strength of relationship between two objects. In the previous part, each user profile and document profile can be represented by the pair of tags and frequencies. In this manner, we perform the transformation of above two matrices and utilize the cosine function to measure the similarity between two tags. Its value ranges from 0 to 1, the higher value of the similarity, the more similar the objects are.

4 Expanding Tag Expression with Tag Neighbors for Recommendation To deal with the challenge of of low tag quality that commonly incurred in social tagging information systems, we propose an approach of expanding tag expression with tag neighbors. Based on the tag neighbor expansion, we devise a new tag-based recommendation. In this section, we give the details.

4.1 Collaborative Approach for Tag neighbor The basic idea of proposed approach is to utilize the tag neighbors to extend the users’ or documents’ profiles which are represented by the tags. In the tag similarity matrix, each tag has different similarity weights with other tags, we assume the higher the weight, the more similar the tags are to the target tag. To realize the task of expanding the tag, the major difficulty is how to define the tag neighbors and how to locate them from the total tags. Here we adapt a statistical definition of

Improving Recommendations in Tag-Based Systems

359

tag neighbor—the tags which are co-occurred most frequently or they have the higher similarity weight in the similarity matrices to the target tag, are the neighbors for each other. So the N tags according to the top-N similarity weight can be defined as the tag neighbors of an individual tag. After such steps, each tag will have an additional neighboring tag set which will help to improve the quantity of the tag expression. Given an arbitrary tag T, its neighboring tags are defined as: NbðTi Þ ¼ fTj : Tj 2 TopNfSMðTi ; Tj Þgg where TopNfSMðTi ; Tj Þg is the tags which possess the top-N highest similarity values to tag Ti :

4.2 Tag Neighbor Filtering based on Spectral Clustering In this section we use tag clustering to find the tag aggregates with similar functions or topics.The spectral clustering is based on the graph partition which maps the original co-occurrence observations onto a new spectrum space with a reduced dimensionality. The obtained partition guarantees the disjoint clusters with minimum cut optimization. Especially in the context of social tagging systems, we know each tag can be expressed a column vector of user profile and document profile, i.e., Ti ¼ TUi and Ti ¼ TDi : Then we stack up TUi and TDi and form a new tag vector over users and documents, Ti ¼ TUi [ TDi : At last, we employ spectral clustering on the integrated tag vectors to get tag clusters. The pseudo codes of the spectral tag clustering are listed in Algorithm 1. From the above tag clustering, we obtain a set of tag clusters: TC ¼ fTC1 ;    ; TCC g ¼ fðT11 ;    T1i1 Þ;    ðTC1 ;    ; TCiC Þg

Algorithm 1: Spectral Tag Clustering Input: The tag-user matrix and tag-document matrix, i.e., TU ¼ fTUi ; i ¼ 1;    ; Kg; TD ¼ fTDi ; i ¼ 1;    ; Kg Output: A set of C tag clusters TC ¼ fTCc ; c ¼ 1;    ; Cg such that the cut of C-partitioning of the bipartite graph is minimized 1 Construct the integrated tag-user-document matrix by stacking up the above two matrix, Ti ¼ TUi [ TDi ; 2 Calculate the diagonal matrices D of T; 3 Form a new matrix RT ¼ D1=2 TD1=2 ; 4 Perform SVD (Singular value decomposition) operation on RT, and obtain k singular vectors Ls to create a new projection matrix RV;

360

R. Pan et al.

5 Execute a clustering algorithm on RV and return clusters of tags in the integrated vectors: TC ¼ fTCc ; c ¼ 1;    Cg: The naive tag neighbors will be calculated by collaborative approach. However, the problem is that not all of the expanded tag neighbors are appropriate for the target tag since there are some noisy tags in the tag neighbors. From the above way, all of the tags are clustered into the several individual clusters where each cluster is a set of tags sharing the similar functions or topics. Bearing this phenomenon in mind, we thereafter propose to utilize tag clusters to determine whether the naive tag neighbors are included in the expansion of tags. The basic idea of clustering filtering is: the neighboring tags from the same tag cluster of the target tag contribute collaboratively to specific function or topic, being kept as the appropriate tag neighbors for tag expression expansion; otherwise be discarded. So the next processing step is to filter out the noisy tags according to the discovered tag clusters. In the previous sections, each tag has an expanded tag neighborhood, which might belong to different clusters. To ensure all neighboring tags are from the same tag cluster, each tag in the expanded neighborhood will be compared with all the tags from the tag cluster where the target tag is assigned. If the expanded neighbor appears in the same cluster, it then can be considered as the appropriate neighbor of the tag tag, making it kept in the expanded tag set; otherwise, it should be filtered out. After such steps above, the left elements could be defined as the tag neighbors for the target tag, and the quality of the tag neighborhood will be accordingly improved. Also in such way the density in the integrated tag-user-document matrix could be increased substantially. For example, given the Tag Ti belongs to a certain cluster Cj ; and the Ti has a expanded tag neighborhood as TN ¼ fTN1 ;    ; TNk ;    ; TNK g; then we can compare each tag TNk with the tags in Cj ; if TNk is existed, it can be defined as the tag neighbor; otherwise it will be discarded. After such steps, the tag neighborhood will be updated.

4.3 Improved Recommendation with Tag Neighbor Expansion After the tag neighbor expansion is completed, we get updated user profiles and document profiles in the forms of tag vector expression with expanded tag neighbors. We then utilize the similarity measure between users and documents to make tag-based recommendations. The whole algorithm for improved recommendation with tag neighbor expansion is described as Algorithm 2. Algorithm 2: Tag Neighbor Expansion for Recommendation Input: A collected tagging data Output: A list of top-N documents for the candidate user

Improving Recommendations in Tag-Based Systems

361

1 Pre-process the tagging data to construct a user profile matrix and a document profile matrix. 2 Represent the tag in the user profile vectors and document profile vectors. 3 Calculate the tag similarity matrix. 4 Get the top-N tags according to the highest values in the similarity matrix for each tag. 5 Partition tags into different clusters by spectral clustering algorithm. 6 Check all the tag neighbors generated in the previous steps whether they belong to the same cluster with the original tag, and filter out noisy tag neighbors. 7 Update the tag vectors of user profiles and document profiles with tag neighbors. 8 Calculate the similarity between the candidate user’s tag vector and the document tag vector, and rank the documents according to the similarity values in a descending order. 9 Select the top-N documents with the first N highest similarities as the recommendions to the candidate user.

5 Experimental Evaluations The experiments are performed on the ‘‘MovieLens’’ dataset. Our experiments focus on results conducted on the implemented tag-based recommender system with tag neighbor expansion. The goal of such experiments is to show that the document recommendations derived by using our proposed approach would result in an improvement of document recommendation performance.

5.1 Dataset and Experimental Setup As for experiment dataset, we utilize the part of the ‘‘MovieLens’’ data, which contains tags provided by users on movies. It includes 114 users, 1006 documents and 1589 tags. The setting of the dataset is that each user and document has at lest 5 annotated tags. Following the traditional protocal which chooses 20–30% data for the testing data, we divide 75% data as the training data and the rest 25% as the testing data.

5.2 Modularity Metric Modularity is originally proposed to assess the division quality of a network. The modularity of a particular division of a network is calculated based on the

362

R. Pan et al.

differences between the actual number of edges within a community in the division and the expected number of such edges if they were placed randomly. Consider a particular division of a network into k communities. We can define a k  k symmetric matrix SM whose element smij is the fraction of all edges in the network that link vertices in community p to vertices in community q. The similarity of smCpq between the two clusters Cp and Cq ;is defined as [6], P P cp 2Cp c 2C cpq Pp q smCpq ¼ P ; p; q ¼ 1; 2. . . m; cp 2C cq 2C cpq where cpq is the element in the similarity matrix SM. When p = q, the smCpq is the similarity between the elements inside the clusters, while p 6¼ q; the smCpq is the and the cluster Cq : So the similarity between the cluster Cp P P condition of a high quality cluster is argmaxð p smCpp Þ and argminð p;q smcpq Þ; p 6¼ q; p; q ¼ 1; 2; . . . m: This quantity measures the fraction of the edges in the network that connects vertices of the same type minus the expected value of the same quantity in a network with the same community divisions. Here we compare the result of Q values by using Spectral Clustering, Single Linkage Clustering and Random Clustering. The result below is based on the average result of executing the same experiment ten times over the same dataset. Of the entire 114 user profiles and 1006 document profiles constructed, we employ various clustering algorithms to build up the tag groups. The number of cluster varies from 2 to 800. The upper number is set to 800 because it is half of the total amount. When the number of cluster is higher than 800, the average number of members in each cluster is lower than 2, which will not provide reasonable clustering information. The results are shown in Fig. 1. From the figure, we observe that the value of Q for Spectral Clustering consistently outperforms bettwer than the other two algorithms, which also justifies the rationale of choosing Spectral Clustering. When the cluster number reaches 120 the Q gets the maximal as 0.344. With the growth of the number of clusters, the value of Q is gradually decreasing to 0.03. It is concluded that for this dataset 120 clusters is the best choice.

5.3 Precision Evaluation In the experiment we have 246 existing documents for testing. We compared the precision for the recommended documents to the individual user. The traditional way is to calculate the similarity between the user profile and the document profile in the tag vector, and the system will recommend the N documents to the user according to the top-N similarity values. Another approach is to calculate such similarity based on the naive tag neighbor expansion by collaborative filtering

Improving Recommendations in Tag-Based Systems

363

Fig. 1 Comparison of three clustering algorithms on 1589 tags to select optimal number of clusters

Fig. 2 Precision comparisons on three different strategies

approach. However, the disadvantage is that such tag neighbors include many noisy tags, degrading the precision of recommendation. Our proposed approach is to filter out the noisy tags by utilizing the clustering. We will calculate the precision in such step: The top-N documents will be recommended to the user by ranking the similarity values derived with various approaches, the recommended documents are compared with the existing documents in the test data. If there are k documents appeared out of the N recommended documents in the test data, the precision for the individual user is defined as t. t¼

Ki 100% N

We average the whole precision for all of the users, and compare the recommendations form top 1 to 40 documents to the users in the experiments. Other two comparable approaches used in the experiments are pure tag vector approach and naive tag neighbor expansion approach. We denote these three approaches as TagNeighborhood with Clustering, TagNeighborhood and Pure Tag Vector. The precision comparison of three approaches for top 40 recommendations is shown as Fig. 2. The ‘‘conditional recommended documents’’ in Fig. 2 means that the compared documents in testing data are with the rating value higher than or equal 4. Because

364

R. Pan et al.

the MovieLens dataset contains the users’ evaluations for the movies by rating value from 1 to 5, with 1 denoting the lest preferred and 5 the best preferred. In the experiments, the average precision of TagNeighborhood with Clustering is 84.1, the TagNeighborhood is 78.3, the pure tag vector is 37.4. In summary, the experimental results validate the advantage of the tag neighbor expansion with clustering in recommendations.

6 Conclusion In this paper, we have proposed a collaborative filtering approach for expanding the tag neighbors. Such expansion helps to enhance the tag expression quality in social tagging systems. We have also introduced to utilize the clustering algorithm to filter out the noisy tag neighbors. We have carried out evaluation experiments on the MovieLens dataset. The evaluation results have shown the improved recommendations by our proposed approach.

References 1. Bayyapu KR, Dolog P (2010) Tag and neighbour based recommender system for medical events. In: Proceedings of MEDEX 2010: the first international workshop on web science and information exchange in the medical web colocated with WWW 2010 conference, 2010 2. Budura A, Michel S, Cudré-Mauroux P, Aberer K (2009) Neighborhood-based tag prediction. The semantic web: research and applications, pp 608–622 3. Chen H, Dumais S (2000) Bringing order to the web: automatically categorizing search results. In: CHI ’00: proceedings of the SIGCHI conference on human factors in computing systems, New York, NY, USA, ACM, pp 145–152 4. Hayes C, Avesani P (2007) Using tags and clustering to identify topic-relevant blogs. In: International conference on weblogs and social media, March 2007 5. Hotho A, Jäschke R, Schmitz C, Stumme G (2006) Folkrank: a ranking algorithm for folksonomies. In: Althoff K-D, Schaaf M (eds), LWA, Hildesheimer Informatik-Berichte, vol 1. University of Hildesheim, Institute of Computer Science, pp 111–114 6. Newman MEJ, Girvan M (2004) Finding and evaluating community structure in networks. Phys Rev E 69(2):26113 7. Pan R, Xu G, Dolog P (2010) User and document group approach of clustering in tagging systems. In: Proceeding of the 18th international workshop on personalization and recommendation on the web and beyond. LWA 2010 8. Xu G, Gu Y, Dolog P, Zhang Y, Kitsuregawa M (2011) Semrec: a semantic enhancement framework for tag based recommendation. In: Proceedings of twenty- fifth AAAI conference on artificial intelligence (AAAI-11), 2011 9. Xu G, Zong Y, Pan R, Dolog P, Jin P (2011) On kernel information propagation for tag clustering in social annotation systems. In: Proceeding of 15th international con- ference on knowledge-based and intelligent information and engineering systems (KES2011), 2011

Event-Centric Test Case Scripting Method for SOA Execution Environment Youngkon Lee

Abstract Electronic collaboration over the Internet between business partners appear to be converging toward well-established types of message exchange patterns that involve both user-defined standards and infrastructure standards. At the same time, the notion of event is increasingly promoted for asynchronous communication and coordination in SOA systems. In collaboration between partners or between components is achieved by the means of choreographed exchanges of discrete units of data—messages or events—over an Internet-based protocol. This paper presents an event-centric test case scripting method and execution model for such systems. Keywords SOA


Event-driven
Process invocation

1 Introduction While current Web Service technologies show much progress, current services are mainly limited to atomic services. Thus, they are not adequate to handle the autonomous and complex services in realistic settings. In dealing with this problem, some research works have developed languages to compose the individual Web Services into transactions or workflows. Web Services Flow Language (WSFL) [1] was designed for service compositions in the form of a workflow, and XLANG [2] for the behavior of a single Web Service. However, these works are not sufficient for providing the adaptive web Services generated from a particular context. Y. Lee (&) e-Business Department, Korea Polytechnic University, Jeongwangdong, Siheung, 2121 Korea e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_35, Ó Springer Science+Business Media B.V. 2012

365

366

Y. Lee

Electronic collaborations over the Internet between business partners (e-Business/e-Government) appear to be converging toward well-established types of message exchange patterns that involve both user-defined standards and infrastructure standards. At the same time, the notion of event is increasingly promoted for asynchronous communication and coordination in Event-Driven Architectures (EDA) that are considered as either complementary to or part of SOA systems. In both cases collaboration between partners or between components is achieved by the means of choreographed exchanges of discrete units of data—messages or events—over an Internet-based protocol. Such systems require an event-centric test case scripting markup and execution model. In e-Business transactions as in EDAs, partners or components must agree on the use of a combination of standards in order to interoperate with each other. Typically, these standards can be classified into three layers: – Messaging infrastructure standards, ranging from transport level to higher-level messaging protocols and quality of service (QoS) including reliability and security, such as those defined as SOAP extensions, or REST (Representational State Transfer). – Multi-message exchange standards as manifested in business processes and choreographies. – Business document standards may be business content structure and semantics, taxonomies in use, code lists, semantic rules, or the XML schema modeling style. They are industry-specific (e.g. RosettaNet PIP schemas, AIAG Inventory Visibility and Interoperability schemas), horizontal document standards, or regional guidelines. There have been conformance and interoperability test suites and testing tools for above layer individually. But the testing of integrations of standards has been ad-hoc, or limited mostly to standards in the messaging infrastructure. Although the need for testing some form of integration of standards has been well recognized for infrastructure standards, there has been little support for testing integrations that extend to the use of standards specific to a business—e.g. for documents or choreographies. Such integrations can be construed as userdefined profiles. For example, the level of QoS required for a business transaction may depend on the nature of business data being exchanged, or on some property defined by the related business process. Testing and monitoring these infrastructure layers and their integration also requires that test cases access a combination of contracts—agreements, policies or business transaction patterns—represented by meta-level documents. This compliance objective goes beyond quality assurance for the messaging function: it requires the monitoring of live transactions in production environments, as well as verifying conformance of business endpoints in operation conditions. This calls for a flexible test execution model that can accommodate performance constraints as well as different timeliness constraints—e.g. where tests are either deferred over log data, or executed on live exchanges in a monitoring mode.

Event-Centric Test Case Scripting Method

367

Consequently, the execution model of such test cases or monitoring cases, must accommodate dynamic conditions requiring real-time or near real-time error detection or measurement, allowing to correct and report on business exchanges as they proceed. The output of a monitoring script also must provide more information than a report of the type pass/fail. Different ways of ‘‘passing’’ or ‘‘failing’’ must be reported on, as well as identifying the types of business transactions. The output must be easy to format and feed to another decision engine, such as a rule engine that will process this input in real-time. For example, a rule may decide to generate an alert if a business transaction lasts too long, depending on the nature of the transaction and on the SLA associated to these business partners. This paper defines a testing and monitoring model, as well as a test script markup, so that test cases or monitoring cases can be fully automated, and portable across test environments. In Sect. 2, we summarize the related works regarding the web service flow language for testing. Section 3 presents the concept of EventCentric Test Case Script (EVEC), Sect. 4 describes the implementation of EVEC, and we conclude in Sect. 5.

2 Related Works In fact, the automatic or semi-automatic management of service flows over the Web has not been achieved yet. In the Web Services model that is quite different from traditional one, there are a large number of similar or equivalent services which user can freely select and use for their application. Since the service is developed and deployed by the third party, the quality of service is not guaranteed. The services may not be adequate as per service requestor’s requirements and kept evolving, without notification to service requestors, according to the provider’s requirements and computing environment. Thus, it is important to provide adaptability to evolving services as well as diverse context of services. Kammer et al. [3] suggested workflow to be dynamic, which allows changes with minimal impact to the ongoing execution of underlying workflow, as well as be reflexive, which provides knowledge about a workflow’s applicability to the context and the effectiveness of its deployment evaluated over time. Understanding constraints and context associated with services may affect the quality of service. From this perspective, optimization may occur through the evaluation and refinement of a previous service flow. Automatic composition of services is challenging, because it is difficult to capture semantics and context of services and measure the quality of services. One exemplary effort that aims for this function is DAML-based Web Service Ontology (DAML-S) [4], which describes the properties and capabilities of Web services. Workflow technology has been around since a decade ago and has been successful in automating many complex business processes. A significant amount of work has been done in this field, which deals with different aspects of workflow

368

Y. Lee

technology process modeling, dynamic workflows, and distributed workflows. Process modeling languages such as IDEF, PIF, PSL or CIMOSA [5] and frame based models of services were used to design process typing, resource dependencies, ports, task decomposition and exception. Current research on web services paves way for web service based workflows, which has obvious advantages pertaining to scalability, heterogeneity, reuse and maintenance of services. Major issues in such inter-organizational service based workflows are service discovery, service contracts and service composition. Web Services Flow Language (WSFL) was proposed to describe compositions of services in the form of a workflow, which describes the order of service invocation. Service composition aids such as BizTalk [6] were proposed to overcome the limitations of traditional workflow tools which manually specify the composition of programs to perform some tasks. Other industrial initiatives such as BPEL4WS [7], and XLANG concentrates on service representation issues to tackle the problems of service contracts, compositions and agreements. Current efforts are to automate the complete process of service discovery, composition and binding, using the machine understandable languages. Some other recent advances are WS-Transaction and WS-Coordination which define protocols for executing Transactions among web services. There is a research for modeling QoS of workflows [8], and defining a QoS based middleware for services associated with the underlying workflow [9], but it doesn’t take into account QoS factors related to Internet based services. Some researchers describe QoS issues related to web services from the provider’s perspective [10]. We believe that the current research has not delved into QoS issues related to Web Service based workflows, and many critical issues related to the availability, reliability, performance and security of Web Services need to be handled. Our approach tactfully utilizes and monitors these QoS parameters to provide a consistent service interface to other applications in the workflow through adaptive QoS based selection, binding and execution of Web Services.

3 Event-Centric Test Case Script Event-centric test case script (EVEC) is designed so that the same scripts can be used either in live monitoring mode, or in analysis of past events from a log (referred to as deferred mode in this paper called hereafter the ‘‘deferred mode’’) or yet in mixed situation. Testing and Monitoring of Business Processes as well as more generally of systems the behaviour of which can be traced by events, fall in the following three categories: – Compliance with specifications. Such specifications may be of a business transaction, business process definition, documents exchanged, or of infrastructure behaviour (e.g. messaging protocol). Enabling the automatic generation of EVEC scripts from such specifications when these specifications are formal—e.g. process definition, choreographies, document schema or rules—is

Event-Centric Test Case Scripting Method

369

part of the requirements although the methodology to achieve this is out of scope of this document. Some test assertion design and best practices, such as those in Test Assertions Guidelines [11] may be used for deriving scripts from such representations even when automatic generation is not possible. – Compliance with agreements. Such agreements may be business agreements such as SLAs, or regulatory compliance rules. They may also be infrastructure configuration agreements (e.g. ebXML CPA, WS-Policy). This category of application includes SLA monitoring, business metrics and aspects of business activity monitoring (BAM) that are closest to operations, e.g. for regulatory compliance. – Business Operation intelligence. Such monitoring is not directly related to compliance, but primarily intended for generating reports and various analytics of business activities. This includes analyzing the logs of processes and business transactions for reports and BI. This category of application includes BAM (business activity monitoring). In its dynamic aspect, this monitoring is addresses the need for visibility in business processes and service-oriented systems, which include problem detection/anticipation, diagnostics and alarm generation. Each one of the above categories may be considered both in a realtime context (e.g. generation of alarms and notifications during operation) and a deferred, off-line analysis context (periodic generation of reports or metrics with no direct, automatic feedback loop to operations). In both cases, the same input—in form of events—is assumed. From the viewpoint of script execution semantics, ‘‘live’’ and ‘‘deferred’’ are not distinguished: the same script is executable on input that is either live or logged. To ensure flexibility for handling various monitoring contexts and situations, mixing of both execution modes must be supported: – A script may start executing ‘‘deferred mode’’ with its events already partially logged, and then catch-up with the on-going logging of events and continue ‘‘live’’. – Conversely, a script may start live, and if its execution engine is interrupted for some reason, may resume its analysis of events that have already been logged while the test engine was stopped, in deferred mode. Then it may catch-up with events and eventually go live again. When events are consumed in a publishsubscribe mode, a simple queuing mechanism is sufficient to provide the above flexibility. However, EVEC must be able to correlate with past events.

4 Implementation of EVEC The EVEC script language is designed for testing and monitoring processes or business transactions of various kinds, and more particularly for analyzing and validating event patterns that are generated by these processes. To this extent,

370

Y. Lee

Fig. 1 Blocking and non-blocking modes of scriplet invocation

EVEC may be described as an event-processing language. The content and structure of these events may be quite diverse, but a common XML wrapper is assumed. The top-level constructs are the script package and the scriplet: – The script package, or ‘‘script’’: This is the main unit of execution. The script package contains an ‘‘execution context’’ (\ execution-context [ element) that defines various global constructs and bindings, e.g. for declaring event boards. The script package also contains one or more ‘‘scriplets’’. The execution context in a script package defines which scriplet to start execution with—or main scriplet. In case other scriplets are defined, the main scriplet is expected to invoke these directly or indirectly. – The scriplet: A scriplet defines a flow (or thread) of execution for a sequence of atomic operations. Scriplets can execute either concurrently or not (see detailed meaning in the next section), and can be started in a blocking or non-blocking way. EVEC is designed so that it leverages existing XML script languages for special features such as logical conditions and event selection. The default language for all logical expressions over XML content is XPath, along with its existing function libraries (e.g. advanced set of functions for time and duration management). The concept of concurrency in EVEC is entirely dependent on the notion of ‘‘virtual present time’’ (VPtime). When a scriplet starts to execute, it is assigned a VP-time which will condition its event consumption and timestamp its event production. The default VP-time assignments are:

Event-Centric Test Case Scripting Method

371

Fig. 2 Fully past backward blocking invocation

– The first scriplet of a script package is assigned the initial VP-time of this script, the default of which is in turn the actual present time (AP-time). – The VP-time of a scriplet S2 started by a scriplet S1, is the value of VP-time in S1 when [start S2] is executed. These default values can be overriden by the \ start [ operation, which allows to set the VP-time of the started scriplet (see the start/@vptset attribute in Sect. 4). Inside a scriplet, the VP-time may be progressed by two operations: – \wait [ : will add some predefined duration to the VP-time, or wait until some date, or yet until some other scriplets complete. – \catch [ : when waiting—virtually or really—for some event to occur, will advance the VP-time to the occurring date of events being caught. Event catching in a scriplet is only considering (by default) events occurring at or after the current VP-time. Besides \ wait [ and \ catch [, the execution duration of other EVEC operations is considered as negligible, as far as the VP-time is concerned: in other words, these operations do not affect the VP-time. The VP-time window of a scriplet execution is defined as the [starting VP-time, ending VP-time] time interval of the execution. Intuitively, concurrent execution is achieved when the VP-time windows of two scriplets overlap. Concurrency is entirely determined by the start VP-time assigned to these scriplets. When a scriplet S1 starts a scriplet S2, it can do so either in a blocking or non-blocking way:

372

Y. Lee

– Blocking invocation: intuitively, the invoking scriplet (S1) will wait until the invoked scriplet (S2) terminates. The next statement in the invoking scriplet S1 (after \ start [), will execute at a VP-time that is same as the VP-time set at the end of the invoked scriplet (S2). In other words, the VPtimes of S1 and S2 are ‘‘synchronized’’ after S2 completes (see start/@vptsync = ‘‘true’’ in Sect. 4). More generally, to accommodate the case where a starting time is set at a date/ time anterior to the invoking time (@vptset = ‘‘a past date/time’’), the VP-time of the next statement in S1 is either the last VP-time value of S2 or the last VPtime value in S1 (just before invoking S2), whichever occurs the latest. – Non-blocking invocation: intuitively, the invoked scriplet (S2) will not cause the invoking scriplet (S1) to wait. In other words, the VP-times of S1 and S2 are not ‘‘synchronized’’ after S2 completes. (see start/@vptsync = ‘‘false’’ in Sect. 4). The next statement in the invoking scriplet S1 will execute at a VP-time that is same as the VP-time value just before executing the \ start [ statement, this regardless of the value of start/@vptset. Non-blocking invocations should not be seen as only useful for actual concurrent (or multi-threaded) processing. In many cases, it makes scripting easier and more intuitive, even when execution is entirely deferred on past (logged) events that could otherwise be processed serially in a single-threaded way. Various cases of blocking and non-blocking invocations are illustrated below. The following figure illustrates both modes of scriplet invocations, and how the VP-time is affected—or not—in the invoking scriplet. When a scriplet S1 does a blocking invocation of a scriplet S2, the VP-time of the invoked scriplet S2 is initiated at the current VP-time of the invoking scriplet S1 (unless a different VPtime value is given using start/@vptset as illustrated in the next figures). The scriplet S1 is then ‘‘blocked’’ until the VP-time at the end of S2 is known and assigned as current VP-time in S1. In the non-blocking invocation (see Fig. 1), S1 will ignore the ending time of S2. A single-threaded execution may still execute S2 first before executing the next statement in S1. The execution semantics would still allow ‘‘concurrent’’ catching (in terms of virtual present) of same events by S1 and S2, as both will select events starting from VP-time t1. In the above figure, S1 is catching an event at time t1b while S2 is catching an event at time t2. Depending on their respective selection expressions, these catches could capture either the same or different event, causing the new VPtime in S1 (t1b) to be either prior or after the new VPtime in S2 (t2). In a single-threaded execution of the non-blocking case that started ‘‘live’’ (VPtime = present time), S2 could be executed first live, then the remaining part of S1 can be executed ‘‘deferred’’ on the log of events, starting from time t1 now in the past. Clearly, more fine-grain serialization of S1 and S2 executions would be required if these two scriplets communicate with each other, e.g. if S1 is consuming an event posted by S2 or vice versa (Fig. 2).

Event-Centric Test Case Scripting Method

373

5 Conclusion In this paper, we present an event-centric test case scripting method and execution model, EVEC, for such systems. EVEC enables testing and monitoring of applications and business processes, the behavior of which can be traced and monitored via events. The notion of event in EVEC is broad, thus, it could cover all of the type of SOA business processes. Further study is required to define and classify the detailed test case metadata or artifacts that would complement EVEC in test environments.

References 1. Leymann F (2001) Web services flow language, TR WSFL 1.0, IBM Software Group, May 2001 2. Thatte S (2001) XLANG Web services for business process design. http://www.gotdotnet.com/ team/xmlwsspecs/xlang-c/default.htm 3. Kammer P, Bolcer GA, Taylor RN, Bergman M (2000) Techniques for supporting dynamic and adaptive workflow. J Comput Support Coop Work 9:269–292 4. DAML-S specifications. http://www.daml.org/services/ 5. Kosanke K (1993) CIMOSA—open system architecture for CIM; ESPRIT consortium AMICE. Springer, Berlin 6. Biztalk. http://www.microsoft.com/biztalk/ 7. Business process execution language for Web services, version 1.0.http://www-106.ibm.com/ developerworks/webservices/library/ws-bpel/. Accessed July 2002 8. Cardoso J, Sheth A, Miller J (2002) Workflow quality of service 9. Sheth A, Cardoso J, Miller J, Koch K (2002) QoS for service-oriented middleware (2002) ‘‘web services and grid computing.’’ In: Proceedings of the Conference on Systemics, Cybernetics and Informatics, Orlando, FL, July 2002 10. Mani A, Nagarajan A, Understanding quality of service for Web services. http://herzberg.ca. sandia.gov/jess/ 11. OASIS Test Assertions Guidelines (TAG) TC, ‘‘test assertions guidelines.’’ http://www.oasisopen.org/committees/tag/

bQoS (Business QoS) Parameters for SOA Quality Rating Youngkon Lee

Abstract With Web services starting to be deployed within organizations and being offered as paid services across organizational boundaries, quality of service (QoS) has become one of the key issues to be addressed by providers and clients. While methods to describe and advertise QoS properties have been developed, the main outstanding issue remains how to implement a service that lives up to promised QoS properties. This paper provides the service level agreement (SLA) parameters for QoS management applied to Web services and raises a set of research issues that originate in the virtualization aspect of services and are specific to QoS management in a services environment—beyond what is addressed so far by work in the areas of distributed systems and performance management. Keywords SOA


Business context
SLA parameter

1 Introduction Whether offered within an organization or as a part of a paid service across organizational boundaries, quality-of-service (QoS) aspects of services are important in a service-oriented computing environment. Dealing with QoS is a sign of a technology going beyond its stage of initial experimentation to a production deployment and many recent activities related to QoS of Web services indicate that this is becoming an increasingly relevant topic.

Y. Lee (&) e-Business Department, Korea Polytechnic University, Jeongwangdong, Siheung 2121, Korea e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_36, Ó Springer Science+Business Media B.V. 2012

375

376

Y. Lee

Efforts in the past years mainly focused on describing, advertising and signing up to Web and Grid services at defined QoS levels. This includes HP’s Web Services Management Framework (WSMF) [1], IBM’s Web Service Level Agreement (WSLA) language [2, 3], the Web Services Offer Language (WSOL) [4] as well as approaches based on WS-Policy [5]. These efforts enable us to describe quality metrics of services, such as response time, and the associated service level objectives flexibly and in a way that is meaningful for the business needs of a service client. However, one of the challenging issues is to associate or derive a system configuration that delivers the QoS of a described Web service using the above mentioned approaches. In many cases, this is non-trivial. Sometimes we can rely on experience with tested, dedicated system configurations to decide, for example, the size of a cluster for a particular workload guaranteeing a particular response time for a given percentile of requests. In addition, managing a service at different QoS levels on the same infrastructure is not easy. While managing QoS in distributed systems is not a novel problem, a number of additional issues arise in the context of a service-oriented computing environment. Those issues arise from the specific properties of Web services. For example, cross-organizational Web services may be accessed through the public Internet and client side QoS metrics have to include network properties in addition to properties of the service-implementing application itself. In addition, composite and recursively aggregated services—and the ability to aggregate is seen as a key benefit of Web services—gain new QoS properties that are not always easily derived from their parts. The objective of paper is to analyze the main QoS factors of Service-oriented Architecture (SOA) in a business context and to provide the service level agreement (SLA) parameters that affect critically the business performance. According to OASIS Reference Model for Service Oriented Architecture [SOA-RM] [6], the Service Oriented Architecture (SOA) is a paradigm for organizing and utilizing distributed capabilities that may be under control of different ownership domains. The service within SOA is a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description. This specification further defines the business service level agreement (bSLA) between the service requester and the service provider for the service which is defined in SOA-RM, within the end-to-end resource planning (EERP) technology [7]. The applications of EERP are any kind of business services, and they are not limited to Web Services only. This applies the well-known technique for service discovery and optimization in a novel way to improve business results. It models the business process and the range of potential services, and then guides the selection and deployment of services based on the end-to-end business value. Modeling the business service-level agreements to manage and evaluate services and establishing agreements about the business service is essential to long-term value chain improvement. The bSLA is different from the SLA in the software/IT world. The bSLA is the contact between the service

bQoS (Business QoS) Parameters for SOA Quality Rating

377

Fig. 1 Points of measurements defining semantics of metrics

requester and the service provider, and the SLA is the contract between the service provider and the network/system provider. The SLA is network/system oriented agreement that deals with network performance and system availability. The bSLA is a business oriented agreement that deals with price, time to deliver, and the quality/rating of the service. In Sect. 2, we summarize the related works about web service selection based on the QoS metrics. Section 3 presents the service process model which enables service performance optimization in the business respect and Sect. 4 details the bSLA model including parties, parameters and obligations, and we conclude in Sect. 5.

2 Related Works The first step to manage Web service’s quality is to define it. While this is important for Web services as well as in traditional distributed systems, explicit definition is particularly important in an environment transcending organizational boundaries. Quality is expressed referring to observable parameters relating to a non-functional property, for example, the response time of a request. A level of quality is agreed upon as a constraint over those parameters, potentially dependent on a precondition. Hence, the party offering a Web service, in agreement with its customers and users, will define the QoS parameters and the particular instances of the service to which these parameters relate. In the case of a Web service, a parameter such as response time can relate to an individual invocation of an operation or a class of operations, all having the same (individual) quality properties of having an aggregate property, e.g., the average response time of this class of operations or another stochastic metric. A further step in managing Web services QoS is the definition of the semantics of the QoS parameters. A Web service and its subscribers and users must understand what it is meant. It is important what is measured where. For performance-oriented metrics this can be at different points, as Fig. 1 illustrates. The definition of QoS parameters corresponds to the establishment of ontology between a service provider and its clients. Ontology can be established in two approaches. (1) It can be a definition of terms and, potentially, or the semantics of the relationships between them, as facilitated by DAML and OIL [8]. This approach results in a fixed set of well understood terms—in our case the QoS parameters. (2) Another approach uses constructive ontology. Based on a set of well-know defined terms (as in 1) and a set of well-know composition operators,

378

Y. Lee

Fig. 2 Contracts defining the scope of quality guarantees

new terms’ (QoS) parameters can be defined by composing new parameters out of existing ones using the operators. Having established common understanding of quality of service parameters and the associated guarantees given by the provider, it also has to be established to which relationships between a client and a server a QoS guarantee applies. A service may provide the same quality to all requesting clients, to each client individually or to a defined set of clients that a provider organization and an organization requiring a QoS level for multiple clients agree upon in a contract, which is also called an SLA (Fig. 2). Clients will refer to the contract when requesting the service according to a particular quality level. The different scoping approaches of QoS guarantees require different means of establishing a particular quality level for a client: If a QoS level is associated with a service, a client searches for a suitable service in a directory, e.g., UDDI and retrieves its quality definition, e.g., stated as a WS-Policy expression. In the case of an individual client or a contract, a negotiation mechanism, which can be very simple, must be provided. Once the contract is established, the provider organization must provision a service-implementing system such that it behaves as it has been agreed upon. This involves deriving the amount of resources needed and the runtime management of resources. However, this is not simple. While we have developed—improvable— approaches to the issues raised above, the issue of provisioning and runtime managing a Web service-implementing system is not equally well understood yet. In the next section, we discuss what distributed systems and performance management approaches can provide. A number of performance management technologies, such as workload managers and network dispatchers, have been developed to control response times of individual systems and clusters and various availability management approaches. However, it is not straight-forward to configure, for example, workload managers to satisfy response time goals for a set of different scopes of quality—for Web services as well as for any distributed system. In this section, we outline some typical approaches of how today’s QoS parameters are managed in distributed systems.

3 Service Process Model This section describes the service process model conceptually. Figure 3 shows the conceptual model, and of messages flows with brief descriptions. We also include timeline and sequence diagrams Fig. 4 to show how an implementation would use

bQoS (Business QoS) Parameters for SOA Quality Rating

379

Fig. 3 Service process model

Fig. 4 Service message sequence without optional messages

service messages and build a continuous business process improvement loop. In Fig. 3, the business quality of Service is abbreviated as bQoS, business rating is abbreviated as Rating, and business service level agreement is abbreviated as bSLA.

380

Y. Lee sla:SLAPartiesTypeType sla:SLAParametersType sla:SLAObligationsType ? sla:SLATermsType ?

Fig. 5 XML schema for bSLA

The service requester is the client system who tries to find an optimal solution provided by service optimization portal (SOP). Service providers provide business services. Each service provider may offer the same service but with different bQoS and Ratings. Services may be running on different platforms with different implementations, but they all support message exchanges of bQoS, Rating, and BSLA information in the XML formats. The SOP accepts the request from the Service requester, performs bQoS and rating queries, calculates optimal solution(s), and then returns the result to the service requester. The Rating Provider is a party unaffiliated with either the requester or the target of the rating request, such as a third party rating organization, given a reference to a particular business service and provider, issues either a number or a classification description. There can be another way to implement the service optimization without the SOP. For example, there can be a case for some services providers and service consumers using SOA Registry-Repository to find each other, to exchange business quality of services information among them, and to begin negotiations for establishing Service Level Agreements (SLAs). The results of messages 2 through 9 in Fig. 4 are used to calculate the optimal deployment for a given set of services requests. A list of alternatives might be returned in message 10. Each step in the process would have a service provider determined for each service and for each alternative. Messages 11 and 12 are exchanged between the service requester and the selected service providers to define the BSLA.

4 bSLA Model The bSLA model is for representing SLA parameters in the respect of business value. The BSLA is the root element for EERP–Business Service-level agreement (bSLA). The bSLA is a formal contract between a service provider and a client guaranteeing quantifiable business quality of service (bQoS) at defined levels. It can have one or more of the following elements (Fig. 5): The following describes the attributes and elements listed in the schema outlined above: – /sla:BSLA is a root element of Business Service-level agreement (bSLA) for EERP. – SLAParties is a required element in bSLA that defines parties invoked in this bSLA for the service. SLAParties element has both the service provider and services requester elements.

bQoS (Business QoS) Parameters for SOA Quality Rating

381

sla:ServiceProviderType sla:SlaUriType sla:ServiceProviderNameType sla:ServiceRequesterType sla:SlaUriType sla:ServiceRequesterNameType ...

Fig. 6 XML schema for bSLA parties

– /sla:BSLA/sla:SLAParties/@{any} is an extensibility mechanism to allow additional attributes, based on schemas, to be added to the SLAParties element in the future. Unrecognized attributes may cause a fault or be silently ignored. – /sla:BSLA/sla:SLAParameters is defined monitoring of bQoS metrics, including service profile uri, operations and other optional elements. It is a required element that uses sla:SLAParametersType. – /sla:BSLA/sla:SLAParameters/@{any} is an extensibility mechanism to allow additional attributes, based on schemas, to be added to the SLAParameters element in the future. – /sla:BSLA/sla:SLAObligations is agreed bSLA obligations aspect of the service, including obligations, action guarantees. It is a optional element that uses sla:SLAObligationsType. – /sla:BSLA/sla:SLAObligations/@{any} is an extensibility mechanism to allow additional attributes, based on schemas, to be added to the SLA Obligations element in the future. – /sla:BSLA/sla:SLATerms is agreed bSLA terms aspect of the service, including bSLA term elements. – /sla:BSLA/sla:SLATerms/@{any} is an extensibility mechanism to allow additional attributes, based on schemas, to be added to the SLATerms element in the future. – /sla:BSLA/@{any} is an extensibility mechanism to allow additional attributes, based on schemas, to be added to the root BSLA element in the future. – /sla:BSLA/sla:BSLAExtension is an optional element that keeps different (extensible) elements to be specified in the future. – /sla:BSLA/sla:BSLAExtension/{any} is an extensibility mechanism to allow different (extensible) elements to be specified in the future. The SLAParties describes the list of parties invoked in the bSLA for the service. There should be one SLAParties element present in the bSLA of service. The following describes the attributes and elements listed in the schema outlined above:

382

Y. Lee

– /sla:SLAParties, bSLA Parties aspect of the service, is for parties invoked in the bSLA for the service, including both service provider and service requester elements. – /sla:SLAParties/sla:ServiceProvider represents the provider for parties. It is a required element for bSLA Parties. – /sla:SLAParties/sla:ServiceProvider/sla:ServiceUri is a required element for Service Provider. – /sla:SLAParties/sla:ServiceProvider/sla:ServiceProviderName is the name of the service provider. It is also a required element for Service Provider. – /sla:SLAParties/sla:ServiceProvider/sla:ServiceProviderName/@languageID is an optional attribute in the ServiceProviderName element, using xsd:language type (Fig. 6). – /sla:SLAParties/sla:ServiceProvider/@{any} is an extensibility mechanism to allow additional attributes, based on schemas, to be added to ServiceProvider element in the future./sla:SLAParties/sla:ServiceRequester represents requester for the service, including requester’s name and the URI that represents the requester. It is a required element for bSLA Parties. – /sla:SLAParties/sla:ServiceRequester/sla:ServiceRequesterUri represents the requester’s identifier in URI format for the service requester. It is a required element for Service Requester. – /sla:SLAParties/sla:ServiceRequester/sla:ServiceRequesterName is requester’s name for the service requester. It is a required element for Service Requester. – /sla:SLAParties/sla:ServiceRequester/sla:ServiceRequesterName/@languageID is an optional attribute in the ServiceRequesterName element. – /sla:SLAParties/sla:ServiceRequester/@{any} is an extensibility mechanism to allow additional attributes, based on schemas, to be added to the serviceRequester element in the future. – /sla:SLAParties/{any} is an extensibility mechanism to allow different (extensible) elements to be specified in the future.

5 Conclusion In this paper, we proposed a new concept of bSLA, whose items are proper to evaluate SOA service performance in the respect of service business benefit. The bSLA includes the service actor information, bSLA parties and the quality information, bSLA parameters, and bSLA obligations of the service parties. We also devised a service optimization portal which provides the best service composition by evaluating the value of bQoS of each service chain. Further study is required to define and classify the quality factor group for business case by case.

bQoS (Business QoS) Parameters for SOA Quality Rating

383

References 1. Catania N, Kumar P, Murray B, Pourhedari H, Vambenepe W, Wurster K (2003) Web services management framework, version 2.0, Hewlett-Packard. http://devresource.hp.com/drc/ specifications/wsmf/WSMF-WSM.jsp. Accessed 16 July 2003 2. Ludwig H, Keller A, Dan A, King R, Franck R (2003) A service level agreement language for dynamic electronic services, vol 3. Electronic Commerce Research Kluwer Academic Publisher, Norwell, pp 43–59 3. Ludwig H, Keller A, Dan A, King R, Franck R (2003) Web service level agreement (WSLA) language specification, version 1.0, IBM corporation. http://www.research.ibm.com/wsla/ WSLASpecV1-20030128.pdf. Accessed 28 Jan 2003 4. Tosic V, Pagurek B, Patel K (2003) WSOL—a language for the formal specification of classes of service for web services. In: Proceedings of ICWS’03 (The 2003 international conference on web services), CSREA Press, Las Vegas, USA, pp 375–381, 23–26 June 2003 5. Box D, Curbera F, Hondo M, Kale C, Langworthy D,. Nadalin A, Nagaratnam N, Nottingham M, von Riegen C, Shewchuk J (2003) Web services policy framework (WSPolicy). http:// www.ibm.com/developer-works/library/ws-policy. Accessed 28 May 2003 6. Mackenzie CM et al (2006) Reference model for service oriented architecture 1.0. OASIS committee draft, 2006.8 7. William C et al (2010) SOA-EERP business quality of service version, OASIS committee draft, 2010.11 8. Connolly D, van Harmelen F, Horrocks I, McGuinness DL, Patel-Schneider PF, Stein LA (2001) DAML ? OIL (March 2001) reference description, W3C. http://www.w3.org/TR/ daml+oil-reference. Accessed 18 Dec 2001

Business-Centric Test Assertion Model for SOA Youngkon Lee

Abstract This paper presents a design method for business-centric SOA test framework. The reference architecture of SOA system is usually layered: business process layer, service layer, and computing resource layer. In the architecture, there are so many subsystems affecting the system’s performance, which relates with each other. As a result, in respect of overall performance, it is meaningless to measure each subsystem’s performance separately. In SOA system, the performance of the business process layer with which users keep in contact usually depends on the summation of the performance of the other lower layers. Therefore, for testing SOA system, test cases describing business process activities should be prepared. We devised a business-centric SOA test assertion model which enables to semi-automatic transform test assertions into test cases by the concept of prescription level and normalized prerequisite definition. The model also minimizes the semantic distortion in the transformation process. Keywords SOA


Business process
Test assertions
Test cases

1 Introduction Service Oriented Architecture (SOA) is generally defined as a business-centric IT architectural approach that supports integrating businesses as linked, repeatable business tasks, or services. SOA enables to solve integration complexity problem and facilitates broad-scale interoperability and unlimited collaboration across the

Y. Lee (&) e-Business Department, Korea Polytechnic University, 2121, Jeongwangdong, Siheung city, Korea e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_37, Ó Springer Science+Business Media B.V. 2012

385

386

Y. Lee

enterprise. It also provides flexibility and agility to address changing business requirements in lower cost and lesser time to market via reuse. SOA has a lot of promises of interoperability, however, at the cost of: lack of enterprise scale QoS, complex standards which are still forming, lack of tools and framework to support standards, and perform penalty. Recently, as SOA has been widely adopted in business system framework, performance issues in SOA are raised continuously from users and developers. SOA system is generally composed of various subsystems, each of which relates intimately with others. Therefore, if performances are issued, it is very difficult to find out the reason clearly. For example, if a business process in SOA system has longer response time than before, there could be various reasons: cache overflow in a business processor, wrapping overhead in service interface, or exceptions in computing resources, etc. One thing clear is that the performance of business process layer depends on the lower layer and measuring the performance of business layer includes indirectly measuring the performance of all the lower layers. But, most of the test frameworks developed focus on measuring SOA messaging performance, as we present in Chap. 2. They almost adopt batch-style testing where all the test cases are executed in a sequence. OMG recommended a standard SOA reference model, MDA (Model Driven Architecture) [1]. It is widely adopted in real world because it is normative and enables SOA system to be implemented in a business-centric approach. In the MDA, a business process is designed firstly in a way for satisfying business requirements and later services are bounded to the activities in the business process. Business processes are described in a standardized language (e.g. WSBPEL) and they are executed generally on a business process management (BPM) system. For testing SOA systems implemented according to the MDA reference model in business-centric way, test harness should have business process simulation functionality so that it can behave as BPM and test overall performance at the same time. This means that the test harness can execute business process, perform tests, and gather metric values. The performance of the business process layer with which users keep in contact usually depends on the summation of the performance of the other lower layers. Therefore, for testing SOA system, test cases describing business process activities should be prepared. In SOA system, test assertions may help develop tighter test cases which could be used as an input for SOA test harness. Any ambiguities, contradictions and statements which require excessive resources for testing can be noted as they become apparent during test assertion creation. Test assertions should be reviewed and approved to improve both the quality and time-to-deployment of the test cases. Therefore, best results are achieved when assertions are developed in parallel with the test cases. Test assertions provide a starting point for writing conformance test cases or interoperability test cases. They simplify the distribution of the test development effort between different organizations while maintaining consistent test quality. Test assertions improve confidence in the resulting test and provide a basis for coverage analysis.

Business-Centric Test Assertion Model for SOA

387

In Sect. 2, we present some related works. Section 3 provides the concept of test assertion. In Sect. 4, we describe a test assertion model. Section 5 presents cases of the test assertion and Sect. 6 shows complex predicates of test assertions. Conclusions are presented in the last section.

2 Related Works This section presents some test frameworks and script languages developed or proposed for SOA system. Web Services Quality Management System This system has been developed by NIA in order to measure Web services’ quality on the criteria of WSQM (Web Services Quality Model) quality factors [2]: interoperability, security, manageability, performance, business processing capability, and business process quality. This system contributes to consolidate the quality factors of SOA. However, it requires expanding its architecture to apply SOA, system, because it targets only to Web services system. ebXML Test Framework This framework has been implemented by NIST and KorBIT for testing ebXML system according to OASIS IIC Specification [3]. It could test packaging, security, reliability, and transport protocol of ebXML messaging system implemented by ebMS specification [4]. The main purpose of this framework is to test conformance and interoperability of ebXML messaging system, and it is not proper for testing service oriented systems. Besides, it cannot test ad hoc status resulting from various events, because it is not event-driven but batch-style test framework. JXUnit and JXU JXUnit [5] and JXU [6] is a general scripting system (XML based) for defining test suites and test cases aimed at general e-business application testing. Test steps are written as Java classes. There is neither built-in support for business process test nor support for the event-driven features. However, as a general test scripting platform that relies on a common programming language, this system could be used as an implementation platform for general e-business tests. Automatic Test Mark-up Language (ATML) In its requirements, ATML provides XML Schemata and support information that allows the exchange of diagnostic information between conforming software components applications [7]. The overall goal is to support loosely coupled open architectures that permit the use of advanced diagnostic reasoning and analytical applications. The objective of ATML is focusing on the representation and transfer of test artifacts: diagnostics, test configuration, test description, instruments, etc.

388

Y. Lee

Test Choreography Languages These are standards for specifying the orchestration of business processes and/ or transactional collaborations between partners. Although a markup like XPDL [8] is very complete from a process definition and control viewpoint, it is lacking the event-centric design and event correlation/querying capability required by testing and monitoring exchanges. Also, a design choice has been here to use a very restricted set of control primitives, easy to implement and validate, sufficient for test cases of modest size. Other languages or mark-ups define somehow choreographies of messages and their properties: ebBP, WS-BPEL, WS-Choreography [9]. The general focus of these dialects is either the operational aspect of driving business process or business transactions, and/or the contractual aspect, but not monitoring and validation. Although they may express detailed conformance requirements, they fall short of covering the various aspects of an exhaustive conformance check, e.g. the generation of intentional errors or simulation of uncommon behaviors. In addition, the focus of these languages is mainly on one layer of the choreography—they for instance, ignore lower-level message exchanges entailed by quality of service concerns such as reliability, or binding patterns with the transport layer.

3 Concept of Test Assertion A test assertion is a testable or measurable expression for evaluating the adherence of an implementation (or part of it) to a normative statement in a specification. A set of test assertions may be associated with a conformance clause in order to define more precisely what conformance entails. Test assertions lie between the specification and any suite of tests to be conducted to determine conformance (See Fig. 1). Such a test suite is typically comprised of a set of test cases. These test cases may be derived from test assertions which address the normative statements of the specification. Judging whether the test assertion is testable may require some knowledge about testing capabilities and resource constraints. Sometimes there is little knowledge of what actual testing conditions will be. In such cases the prime objective of writing test assertions is to provide a better understanding of what is expected from implementations, in order to fulfill the requirements. In other cases, the test assertions are designed to reflect a more precise knowledge of testing conditions. Such test assertions can more easily be used as a blueprint for test suites.

4 Test Assertion Model This section aims to cover the simpler aspects of test assertions. Some more complex aspects are covered later in this section. Figure 2 below shows the anatomy of a typical test assertion, and how its parts relate to the specification

Business-Centric Test Assertion Model for SOA

389

Fig. 1 Role of test assertion

being addressed, as well as to the implementations under test. Some optional parts are not shown in the figure. Some of the elements which comprise a test assertion are considered core while others are optional. A test assertion includes, implicitly or explicitly: Identifier A unique identifier of the test assertion facilitates tools development and the mapping of assertions to specification statements. It is recommended that the identifier be made universally unique. Normative Sources These refer to the precise specification requirements or normative statements that the test assertion addresses. Target The target categorizes an implementation or a part of an implementation of the referred specification, which is the main object of the test assertion and of its Normative Sources. Predicate A predicate asserts, in the form of an expression, the feature (a behavior or a property) described in the specification statement(s) referred by the Normative Sources. If the predicate is an expression which evaluates to ‘‘true’’ over the test assertion target, this means that the target exhibits this feature. ‘‘False’’ means the target does not exhibit this feature. In addition, a test assertion may optionally include following components. Description This is an informal definition of the role of the test assertion with some optional details on some of its parts. This description must not alter the general meaning of the test assertion and its parts. This description may be used to annotate the test assertion with any information useful to its understanding. It does not need to be an exhaustive description of the test assertion.

390

Y. Lee

Fig. 2 General anatomy of a test assertion

Prescription Level This is a keyword that indicates how imperative it is that the Normative Statement referred to in the Normative Source, be met. See possible keyword values in the Glossary. Prerequisite A test assertion Prerequisite is a logical expression (similar to a Predicate) which further qualifies the Target for undergoing the core test (expressed by the Predicate) that addresses the Normative Statement. It may include references to the outcome of other test assertions. If the Prerequisite evaluates to ‘‘false’’ then the Target instance is not qualified for evaluation by the Predicate. Tags Test assertions may be assigned ‘tags’ or ‘keywords’, which may in turn be given values. These tags provide you with an opportunity to categorize the test assertions. They enable you to group the test assertions; based on the type of test they assume or based on their target properties. Variables Test assertions may also include variables for convenience in storing values for reuse and shared use, as well as for parameterization. As a test assertion has parts that can be evaluated over a Target instance (i.e. the Prerequisite and the Predicate), the following semantics apply to a test assertion: – ‘‘Target not qualified’’: if the Prerequisite (if any) evaluates to ‘‘false’’ over a Target instance. – ‘‘Normative statement fulfilled [by the Target]’’: if the Prerequisite (if any) evaluates to ‘‘true’’ over a Target instance, and the Predicate evaluates to ‘‘true’’.

Business-Centric Test Assertion Model for SOA

391

– ‘‘Normative statement not fulfilled [by the Target]’’: if the Prerequisite (if any) evaluates to ‘‘true’’ over a Target instance, and the Predicate evaluates to ‘‘false’’.

5 Case Study of Test Assertion Consider the following statement in the widget specification: [requirement 101] ‘‘A widget of medium size MUST use exactly one AA battery encased in a battery holder.’’ There are actually two requirements here that can be tested separately: (requirement 101, part 1) A medium-size widget MUST use exactly one AA battery. (requirement 101, part 2) A medium-size widget MUST have a battery holder encasing the battery. Because of this it is possible to write two test assertions: – TA id: widget-TA101-1a Normative Source: specification requirement 101, part 1 Target: medium-size widget Predicate: [the widget] uses exactly one AA battery. Prescription Level: mandatory and – TA id: widget-TA101-1b Normative Source: specification requirement 101, part 2 Target: medium-size widget Predicate: [the widget] has a battery holder encasing the battery. Prescription Level: mandatory The granularity of a test assertion is a matter of judgment. A single test assertion instead of two can be written here, with the predicate: ‘‘[the widget] uses exactly one AA battery AND has a battery holder, encasing the battery’’. This choice may later have an impact on the outcome of a test suite written to verify the conformance of widgets. With a single test assertion, a test case derived from this test assertion will not be expected to distinguish between the two failure cases. Using two test assertions—one for each sub-requirement—will ensure that a test suite can assess and report independently about the fulfillment of each subrequirement. Other considerations such as the different nature of tests implied or the reuse of a test assertion in different conformance profiles [VAR], may also lead to the adoption of ‘‘fine-grained’’ instead of ‘‘coarse-grained’’ test assertions. Usage considerations will dictate the best choice.

392

Y. Lee

6 Complex Predicates Recall the previous example of [requirement 101]. The target can be defined as ‘‘a medium-size widget’’ or as just ‘‘a widget’’. The latter is a natural decision if the specification requirement uses the wording: ‘‘[requirement 101] If a widget is medium size, then it MUST use exactly one AA battery and be encased in a battery holder.’’ For the simplicity of this example, if the two test assertion predicates for widget-TA101-1a and widget-TA101-1b are combined into one example, one possible outcome is: TA id: widget-TA101-2a Normative Source: requirement 101 Target: widget Predicate: if [the widget] is medium-size, then [the widget] uses exactly one AA battery AND the battery is encased in a battery holder. Prescription Level: mandatory The target category is broad, but the predicate part is really of interest only for a subset of this category (the medium-size widgets). Usage considerations should again drive the decision here: a test suite that is designed to verify all widgets, and does not assume a prior categorization of these into small/medium/large sizes, would be improved with test assertions that only use ‘‘widget’’ as the target, such as widget-TA101-2a. A test assertion predicate may, then, be a Boolean expression—a composition of atomic predicates using logical operators AND, OR, NOT. A test assertion predicate may also be of the kind: ‘‘if (condition) then (expression)’’. The predicate is worded in an abstract way, still close to the wording of the specification. No indication of what kind of test procedure will be used, such as how to determine the number and type of batteries, is given. Detailed criteria for the condition evaluation, such as what kind of battery holder is acceptable, are also not provided. These details are normally left to the test cases that can be derived from the test assertions. These test cases will determine the precise criteria for conforming to the specification. However, if a precise criterion for interpreting the battery holder requirement is provided in an external specification, either referred to directly by the widget specification or by a related conformance clause, then a test assertion must use this criterion in its predicate. Such a test assertion must then refer not only to the specification requirement in its reference property, but also to the external specification or to the conformance clause that refers to this specification. Another case where a predicate is more complex is when its conditional expression involves more than one part of an implementation (or implementations). In some cases it is clear which one of these objects must be considered the target, while others are just accessory objects. Consider the following predicate: ‘‘the [widget price tag] matches the price assigned to the widget in its [catalog entry]’’, where price tags and catalog entries are both items that must follow the store policy (in effect the specification). In this case it may be reasonably assumed that the ‘‘catalog’’ content is authoritative over the price tag. The price tag can then

Business-Centric Test Assertion Model for SOA

393

be considered as the test target, while the accessory object may be identified by a variable which is then used in the predicate.

7 Conclusion We presented a SOA test assertion model, which facilitates to make test cases in normalized form. In the model, we devised the concept of prescription level and normalized prerequisite for preparing test cases. By the concepts, test assertion can be transformed into test cases without semantic distortion. The model minimizes the human intervention in preparing test cases by automating some processes for translating test assertions into test cases. We showed two cases of complex predicates. Further studies are required to develop a test framework to check out automatically that test cases are conformed to test assertions.

References 1. Miller J, Jishnu M (2003) MDA Guide Version 1.0.1.http://www.omg.org/docs/omg/03-0601.pdf. OMG, June 2. Lee Y et al (2008) Web Services Quality Model 1.1.OASIS WSQM TC, October 3. Durand J et al (2004) ebXML test framework v1.0. OASIS IIC TC, October 4. Peter W et al (2007) ebXML messaging services 3.0, OASIS ebMS TC, July 5. Java XML Unit (JXUnit). http://jxunit.sourceforge.net 6. JUnit, Java for Unit Test. http://junit.sourceforge.net 7. ATML (2006) Standard for automatic test markup language (ATML) for exchanging automatic test equipment and test information via XML. IEEE, Dec 8. XPDL (2005) XML process definition language (workflow management coalition) document number WFMC-TC-1025: version 1.14, 3 October 2005 9. OASIS (2006) Business process specification schema 1.0.1, May 2001 and ebBP, v2.0.4, October

A Classification of Model-Driven Approaches to Develop Semantic Web Service Alaeddin Kalantari, Suhaimi Ibrahim and Hamed Taherdoost

Abstract Nowadays, the number of infrastructure strategies based on Serviceoriented architecture (SOA) is growing and demands for reusing and composing available Web services have increased significantly. Therefore, a rich Semantic Web service description is required to reach an automated Web service discovery, composition, and invocation. Due to the complexity of the Semantic Web service languages as well as lack of enough knowledge in the field of knowledge engineering and knowledge representation among developers, the adoption of Semantic Web service is yet slow down. There are several Model-driven architecture (MDA) based approaches to tackle that issue. In order to evaluate the model-driven approaches which are used to develop Semantic Web services, it is vital to have a proper classification of these approaches. The aim of this paper is to classify several model-driven approaches for developing semantic Web services into two main categories namely software methodology and UML formalist based approaches. The results of the paper give an essential perspective to do research work on Model-driven approaches for Semantic Web service development.







Keywords Web service Semantic web service Model driven architecture (MDA) Software methodology-based UML formalist-based







A. Kalantari (&) Faculty of Computer Science and Information Systems, Universiti Teknologi Malaysia (UTM), Kuala Lumpur, Malaysia e-mail: [email protected] S. Ibrahim Advanced Informatics School (AIS), Universiti Teknologi Malaysia (UTM), Kuala Lumpur, Malaysia e-mail: [email protected] H. Taherdoost Faculty of Computer Science, Islamic Azad University, Semnan Branch, Tehran, Iran e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_38, Ó Springer Science+Business Media B.V. 2012

395

396

A. Kalantari et al.

1 Introduction Presently, the number of available Web services based on renowned standards such as Simple Object Access Protocol (SOAP), Universal Description, Discovery and Integration (UDDI), and Web Service Description Language (WSDL) are increasing. Thus, organizations are embarking to use infrastructure strategy based on Service Oriented Architecture (SOA) in a platform-independent manner to facilitate interoperability, reusability, and flexibility among B2B enterprise applications [1]. Regardless of the benefit of SOA, discovery and integration of Web services are not fully automated because, each service in the Internet may have several different descriptions [1, 2]. One of the most possible solutions to overcome this matter is using Semantic Web for describing Web services in unambiguous and machine-interpretable fashion. Some proposals for Semantic Web services have been submitted in W3C such as OWL-S [3], WSMO [4], and SAWSDL [5] to facilitate the automation of Web service tasks such as discovery, selection, composition, and execution. Indeed, writing Semantic Web service descriptions through aforementioned submissions are not easy for developers and need additional knowledge in the field of knowledge engineering and knowledge representation. Although, several tools and editors such as OWL-S Editors, WSMO studio [6], and WSMOViz [7] have been proposed to facilitate writing Semantic description, developers have to learn the syntax and concepts of the Semantic Web service languages and how to write them. This lack of knowledge and also the complexity of these languages cause the adoption of Semantic Web services slow down [8]. In order to overcome this problem, several Model Driven Architecture (MDA) [9] based approaches have been proposed to generate Semantic description of Web services from a set of models. MDA focuses on modeling instead of coding and is based on several OMG standards such as Meta-Object Facility (MOF), Unified Modeling Language (UML) [10], Common Warehouse Meta-model (CWM), and Meta-Data Interchange (XMI). MDA defines an architecture based on computation independent models (CIMs), platform independent models (PIMs), platform specific models (PSMs) and model transformation. The transformation model is written based on a model transformation language such as eXtensible Stylesheet Language Transformation (XSLT) [11], Atlas Transformation Language (ATL) [12]. In this paper, the state-of-the-art MDA-based Semantic Web service description approaches are classified into two categories: (a.) software methodology based approaches that propose the creation of Semantic annotations in the way of software engineering methodology and specify Semantic descriptions during the Web services development lifecycle and (b.) UML formalist based approaches that only extract Semantic description of Web services from a set of diagrams provided by UML tools. The rest of this paper is organized as follows: Sect. 2 offers a survey of the most important model-driven approaches for providing Semantic Web service which is

A Classification of Model-Driven Approaches to Develop Semantic Web Service

397

MDA-based semantic Web service description approaches

Software Methodology base

Torres et al (2006)

Brambila et al (2007)

MIDAS-S (2006, 2010)

UML formalist base

Gronmo et al (2005)

Yang and Chung (2006)

Lautebacher and Bauer (2007)

Timm and Gannod (2008)

Bensaber and Malki (2008)

Sun et al (2009)

Kim and Lee (2009)

Belouadha et al (2010)

Fig. 1 Taxonomy of MDA-based semantic web service description approaches

followed in Sect. 3, by providing a summary of these approaches into the appreciable table. Finally, Sect. 4 concludes the paper.

2 Classification of MDA-Based Approaches This section will present state-of-the-art MDA based approaches that provide Semantic Web service descriptions. These approaches are classified into two categories namely: software methodology-based and UML formalist–based. Fig. 1 shows the taxonomy of MDA-base Semantic Web service description approaches.

2.1 Software Methodology-Based Software methodology-based approaches are methodological solution based on the Software Engineering disciplines. In these approaches, the Semantic annotations are created during the service developments using concrete methodology which is able to combine the best practices in Semantic specification and Web service development. Torres et al. 2006. Torres et al. [13] approach extends a Web Engineering method such as Object Oriented Web Solution (OOWS) with an ontology model to specify processes and data that are published for external usage. In addition, an OOWS-based ontology model is created by using the application of a set of transaction rules. This approach specifies the functionalities and operations of the

398

A. Kalantari et al.

services using a set of structural and behavioral models such as class, state-chart, and activity diagrams. The service profile is not supported and should be created manually. In order to model OWL and OWL-S based constructs, a profile with state-chart and activity diagrams along with the domain ontology representation that uses UML class diagram are specified. Brambila et al. 2007. Brambila et al. approach [14] uses Business Process Model and Notation (BPMN) [15] and Web Modeling Language (WebML) [16] to develop WSMO-based Semantic Web services in the way of model-driven methodology. The main aim behind this approach is to alleviate the difficulty of Semantic Web service description using current Semantic and Web application technologies. In addition, Semantic descriptions are extracted from application design models via a semi-automated generator. This approach adopts WSMX as the Semantic execution environment. Indeed, all diagrams of BPMN and WebML are serialized in an XML serialization such as XMI. Then, this approach uses XSLT to map models into WSMO-based descriptions. Furthermore, CASE tool WebRatio [17] converts WebML skeletons into codes before they are being executed by WSMX at run-time. MIDAS-S (2006, 2010). MIDAS-S approach [18, 19] is based on a modeldriven software methodology for developing Web Information System (WIS) like MIDAS [20] framework. The Semantic aspect is added into the PIM and PSM levels of model to develop Semantic Web service in a MDA manner in conjunction with WSML specification using UML models. Hence, other aspects of this approach such as hypertext, content, and behavior are able to integrate with the development of Semantic Web service. The four main concepts of WSMO such as ontologies, goals, mediators, and Web services for generating the WSML description are provided into the PSM level. In this approach Goal, Web service, and ontology model are divided into two different models namely content and context models. The ontology context model collects information about namespaces, used mediators, and imported ontologies and enlarges the package diagram of UML while the ontology content model is designed for enlarging the UML class diagram and representing other elements such as concepts, attributes, axioms. The Web service context and content models are defined similar to the aforementioned ontology models in the scope of Web service with the difference that the content models represent the elements of Web service such as capabilities, interfaces, and the like.

2.2 UML Formalist-Based UML formalist-based approaches extract the Semantic description of Web services from a set of diagrams provided by UML tools. In fact, these approaches keep the elaboration of Semantic descriptions separate from the actual development of the underlying services [14].

A Classification of Model-Driven Approaches to Develop Semantic Web Service

399

Gronmo et al. 2005. Gronmo et al. approach [21] provides a UML profile that supports OWL-S, WSMO, and able to design an independent service model. In addition to the UML profile, a transformation mechanism is provided to transfer OWL-S into UML diagrams and vice versa using UML Model Transformation Tool (UMT). This approach uses XSLT script to transfer one model to another model. In addition, it takes advantage of OCL in UML models to represent the conditions. In this approach, matchmaking algorithms support Semantic matching of categories, input, and output. This approach consists of three main steps: modeling the composition, discovery, and selection. In the first step, the composition model is designed without considering particular Web services. Next, from the available service registry, the discover process is handled using a matchmaking algorithm. It uses the semantically described composition that is converted from composition design in the first step. In this step, the selected services are defined semantically based on OWL-S and they should be converted to UML diagrams using an automated tool. Finally, the appropriate services are selected to perform desired tasks in a UML model. Once the Semantic composite model of Web services is provided, it is transformed to OWL-S description. Yang and Chung 2006. This approach [22] generates OWL-S service model ontology using UML class and state-chart diagram. The focus of this approach is only the OWL-S service model ontology that describes the information about service interoperation. The process of generating service model ontology is divided into two sub-processes: generating atomic services with extracting information from UML class diagram and generating information that are extracted from statechart diagrams for composite services. In addition, a set of mapping rules are provided to carry out the mapping between UML diagrams and OWL-S. This approach takes advantage of GUL for representing conditions in UML model instead of OCL. Moreover, SWRL is used for representing conditions in OWL-S specifications. SWRL atom name is described using stereotyped class and SWRL atom type is represented using stereotyped dependency. This approach uses XSLT as transformation language for converting XMI to OWL-S service model ontology. Lautebacher and Bauer 2007. This approach [23] presents a meta-model and UML-profile for Semantic Web services. The meta-model is provided based on Ontology Definition Meta-model (ODM) to support OWL-S, WSMO, WSDL-S, and SWSF. In addition, a set of informal transformation rules along with the openArchitectureWare-language Xpand [24] are used to extract codes from the meta-model. The meta-model in this approach consists of five packages that interact with each other. All concepts of ontology are represented in ontology package similar to ODM. The interface package represents WSDL service elements model and its Semantic description. The non-functional descriptions of one or more Semantic Web services are modeled in ServiceProvider package. The functional package includes all aspects to annotate every single step. ProcessFlow package merges the extended concepts that are defined in ServiceProvider package. In this approach, Semantic Web service grounding and WSDL documents are modeled using UML diagrams. Practically, UML class diagrams are used to represent non-Semantic and WSDL-based Web services and UML activity

400

A. Kalantari et al.

diagrams model Web service composition. Output and Input pins are used to achieve data binding between processes. Moreover, constraints are used to model conditions on control constructs. Timm and Gannod 2008. Timm and Gannod approach [8] provides a modeldriven architecture Semantic Web service development for OWL-S based Semantic using UML class and activity diagrams along with their UML profiles for OWL-S. The approach is able to use WSDL and Semantic based Web services. The workflow of this approach consists of four steps: modeling, conversion, grounding, and execution. The UML class diagram is used to model the structure of the service and the UML activity diagram aids the modeling of composition. This approach uses OCL for representing conditions in PIM and takes advantage of XSLT scripts for transformation. In addition, an automated SPesification and EXecution (SPEX) tool is provided to load any number of WSDL documents and facilitate description, grounding, and execution. Bensaber and Malki 2008. This approach [25] first, convert WSDL document into the UML profile model in the reverse engineering manner to gain high level graphical models and then, uses imported ontologies to define UML model semantically. Finally, the UML models are converted to the OWL-S description. This approach divided into three main steps namely, Reverse Engineering Process, Annotation Process, and Conversion Tool respectively. The automatic conversion from WSDL to UML is provided via implemented conversion rules. The UML profile is used to define stereotypes and tagged values. The approach takes advantage of OCL for specifying effects, pre and post conditions in UML modeling. Finally, the OWL-S ontology is obtained from an XMI document generated from UML model. In this approach the access URIs and bindings are transformed from WSDL documents to UML activity diagrams before the grounding. Therefore, the grounding process is performed automatically. Sun et al. 2009. Sun et al. approach [2] is based on Gronmo et al. [26] and Bensaber and Malki [25] and uses existing Web services available in the internet to create composite Web service. The purpose of this approach is to increase the degree of automation for service composition. This approach includes five steps: Discovery Web Service, Reverse Engineering, Model Composite Web Service, Transform Composite Web service, and Publish Composite Web service. The Discovery Web Service is used to find relevant Web services from the registries and extract the equivalent WSDL. The output of this step is WSDL documents. The Reverse Engineering provides two UML diagrams like Class diagram for service model and Activity diagram for workflow model. In Model Composite Web Service, a UML tool is used to integrate imported ontologies, extracted UML diagrams, and information from UML to create a model of composite Web service. The obtained UML model is transformed into XMI document through an adequate mapping rules and UML profiles in Transform Composite Web Service. Finally, the composite web service is published in the internet. In this approach, the ontology model is based on ODM. All UML models are transformed into XMI document through a set of adequate mapping rules. This approach uses ATL for creating transformation rules.

A Classification of Model-Driven Approaches to Develop Semantic Web Service

401

Kim and Lee 2009. Kim and lee approach [1] is a method to moderate the complexity of UML diagrams for developing Semantic Web service descriptions. The method follows three steps, ontology, process, and transformation modeling. First step provides the capability to extract a set of class diagrams from imported ontologies. In the next step, sequence and activity diagrams are used along with UML profile and its elements like stereotype, tagged value, and constraint to accomplish the process tasks. The objects of sequence diagrams are related to a relevant class diagram. In the process of designing sequence diagrams, interaction fragments are used to describe OWL-S control constructs. Activity diagrams are used to represent OWL-S composite process because their provided notations can describe various control constructs related to OWL-S. Finally, the UML diagrams are exported to an XMI document which is transformed into the OWL-S description via an XSLT script. In addition, the conditional statements are transformed into SWR syntax in OWL-S descriptions. Belouadha et al. 2010. Belouadha et al. [27] provides a method to create a model-driven SAWSDL Semantic Web service composition. The approach provides an independent UML profile for modeling Web services. In addition, transaction rules are used to generate the SAWSDL interface files. Furthermore, the BPMN notation is used to model the behavior of Web service composition and for generating its executable BPEL file. In this approach, a transformation tool is created to map WSDL to SAWSDL and vice versa. Practically, their approach is based on semantic web service composition meta-model which is independent of the SAWSDL description. The interface of this meta-model is realized by a business service which is modeled as web service. This interface contains a set of operations that have some parameters like input, output and faults. Each parameter specified by specific data type. In this meta-model, each parameter, operation, and interface can associate with a semantic concept which can be an ontological concept. Two type of mapping are specified in this meta-model between given parameter data types and its semantic concept: LiftingSchema and LoweringSchema attributes. Thus, mapping class is specified using these attributes as its properties. This approach takes advantage of ATL to implement the transformation rules.

3 Summary of Approaches In this section, the aforementioned approaches are represented in terms of MDA standard properties such as modeling language, transformation languages, and Semantic Web service languages as shown in Table 1. Indeed, each approach must have these properties to be considered ‘‘model-driven approach’’.

402

A. Kalantari et al.

Table 1 A summary of approaches in terms of MDA standard properties Modeling language UML Approaches CD AD SQD STD UP UML formalist-based Gronmo H H H et al. [21] Yang & H H Chung [22] Lautebacher H H H & Bauer [23] H H H Timm & Gannod [8] Bensaber & H H H Malki [25] Sun et al. [2] H H Kim & Lee H H H H [1] Belouadha H H et al. [27] Software methodology-based Torres et al. H H H H [13] Brambila et al. [14] MIDAS-S H H [18, 19]

Non-UML

Semantic Transformation web service language standards

Condition BPMN WebMl OCL

OWL-S

XSLT

GUI

OWL-S

XSLT

CNT

OWL-S

Xpand

OCL

OWL-S

XSLT

OCL

OWL-S

XSLT

N/A CNT

OWL-S OWL-S

ATL XSLT

SAWSDL

ATL

OWL-S

M2T

WSMO

XSLT

WSMO

N/A

H

OWL H OCL

H

CD class diagram, AD activity diagram, SQD sequence diagram, STD statechart diagram, UP UML profile, CNT constraints, M2T model to text, N/A not applicable

4 Conclusion and Future Work In this paper, we have provided an overview of most prominent model-driven approaches that describe Web Services based on current W3C submissions such as OWL-S, WSMO, and SAWSDL. Moreover, we have classified these approaches into two main categories namely, software methodology and UML formalist based. This classification helps to evaluate the approaches for further researches and investigations. The initial evaluation shows that almost all approaches that are placed in the UML-formalist based category are based on OWL-S. Instead, all approaches that provide semantic Web service based on WSMO have been provided in software methodology manner. Most approaches use UML as their modeling language and in most of them the UML class diagrams are used to represent atomic process. There are still some issues that we are working on.

A Classification of Model-Driven Approaches to Develop Semantic Web Service

403

Currently, we are comparing these approaches and categories based on some known criteria such as independency, expressiveness, completeness, readability, and etc.

References 1. Kim I-W, Lee K-H (2009) A model-driven approach for describing semantic Web services: from UML to OWL-S. IEEE Trans Sys Man Cybern Part C Appl Rev 39(6):637–646 2. Sun W et al (2009) A model-driven reverse engineering approach for semantic web services composition. IEEE Comput Soc, Xiamen 3. David M, Burstein M (2004) OWL-S: semantic markup for Web services, [cited 2010 August 4]. http://www.w3.org/Submission/OWL-S/. Accessed 22 Nov 2004 4. Jos de Bruijn CB (2005) Web service modeling ontology (WSMO), [cited 2010 Feb 21]. http://www.w3.org/Submission/WSMO/. Accessed 3 June 2005 5. Semantic Annotations for WSDL and XML Schema, [cited 2010 September 12]. http:// www.w3.org/TR/sawsdl/. Accessed 28 August 2007 6. WSMO Studio (2009) [cited 2010 May 18]. http://www.wsmostudio.org/ 7. Kerrigan M (2006) WSMOViz: an ontology visualization approach for WSMO. Institute of Electrical and Electronics Engineers Inc, London, United kingdom 8. Timm JTE (2008) A model-driven framework for the specification, grounding, and execution of semantic Web services. Arizona State University, p. 170 9. Miller J, Mukerji J (2003) MDA Guide Version 1.0.1, [cited 2010 Jan 15]. www.omg.org/ mda/mda_files/MDA_Guide_Version1-0.pdf 10. Group OM (2010) OMG unified modeling language (UML), [cited 2010 September 21]. http://www.omg.org/spec/UML/2.3/. Accessed May 2010 11. Clark J (1999) Extensible stylesheet language transformations (XSLT), [cited 2009 October 27]. http://www.w3.org/TR/xslt. Accessed 16 November 1999 12. ATL User Guide (2010) [cited 25 November 2010]. http://wiki.eclipse.org/ATL/User_Guide 13. Torres V, Pelechano V, Pastor O (2006) Building semantic web services based on a model driven web engineering method. Springer, Tucson 14. Brambilla M et al (2007) Model-driven design and development of semantic Web service applications. ACM Trans Internet Technol 8(1): 15. Business Process Model and Notation (BPMN) (2009) Version 1.2, [cited 2010 August 6]. http://www.omg.org/spec/BPMN/1.2 16. Stefano C et al (2002) Designing data-intensive web applications, vol 500. Morgan Kaufmann Publishers Inc., San Francisco 17. Webratio site development (2009) [cited 2010 March 14]. http://www.webratio.com 18. Acuna CJ, Marcos E (2006) Modeling semantic web services: a case study. Association for Computing Machinery, Palo Alto 19. Sanchez DM et al. Toward UML-Compliant semantic web services development. Int J Enterp Inf Sys 6(1):44–56 20. Cáceres P, Marcos E, Vela B (2003) A MDA-based approach for web information system development. In: Workshop in software model engineering 21. Grnmo R, Jaeger MC, Hoff H, (2005) Transformations between UML and OWL-S. Springer, Nuremberg 22. Yang JH, Chung IJ (2006) Automatic generation of service ontology from UML diagrams for semantic web services. Springer, Beijing 23. Lautenbacher F, Bauer B (2007) Creating a meta-model for semantic web service standards. INSTICC Press, Barcelona 24. Xpand, [cited 2010]. http://www.eclipse.org/workinggroups/oaw/

404

A. Kalantari et al.

25. Bensaber DA, Malki M (2008) Development of semantic web services: Model driven approach. Association for Computing Machinery, Lyon 26. Grnmo R et al (2004) Model-driven Web services development. IEEE Comput Soc, Taipei 27. Belouadha F-Z, Omrana H, Roudies O, A model-driven approach for composing SAWSDL semantic Web services

A Conceptual Model Explaining IT/IS Usage Behavior Sabine Hoesch and Siegfried Voessner

Abstract Information Systems and Information Technology have become a crucial success factor for business environments. To ensure optimal use of Information Systems it is especially important to understand usage behavior and factors influencing their adoption by an organization. In recent decades some models have been developed to describe information technology and information systems (IT/IS) usage behavior under voluntary or mandatory conditions focusing mainly on technological or task-related perspectives. These models however can not explain observed phenomena like underperformance of technologically optimal information systems or over-performance of suboptimal systems. In an attempt to close this gap we present a causal model integrating both human and social change processes as well as feedback factors affecting the usage of IT/IS on a regular basis. We further show implications for the system adoption process in a business environment. Keywords IT/IS usage Causal loop




Acceptance




Usage behavior




Commitment




S. Hoesch (&)
S. Voessner Department of Engineering- and Business Informatics, Graz University of Technology, Graz, 8010 Austria e-mail: [email protected] S. Voessner e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_39, Ó Springer Science+Business Media B.V. 2012

405

406

S. Hoesch and S. Voessner

1 Introduction In the field of IT/IS, to predict acceptance, usage and success many models were proposed [1–6]. Ajzen’s theory of planned behavior (TPB) explains motivational based behavior [1], the technology acceptance model (TAM) is intended to predict user’s acceptance towards technology in both volitional and mandatory settings. TAM is able to predict about 40% of system use. It misses out negative emotions regarding technology and emotions related to the process of implementing a new technology [1] as well as TPB. Additionally, most of the models do not provide information about the types of relationships.

2 Research Objectives With this study we aim to explain the IT/IS usage behavior at an individual level with respect to the possibility of enforcing IT/IS usage in organizations. Moreover even the technological and social change processes are taken into account. In a business environment users usually are not decision makers. Users have to get along with the available system or perhaps have to decide whether to use it or not. Furthermore changing organizational structures and processes resulting from IT/IS implementation often unsettle employees. With the proposed model we show implications for the individual IT/IS adoption process in a business environment.

3 Methodology In a literature review we considered articles related to the various models explaining IT/IS usage, acceptance and success. A synthesis of these models and studies which focus mainly on specific relationships led to the main behavior model. Based on the behavior theories Motivation Model, Theory of Planned Behavior [7, 8] and the behavior theories in relation to IT/IS usage and acceptance TAM [4] and IS Success Model [3], we developed an Extended Intention Model. We use System Dynamics to put components into perspective and add loops to explain business IT/IS usage at an individual level.

4 Existing Behavioral Models and its Factors In this paper we build on the relevant main behavior models explaining either behavior in general or especially targeting IT/IS usage behavior and will therefore review them in brief. These models are Theory of Planned Behavior, Technology

A Conceptual Model Explaining IT/IS Usage Behavior

407

Acceptance Model, IS Success Model and the Motivation Model. In the following we give a brief explanation of the models and its factors.

4.1 Behavioral Models The Theory of Planned Behavior suggests that an individual’s intended behavior is determined by the attitudes toward a specific behavior, subjective norms and control beliefs regarding the behavior [1]. TAM [4] examines the effect of perceived usefulness and perceived ease of use on intention to system use. It includes task and technology related determinants. References [2, 3] proposed the IS Success Model. It suggests that any organizational or individual benefits are determined by usage and user satisfaction. This model neither includes any individual differences nor emotions related to the process of change. The Motivation Model [8] suggest that any behavior is related to extrinsic or intrinsic motivation [8, 9].

4.2 Factors in Behavioral Models In these existing models the main factors are: intention, attitude, subjective social norms, motivation, perceived behavioral control, system usage, satisfaction, usefulness, ease of use and enjoyment. Additionally several studies show a relevant effect of commitment [7, 10, 11]. The core element is the intention to perform a specific behavior. It results from motivational factors indicating the willingness how hard to try and how much effort to spend to perform a behavior [1]. Attitude is the individual’s evaluative or affective response to a specific object [7] or a behavior in question and affects the intention to perform a behavior [1]. Intrinsic and extrinsic sources motivate to perform an intended behavior. The more motivated an individual is the more likely is the behavior in question [8]. Intrinsic motivation covers the engagement for pleasure and satisfaction. Extrinsic motivated behavior is performed due to instrumental reasons [8], e.g. attaining rewards or avoiding punishments [9]. Someone can even be motivated by subjective social norms. These refer to the degree to which an individual perceives that most people who are important to him think he should or should not use the system [1, 6, 7, 12]. Perceived behavioral control (PC) is the emotional response to the difficulty of performing a behavior [1, 13–15]. Reference [16] found self-efficacy, requisite resources and technical support to determine PC. The absence of PC control refers to any negative emotional attachment regarding the task in question that requires a specific behavior, e.g. lack of resources. User commitment describes the user’s psychological attachment to system use [10]. Rather than just complying with the beliefs of other’s the user wants to comply following its own beliefs. There are three types of commitment, each satisfying different personal goals. Internalization

408

S. Hoesch and S. Voessner

relates to behavior adopted because of its content, that is evaluated to be consistent to the user’s own personal goals [10]. Identification occurs when the user adopts a behavior not because of the content, but he wants to achieve a satisfying, selfdefining relationship to other persons or groups. Identification and Internalization are subsumed to affective commitment. In compliance, the content and the salience of relationships are irrelevant. The adopter of a particular behavior wants to achieve a favorable reaction [3, 8, 10, 11, 17, 18]. According to the definition of motivation [8] this kind of commitment pertains to extrinsic motivation that is regulated from outside the individual user [10]. System usage is defined as the amount of system use by the individual user. As satisfaction refers to prior usage of IT/IS, from the perspective of a causal model system usage only acts as an enabler for satisfaction. In consumer behavior satisfaction is determined by user’s experiences/perceptions and the expectations the consumer established [20–22]. In the field of IT/IS user satisfaction is defined as the net feeling of pleasure or displeasure that results from a specific interaction with the information system [23]. According to this definitions user satisfaction requires usage prior to the evaluation. Satisfaction reflects the hierarchy of consumer needs: Functionality, Usability and Pleasure [24] and is mediated by the expectation of success [20, 25]. Usefulness is considered as the perceived usefulness from the user’s perspective and is always related to the user’s tasks. It includes the items Work More Quickly, Job Performance, Increase Productivity, Effectiveness, Makes Job Easier and Is Useful. Reference [26] proposed task-technology fitness as an appropriate evaluator for meeting user’s needs. Therefore we conclude that perceived usefulness copes with the ability to support user’s tasks. Ease of use, refers to ‘‘the degree to which a person believes that using a particular system would be free of effort.’’ This follows from the definition of ‘‘ease’’: ‘‘freedom from difficulty or great effort’’ [4]. Enjoyment is related to stimulating, beautiful and pleasurable system attributes[24]. Expertise is related to the level of education [27] and describes an individual’s characteristics of domain experience [27, 28], prior similar experience with and knowledge about technologies and systems in general and especially regarding the IT/IS in question. These models do not provide any insight into individual reactions to social and technological change processes. Therefore we have to extend these models and factors described above to be applicable to a business IT/IS context. They provide the basis for our extended behavior model which we present in the following.

5 An Extended Intention Model as a Basis for Business IT/IS Usage Based on the TPB factors, IT/IS usage intention underlies subjective norms, attitude and perceived behavioral control (Fig. 1). Subjective norms are formed by the extrinsic reinforcement of compliance; another form of subjective norm is

A Conceptual Model Explaining IT/IS Usage Behavior

409

Fig. 1 An extended IT/IS usage intention model based on Ajzen’s TPB [1]

Identification [12]. Additionally to this extrinsic type of motivation employees can be intrinsically motivated, that is, when users use the system because usage of the system satisfies them and helps them to achieve goals, e.g. accomplish tasks. With respect to user’s uncertainty and self-efficacy the intention model is extended with the perceived control of change which is subsumed with perceived behavioral control to perceived control of behavior and change (PCBC). The uncertainty when introducing new IT/IS is expressed in its opposite, selfefficacy, which refers to the ability to deal with uncertainty in a changing sociotechnical system breaking up processes and organizational structures. Based on Sydow’s model of a socio-technical system [29], that includes the items Task, Technology, User and Role in the primary work environment, we define the uncertainties: Task Self-Efficacy (User—Task): The user is aware that he has the required skills. He is sure to be able to accomplish the new or changing tasks with the available resources. Technology Self-Efficacy (User—Technology): The user is sure that he will be able to use the new IT/IS. Structure Self-Efficacy (User— Role): The user is sure, that although tasks and technologies are changing he will hold his role within the organizational structure. Process Self-Efficacy: Process self-efficacy refers to the overall control of task execution in the primary work environment. The user is sure that he will be able to transform the given input into the desired output within the business process. As all these self-efficacies regard to both control beliefs and fears of change we combine these to the perceived control of behavior and change (PCBC). Similar to perceived behavioral control directly related to the behavior in question, PCBC affects the psychological attachment regarding a specific behavior and therefore the intention to use.

6 An Extended Behavior Model for Business IT/IS Usage By putting all building blocks together (factors, motivation models and polarities of relationships) we can finally construct an extended behavior model for business IT/IS usage. Figure 2 presents relationships between the system variables and the type of relationship, which is related to the causal loop diagram notation. ‘‘+’’ refers to a positive relationship: increasing the value of a variable increases the value of the related variable. On the other hand ‘‘–’’ represents a negative

410

S. Hoesch and S. Voessner R B

Reeinforcing loop Balancing loop Direction of relationship

+ -

+

-

Usefulness

R

R

+

Expertise

+

Usage

B

+ +

Positive relation Negative relation

+

B

Ease of Use

+

Task-related System Quality

Enjoyment

+

R

+ +

R

Perceived Control of Behavior and Change

System Improvement

Satisfaction

+

Compliance

B

B

+

+

Expectations Intention to Use

+

+

-

+

Feedback

+

B

Affective Commitment

R

+

Fig. 2 The Business IT/IS usage behavior model shows the factors and the feedback loops

relationship, indicating that a modified value of the independent variable causes the value of the related variable to change in the other direction [30]. Based on the presented intention model the types of relationships and feedback loops are added to form the causal model. Additionally we add the following relationships: Using the system enhances user’s expertise with the IT/IS and in general with technology through the cognitive learning process (Usage is positively related to Expertise). This familiarity and experience [1] with IT/IS improve task and technology self-efficacy (Expertise is positively related to PCBC). Similar prior experiences and expertise with the new IT/IS enable greater learning and a more positive perception of ease of use [12, 27, 31] (Expertise is therefore positively related to Ease of Use). [24] defined the overall needs to be functionality, usability and pleasure (enjoyment). The fulfillment of personal needs results in satisfaction. Thus the higher usefulness and ease of use the higher the satisfaction [23]. According to this concept enjoyment improves satisfaction too [32–34] (Usefulness, Ease of Use and Enjoyment are positively related to Satisfaction). A good non-subjectively defined task-related system quality will by trend lead to high usefulness and ease of use. Even enjoyment can result from a good task-related system quality (Task-related System Quality is positively related to Usefulness, Ease of Use and Enjoyment). According to [4] ease of use enhances usefulness (Ease of use is positively related to Usefulness). Missing satisfaction in the best case brings users to complain (Satisfaction is negatively related to Feedback). Another possibility to gain feedback is to drive it through affective commitment. When users commit to a good system quality they are engaging more in system

A Conceptual Model Explaining IT/IS Usage Behavior

411

improvement (Affective Commitment is positively related to Feedback). In case of the availability of resources this feedback can be transformed into system improvement leading to a better task-related system quality and a higher commitment. Due to affective commitment users will invest greater effort in learning and using the system and find it more useful [10]. Even compliance can enforce intention to use [35]. On the other hand it was found to reduce perceived usefulness and ease of use [9, 10].

7 Analyzing the Model The model consists of five reinforcing and four balancing loops. In a reinforcing loop starting from one factor and going around the loop ends up in increasing the value of the factor. E.g. increasing Intention to Use enhances Usage of the system, which in turn enables Satisfaction, which will increase Intention to Use. The second reinforcing loop includes Intention to Use, Usage, Expertise, PCBC. Intention to Use enhances system Usage, that leads to improved Expertise. As Expertise reduces the anxieties regarding the new technology and the modified task, PCBC is improved, which in turn leads to higher Intention to Use. Expertise also occurs in the third reinforcing loop: better Expertise leads to the perception that the system is easier to use, improving Satisfaction, which leads to a higher Intention to Use, Usage and improved Expertise. As Ease of Use is improving Usefulness, a fourth reinforcing loop can be identified (Expertise, Ease of Use, Usefulness, Satisfaction, Intention to Use, Usage). From the fifth loop Affective Commitment turns out to be a driver for System Improvement. In a balancing loop increasing one factor and going around the loop ends up in decreasing the original value of the factor (or vice verse). E.g. high Expectations tend to lower Satisfaction, which in turn over time leads to reduce expectations. The other balancing loops encourage feedback from dissatisfied users. Providing that there are enough monetary and human resources and knowledge for system development, feedback can yield to system improvement activities, which improve the Task-related System Quality. As Usefulness, Ease of Use and Enjoyment result from Task-related System Quality and end up in Satisfaction, three balancing loops can be identified. Identifying these reinforcing loops it is obvious that there are several factors driving IT/IS usage and few factors limiting IT/IS usage. PCBC represents the anxiety and uncertainties regarding the change in a socio-technical system. The lack of PCBC may inhibit the intention to use and therefore usage of the system and is therefore absolutely required for intention to use. Even usage is limited to the amount that is useful to accomplish tasks in the primary work environment. Bringing users to comply decreases the perceptions about usefulness of a system and therefore lowers the impact of one main driver of Intention to Use, which is Satisfaction. As we found the relationship between Satisfaction and Feedback to be rather weak, Affective Commitment enables greater System Improvements than

412

S. Hoesch and S. Voessner

it can be achieved by dissatisfied users. As Usage only acts as an enabler to Satisfaction, the reinforcing loop obviously is a rather weak one.

8 Future Work The model presented in this paper integrates technological and task related issues as well as the human and social change processes. Feedback factors were identified demonstrating that continuous usage of IT/IS is a consequent evaluation process of individual positive and negative perceptions. The causal model enables the identification of barriers to IT/IS implementation success and therefore offers the opportunity to soften impeding factors and reinforce motivating factors. Nevertheless IT/IS usage in business environment remains a complex field of investigation. Future work has to be done to validate the conceptual model further using comparative, detailed case studies throughout industries. This will provide possibilities to support an optimal IT/IS implementation process. Further strategies have to be identified to ensure the individual adoption of IT/IS in an business environment.

References 1. Ajzen (1991) The theory of planned behavior. J Organ behav hum decis process 50(2): 179–211 2. Delone W, McLean E (1992) Information systems success: the quest for the dependent variable. J Inf Sys Res 3(1):60–95 3. Delone W, McLean E (2003) The DeLone and McLean model of information systems success: a ten-year update. J Manag Inf Sys 19:9–30 4. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 13:319–339 5. Venkatesh V, Morris MG, Davis GB, Davis FD (2003) User acceptance of information technology: toward a unified view. MIS Q 27(3):425–478 6. Venkatesh V, Bala H (2008) Technology acceptance model 3 and a research agenda on interventions. Decis Sci 39(2):273–315 7. Fishbein M, Ajzen I (1975) Belief, attitude, intention and behavior: an introduction to theory and research. Addison-Wesley Reading, Boston, pp 89–114 8. Vallerand RJ (1997) Toward a hierarchical model of intrinsic and extrinsic motivation. Academic Press, Cleveland, pp 271–360 9. Deci EL (1971) Effects of externally mediated rewards on intrinsic motivation. J Per Soc Psychol. 18:105–115 10. Malhotra Y, Galletta D (2005) A multidimensional commitment model of volitional systems adoption and usage behavior. J Mgmt. Inf Sys. 22(1):117–151 11. Kelman HC (1961) Processes of opinion change. Public Opin Q. 25:57–78 12. Venkatesh V, Davis FD (2000) A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci. 46(2):186–204 13. Bandura A, Adams NE, Hardy AB, Howells GN (1980) Tests of the generality of selfefficacy theory. Cogn Ther Res. 39–66

A Conceptual Model Explaining IT/IS Usage Behavior

413

14. Bandura A (1982) Self-efficacy mechanism in human agency. Amn Psychol. (37):122–147 15. Bandura A (1978) Self-efficacy: Toward a unifying theory of behavioral change. Adv Behav Res Ther 1(4):139–161 16. Huang E, Chuang MH (2007) Extending the theory of planned behaviour as a model to explain post-merger employee behaviour of IS use. Comput Human Behav 23(1):240–257 17. Engel P (1998) Comments on logical and psychological partitioning of mind: depicting the same map?. New Ideas Psychol 16(3):107–113 18. Wixom BH, Todd PA (2005) A theoretical integration of user satisfaction and technology acceptance. Inf Sys Res 16(1):85–102 19. Seddon PB (1997). A respecification and extension of the DeLone and McLean model of IS success. Inf Sys Res 8(3):240–253 20. Oliver RL (1981) Measurement and evaluation of satisfaction processes in retail settings. J Retail 57(1):25–48 21. Liao C, Palvia P, Chen J (2009) Information technology adoption behavior life cycle: Toward a Technology Continuance Theory (TCT). Int J Inf Manag 29(4):309–320 22. Brown SA, Venkatesh V, Kuruzovich J, Massey AP (2008) Expectation confirmation: an examination of three competing models. organ behav human decis process 52–66 23. Seddon PB, Kiew M (1994) A partial test and development of the DeLone and McLean model of IS success. In: Proceedings of the Fifteenth International Conference on Information Systems. 99–110 24. Jordan PW (2000) Designing pleasurable products: An introduction to the new human factors. CRC 25. Bhattacherjee Aab (2001) Understanding information systems continuance: an expectationconfirmation model. MIS Q 25(3):351–370 26. Goodhue DL, Thompson RL (1995) Task-technology fit and individual performance, MIS Q 19(2):213–236 27. Agarwal R, Prasad J (1999) Are individual differences germane to the acceptance of new information technologies?. Decis Sci 30(2):361–390 28. Olfman L, Mandviwalla M (1994) Conceptual vs. procedural software training for graphical user interfaces: a longitudinal field experiment. MIS Q 18(4):405–426 29. Sydow J (1985) Der soziotechnische Ansatz der Arbeits—und Organisationsgestaltung. Campus-Verl, Frankfurt 30. Sterman J (2000) Business Dynamics. McGraw-Hill, New York 31. Gick ML, Holyoak KJ (1987) The cognitive basis of knowledge transfer. Trans Learn: Contemp Res Appl 9–46 32. Diefenbach S, Hassenzahl M (2009) The beauty dilemma: beauty is valued but discounted in product choice. ACM, New York, pp 1419–1426 33. Karapanos E, Hassenzahl M, Martens J (2008) User experience over time. ACM, New York, pp 3561–3566 34. Igbaria M, Schiffman SJ, Wieckowski TJ (1994) The respective roles of perceived usefulness and perceived fun in the acceptance of microcomputer technology. Behav Inf Technol 13:349–361 35. Díez E, McIntosh BS (2009) A review of the factors which influence the use and usefulness of information systems. Environ Model Softw 24(5):588–602 36. Al-Gahtani SS, King M (1999) Attitudes, satisfaction and usage: factors contributing to each in the acceptance of information technology. Behaviour and Information Technology, Taylor and Francis Ltd, London, pp 277–297 37. Barki H, Hartwick J (1994) Measuring user participation, user involvement, and user attitude. MIS Q 18(1):59–82 38. He J, King W (2008) The role of user participation in information systems development: implications from a meta-analysis. J Manag Inf Sys, M. E. Sharpe, Inc, Armonk, New York, 301–331

Offering a Conceptual Model of Technology Acceptance in e-Banking Based on the Customers’ Satisfaction and Trust Alireza Dehghan, Parham Arjomand, Amirahmad Nayyeri and Masoud Tabatabaey

Abstract By facilitating the relationship with customers and increasing its speed and effectiveness, the Information Technology paves the way to improve the functions and innovations of offering services. One of the issues that is always raised when investigating a new technology is the amount of the Acceptance of that technology by users and customers. Getting highly improved in the two recent decades, electronic banking is not an exception. The Technology Acceptance Model is one of the models which has been widely used to describe the influential factors in the Acceptance of information systems and new technologies. The two factors of conceived using facility and conceived profitability have been regarded as basic factors in determining the Acceptance of various information technologies in the last two decades. In this article, while investigating the technology Acceptance model, efforts have been made to offer a combined model based on two significant factors, including customers’ satisfaction and trust, as other influential factors in e-banking Acceptance. Keywords e-Banking


Technology Acceptance Model
Satisfaction
Trust

A. Dehghan (&)
P. Arjomand Faculty Member of Department of Information Technology Engineering, Kazeroun, Iran e-mail: [email protected] P. Arjomand e-mail: [email protected] A. Nayyeri Faculty Member of Department of Computer Science, Kazeroun, Iran e-mail: [email protected] M. Tabatabaey Student of Information Technology Engineering, Kazeroun Higher Education Complex, Fars, Kazeroun, Iran e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_40, Ó Springer Science+Business Media B.V. 2012

415

416

A. Dehghan et al.

1 Introduction One of the most important elements to develop e-commerce is an e-banking system that facilitates transactions of e-commerce instantaneously with global financial system changes. In fact, e-commerce is not independent of e-banking. Use of electronic systems in finance and credit institutions is increasing fast all over the world and customers of e-banking services are growing. According to Data Monitor research, number of e-banking services customers in 8 countries, France, Germany, Italy, Sweden, Switzerland, The Netherlands, Spain and England from 4.5 million in 1999 got up to 22 millions in 2004. In 2005 more than 75% of companies in developed countries use at least one of the e-banking services [19]. E-banking is one of the most successful services in e-commerce based on increase of customer satisfaction which is relied to access to value-added services. Actually, this method has transformed the way access to accounts and type of transactions. In this type of banking, customer needs should be identified and ranked to know the importance and influence of each of these needs on customer satisfaction. It can work with banks to determine what factors and with what degree of influence, have an impact on customer satisfaction and as a result try to satisfy that factors. One of the most significant results of customer satisfaction is their loyalty. Regarding to rapid growth of communication technology and emerging electronic banking system, pioneer banks in this field that make customer satisfaction their highest priority, can get up great benefits. In the past, because of the state banking system and the same services provided by banks, customers paid no fixation to any particular bank. But in recent years by growth of nongovernmental banks and having more competition among banks, all banks try to gain competitive advantage, particularly through better satisfying customers’ needs. In this regard, one of the most strategic services is electronic banking and customer satisfaction of that. There is always a concern in the implementation of electronic banking whether this new technology and new services to their users and customers will be admitted or not. The Technology Acceptance Model (TAM) is one of the models which has been widely used by users and customers to describe the influential factors in the Acceptance of information systems and new technologies [22]. In this paper in addition to examine each of the satisfaction and trust factors, will pay attention to the relationship between this two factors in e-banking and at the end we will provide a conceptual model of technology Acceptance in electronic banking based on satisfaction and trust of customers.

2 Acceptance of IT Acceptance of IT systems plays an important role in investing in this technology. Acceptance of this technology by the users is one of the factors affects the success and has positive results for banking organizations. Impact of technology, has

Offering a Conceptual Model of Technology Acceptance in e-Banking

417

positive relationship with its acceptance. So, if the potential users of this technology show resistance against it, the desired objectives can’t be achieved. Low level of acceptance of information technology causes low-level use. Low level of computer use at work is one of the main problems that affect the effectiveness and efficiency of banking institutions and has negative effects. Since the new banking system based on computer use, factors affecting customers and users of information technology should be known to increase the use of computers.

3 The Technology Acceptance Model (TAM) The Technology Acceptance Model is one of models which widely used to describe the factors affecting the Acceptance of information systems and communication technology by users and customers [22, 30]. This model should be regarded as a powerful model for predicting technology acceptance by users [30]. Understanding the factors that cause the Acceptance of a technology and make information technology to be accepted is important researches in the field of information technology [9, 11]. Technology Acceptance model was made by Davis In 1989 to understand the relationship between attitudes, behavior of IT consumer and also to determine the factors affecting IT Acceptance by users of information systems and in order to clarify and predict the behavior of computer users was developed [8].(see Fig. 1). As shown in Fig. 1, technology Acceptance model derived from the Theory of Reasoned Action (TRA) (Fishbein-Ajzen). The logical operation theory that is a general model expresses that: attitude determines the social behavior [23] and attitude is a function of the beliefs about the consequences of the behavior and evaluates the result [1]. In the technology Acceptance model an implementation of the logical operation theory is done to model the Acceptance of information systems by users. To beliefs, Perceived Ease of use (PEOU) and Perceived Usefulness (PU) has been considered as two basic structures in the Acceptance of IT in the past decades [12]. According to The technology Acceptance model, perceived usefulness of the system by users is influenced by the fact that the system is easy to use. This model also assumes that the perceived ease of use and perceived usefulness has a direct impact on people’s attitude to use of information systems and ICT [2]. The aim of technology model is to provide a description of the computer acceptance parameters which is general and has the ability to describe the behavior of users in a wide range of computing technology and its users are of different types [20]. The model has descriptive approach, in addition to predictive aspects. Thus, researchers and administrators can identify why a specific system may not be accepted and the appropriate corrective steps to be done. In 2000, Davis and Venkatesh [30] expanded the TAM model theoretically and described the usefulness and desires and intentions to use words the social impact and Cognitive instrumental processes. The model developed is known as TAM II

418

A. Dehghan et al.

Fig. 1 Original technology acceptance model

Fig. 2 Technology acceptance model II

[21] (see Fig. 2). Using collected data from four different systems that in two examples voluntary use and in two others compulsory use was analyzed. Model structures at three different times: before deployment, a month after deployment, and three months after system implementation in any organization was measured. So, the developed model was confirmed in all organizations at all three times [16]. By using TAM I as starting model, in TAM II new theoretical structures including social impacts (subjective norms, voluntary and imagine) and cognitive instrumental processes (job relationship, quality of output, the ability to prove the results and ease of use) was added to the model [20].

4 The First Principle of e-Banking, Customer Orientation Because the economic cycle of all financial institutions will continue with the support and investment of customers of in e-banking systems should pay special attention to the customer. Research shows that customer retention is how crucial for businesses and service quality is a factor that is likely to affect customer retention rate. Even a 5% increase in customers, raises the profit level 25–30% and if a institution 5% decrease rate of losing customers, the final net value of each customer up to 75% increase for the Institute [15]. Because of today’s competitive business world, the organization will succeed in attracting customers that provides

Offering a Conceptual Model of Technology Acceptance in e-Banking

419

the best and most to services to its customers. Therefore, the role of customers in the development of economic activities should be a principle and E-banking applications regarding to this principle of structures [18]. One of the major flaws in the current banking system is that the customer’s is classified by the bank branch where he has account and other branches do not know him as one of users of their bank’s services. This causes the customer refers to several banks for performing banking activities and incurs costs and wastes a lot of time [19]. Development of ebanking creates the possibility that customers be serviced in any branch of that bank and not just a specific branch [18]. In other words, the extensor branch of account will register his signature and his name and through connected systems all the branches of the bank, and if all systems are connected, all the banks knew him and will service him [19].

5 Customer Satisfaction According to this research, the concepts related to customer satisfaction and its measurement, the first time was expressed in 1977 by ‘‘Oliver’’. He proposed customer satisfaction as pleasant experience of shopping of last product or service [10]. In other words, customer satisfaction means that customers are satisfied with treatment of employees and services of organizations and the bank has succeeded in attracting and retention of them. These satisfied customers, how spend more time and money, to will expect services with higher quality [7]. In other words, Kotler defines customer satisfaction as a scale to which a company’s actual performance to satisfy customer expectations is measured. In his opinion If the performance of the company meets customer expectations, customer is satisfied and otherwise make him sense discontent [25]. Customer satisfaction provides many benefits for the company and higher levels of customer satisfaction leads to loyalty. The customers, who are satisfied a lot, tell their good experiences to others. High customer satisfaction is a kind of insurance for the company against possible mistakes that because of changes associated with their services is inevitable. Permanent customers in dealing with such situations are more tolerant and due to previous good experiences, easily ignore these few mistakes. Therefore, it is not surprising that customer satisfaction is the most important function of organizations and institutions [17]. In accordance with principles of communication marketing, customer satisfaction is the beginning of relationship between customer and seller. Communication satisfaction is the prerequisite for quality of communication. But in electronic banking there is no relationship between customer and seller. Therefore for satisfying customers other solutions such as giving special discounts to customers using electronic banking, giving services to customers conveniently, providing high-speed internet for transaction should be used.

420

A. Dehghan et al.

Fig. 3 Technology acceptance model and its impact on trust

6 Trust The first step in describing a structure is to understand its meaning. The important element among organizations and transactions is trust which facilitates risk and significantly eases the economic transactions. Trust facilitates human interactions. Trust helps the economy and helps people to do business transactions smoothly. In other side, Lack of Trust is a beneficial mental state that makes us able to avoid the systems or individuals or organizations which are unreliable and unsafe [6]. Berry mentioned trust as the most powerful marketing tools based on relationship [3]. According to studies of Fui-Hoon [5] Customers decide to buy online just because of trust [28]. Meyer has defined trust as a person willing to be vulnerable in front of the opponent, Based on this assumption that the other side will perform the expected activity without his direct supervision and control [26]. In another definition trust given as a belief or expectation to which the seller states are reliable and seller is not exploiting vulnerability of customer [6]. Trust and risk completely relevant together and in other words, risk is core of trust [26].

7 The Impact of Trust on Electronic Banking Today, security and distrust are the main concerns of users of electronic banking. For users of electronic banking that connect to the Internet to use services of this type of banking, trust has a great importance. Trust happens when a person believes that the other side has useful factors in addition to profitability [27]. It is essential that customers know the product or service provider because without reducing the risk customers can’t continue to interact with the product or service providers. Trust is one of the most effective methods of reducing risk for customers [14]. Trust in electronic banking is more important than in offline banking [26]. When it is important to make others to trust us that uncertainty and risk is

Offering a Conceptual Model of Technology Acceptance in e-Banking

421

Fig. 4 Factors affecting customer satisfaction and loyalty

widespread [24]. In the Internet, users all around the world have the ability to find important files and information on the computers and transfer information through internet. Marketing researchers consider trust as a key structure in relationship marketing [4], [14]. The researchers believe that customer trust has a significant effect on customer loyalty. This could be considered as customer willingness to maintain an ongoing relationship with the seller. Bashiri and Joneidi in an article titled ‘‘Effect of Trust on customer acceptance of Internet banking, based on TAM model, Mellat Bank case’’ studied the basic model of TAM (see Fig. 3). The researchers by adding another structure to the basic model, Concluded that the Trust as a third influential structure in the basic model will increase the acceptance of Internet banking by customers [20].

8 The Relationship Between Satisfaction and Customer Trust Since different models for the study of loyalty, customer satisfaction and Trust in the marketing issues are discussed. For example Deng and Zhaohua in an article titled ‘‘Understanding customer satisfaction and loyalty’’ offered a model where factors such as Trust, service quality, perceived value (functional, emotional, social and financial) and the costs for change are considered as factors influencing customer loyalty and satisfaction but these factors are not necessarily affected by the companies trade marks (see Fig. 4) [13].

9 Technology Acceptance Model Based on Trust and Customer Satisfaction Based on discussed models and cases and after studies done in this research, these models were combined and eventually a conceptual model for acceptance of technology with emphasis on satisfaction and Trust was provided (see Fig. 5).

422

A. Dehghan et al.

Fig. 5 TAM with an emphasis on customer satisfaction and trust

As was stated in the previous sections, TAM model has two basic structures which are perceived usefulness and perceived ease of use. These two structures are always influenced by external variables such as organizational factors, social factors, mental perceptions and system features such as computer hardware, training method and etc. After upgrade of this model to TAM II, new theoretical structures were added to the TAM model. Regarding to interplay of these two basic structures on Trust and effect of trust on tend to use technology we can introduce trust as the third affecting structure on this model. According to research done by Zhohavi Dong and his colleagues, factors such as Trust, service quality, perceived value (functional, emotional, social and financial) and costs for are considered as factors influencing customer loyalty and satisfaction (see Fig. 4) Hence the usefulness of Trust and Perceived ease of use and perceived value creates customer satisfaction that this trust has a significant impact on customer satisfaction and tend to use technology and Finally leads to two different sense of loyalty and technology adoption ore on the other side, will make customer to complain that causes customer temporarily dismissal or possibly will Completely separate him of technology.

10 Conclusion Economic life of all financial institutions continues with investment and support of customers. In launching e-banking systems should pay special attention to the customers. Due to global extent of business competition, an Institution will be

Offering a Conceptual Model of Technology Acceptance in e-Banking

423

successful in attracting more customers that provides its services in high quality and quantity. But, to fulfill this need, these institutions should respect important principles that always focuses on creating Satisfaction and confidence among customers. Decades of electronic banking passed, and there is still the concern whether this technology can be fully accepted by customers. For this reason, various researches has been done about the electronic banking. One of the ways to evaluate acceptance of new technology is the TAM model. In theoretical part of this research relationship between technology acceptance and the satisfaction and trust was widely described and then the relationship between satisfaction and trust introduced. These issues show that there is a close relationship between these two structures and this model. Finally, Due to studies done by researchers in other researches, expressed a combined model obtained from trust and satisfaction which are important in acceptance of electronic banking.

References Journal 1. Al-Hajri S, Tatnall A (2008) Technological innovation and the adoption of internet banking in Oman. Electron J Virtual Organ Netw (eJOV) 10:59–83. Special issue on living labs, August 2008 2. Alsajjan B, Dennis C (2010) Internet banking acceptance model: across-market examination. J Bus Res 63:0148–2963, forthcoming. http://www.arraydev.com/commerce/JIBC/031101.htm 3. Berry LL (1995) Relationship marketing of services-growing interest: emerging perspectives. J Acad Mark Sci 23(4):236–2450 4. Crosby LA, Evans KR, Cowles D (1990) Relationship quality in services selling: an interpersonal influence perspective. J Mark 54(3):68–81 5. Nah F, Davis S (2002) HCI research issues in electronic commerce. J Electron Commer Res 3(3):98–113 6. Geyskens I, Steenkamp JBEM, Scheer LK, Kumar N (1996) The effects of trust and interdependence on relationship commitment: a trans-Atlantic study. Int J Res Mark 13(4):303–317 7. Kanthawongs P (2011) A structural model of student satisfaction in web-based ERPsimulated learning environments. Int J Educ Inf Technol 5(2): 8. Klopping IM, Mckinney E (2004) Extending the technology acceptance model and tasktechnology fit model to consumer e-commerce. Inf Technol Learn Perform J 22(1)35–48. Spring 9. Rigopoulos G (2007) A TAM framework to evaluate user’s perception towards online electronic payments. J Internet Bank Commer (JIB) 12(3):1–6. December. http:// www.arraydev.com/commerce/jibc/2007-12/George_final.pdf 10. Schoder D, Madeja N (2004) Is customer relationship management a success factor in electronic commerce? J Electron Commer Res 5(1):38–53 11. Yousafzai SY, Foxall GR, Pallister JG (2010) Explaining internet banking behavior: theory of reasoned action, theory of planned behavior, or technology acceptance model? Cardiff Univ J Appl Soc Psychol 40(5):1172–1202

424

A. Dehghan et al.

12. Wan WWN, Luk C-L, Chow CWC (2005) Customers adoption of the banking channels in Hong Kong. Int J Bank Mark 23(3): 13. Zhaohua D, Yaobin L, Kwok KW, Jinlong Z (2010) Understanding customer satisfaction and loyalty: an empirical study of mobile instant messages in China. Int J Inf Manag 30:289–300

Book 14. Gefen D (2000) E-commerce: the role of familiarity and trust. Omega 28(6):725–737 15. Hapson J, Logari B (2005) Service management, customer-oriented culture. In: Irannezhad Parizi M (ed) Interpreter, 2nd edn. Modiran, Tehran 16. Kripanont N (2007) Examining a technology acceptance model of internet usage by academics within Thai business schools. Victoria University, Melbourne March 17. Venoos M, Safariyan D (2001) Marketing banking services. Negahe Danesh, Iran

Conference Paper or Contributed Volume 18. Abdolah Poor M, Kaviyani A (2010) Explain the role of online service quality in Internet banking and its impact on customer satisfaction. The set of articles for 2nd international conference on financial services marketing, Tehran, Iran 19. Ashrafi Joo B, Shojaei M (2008) Elimination infrastructure challenges for the development of e-Banking. The set of articles for 2nd international conference on e-Banking, Tehran, Iran 20. Bashiri M, Joneidi M (2007) Effect of trust on customer acceptance of internet banking, based on TAM model, Mellat bank case. The set of articles for 2nd international conference on eBanking, Tehran, Iran 21. Chismar WG, Wiley-Patton S (2002) Does the extended technology acceptance model apply to physicians. The 36th Hawaii international conference on system sciences (HICSS’03), vol 36, no 03, 2002 IEEE 22. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 13(3):319–340 23. Deng Z, Lu Y, Chen Z (2007) Exploring Chinese user adoption of mobile banking. The sixth Wuhan international conference on e-business. e-Business track, Wuhan, China, pp 176–182, May, pp 26–27 24. Grazioli S, Jarvenpaa SL (2000) Perils of Internet fraud: an empirical investigation of deception and trust with experienced internet. IEEE Trans Syst man Cybern A Syst Hum 30(4):395–410 25. Kotler P, Armstrong G, Saunders J, Wong V (2001) Principles of marketing, 3rd edn. Veronica, Prentice Hall 26. Mayer RC, Davis JH, Schoorman FD (1995) An integrative model of organizational trust. Acad Manag Rev 20(3):709–734 27. McKnight DH, Choudhury V, Kacmar C (2002) Developing and validating trust measures for e-commerce: an integrative typology. Inf Syst Res 13(3):34–59 28. Misztal BA (1996) Trust in modern societies: the search for the bases of social order. Polity Press, Cambridge 29. Ratnasingham P (1998) The importance of trust in electronic commerce, internet research: electronic networking applications and policy 8(4):313–321 30. Venkatesh V, Davis FD (2000) A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci 46(2):186–204

Part II

Computer Convergence

Architecture for Secure SCADA System Based on IF-MAP Scheme Chuan Deng, Lin Yang, Linru Ma and Yuan Cao

Abstract SCADA system is the brain of ICS. Along with more and more interconnection, many security challenges have risen up. Protection SCADA system is significant critical to national infrastructure. Based on IF-MAP scheme published by TCG, we research the integration of IF-MAP and SCADA system, present an architecture for secure SCADA system by utilizing IF-MAP function of publish, subscribe and search network metadata in real-time, to unify and coordinate all the potential loosing security and safety entities, implementing defense-in-depth and defense-in-coordination. Keywords SCADA


IF-MAP
Coordination
Security

C. Deng (&) Institute of Communication Engineering, University of Science and Technology of PLA, Nanjing, People’s Republic of China e-mail: [email protected] L. Yang
L. Ma Institute of Electronic Equipment System Engineering Corporation of China, Beijing, People’s Republic of China e-mail: [email protected] L. Ma e-mail: [email protected] Y. Cao National Defense Technology University, Changsha, People’s Republic of China e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_41,  Springer Science+Business Media B.V. 2012

427

428

C. Deng et al.

1 Introduction SCADA systems, as the brains of the national critical infrastructures, such as water distribution and wastewater collection systems, oil and gas pipelines, electrical utility transmission and distribution systems, and rail and other public transportation systems, have become a major security concern recently. The widespread interconnectivity of these industrial control systems with enterprise IT networks, combined with legacy systems that can’t be easily patched due to operational considerations and protocols never designed for security, have triggered a potential attacks from various adversaries. A sophisticated targeted ICS attack started by Stuxnet [1] worm in July 2010 has greatly shocked ICS engineers and IT security professionals. Even though there has been numerous articles and guidelines published to aid SCADA system, considering the significant role of it, now available security schemes for SCADA are far from preventing system from all the discovered vulnerabilities and potential penetration. Complicating the situation, most of these SCADA devices and protocols have little or no security capability on their own and will take decades to be replaced due to cost factor. Common security solutions applied to SCADA system are limited to paired firewall between corporate network and control network, simple authentication by password, VPN for remote access, IDS for abnormal flow and others. What worse, in order to mitigate influence with routine operation, some of these security solutions are configured as weak as possible. For example, password in HMI cannot be set very long or complex for sake of emergent cases. And firewall policy is configured to avoid blocking any instruction data flow. These practical paradigms may take the essential difference between SCADA and legacy IT system into consideration. But it is obviously disobey the original principles for SCADA security protection. How to secure SCADA and still leave the system reliable enough to provide critical services such as power or water 24 9 7 9 365 is worthy of endeavor. IF-MAP Binding for SOAP specification version 2.0 [2] and IF-MAP Metadata for Network Security version 1.0 [3] published by TCG in July 2010, provide a possible robust solution to satisfy both reliability and security, which can be used seamlessly with legacy SCADA equipment. By introducing a MAP server, IFMAP enhanced equipments can provide rapid information sharing and notification of problems (metadata of security) around whole the system. Utilizing these metadata of security, all security assets and even physical safety equipments can coordinate to establish a real Defense-in-Depth. It is an ideal match for industrial control systems where time is of the essence. While our framework is a Proof of concept, Byres Security teamed with Juniper Networks to secure SCADA with Tofino [4] solution has made some practical progress. This paper briefly shows new implication of IF-MAP introduced by TCG in Sect. 2, presents a novel framework applying IF-MAP to protect SCADA system in Sect. 3, and demonstrate some possible use cases in Sect. 4 to prevent ‘‘Possible

Architecture for Secure SCADA System Based on IF-MAP Scheme

429

Incident Scenarios’’ by some use cases. At last we conclude our paper and future work.

2 IF-MAP Background In 2008, the Trusted Computing Group’s Trusted Network Connect Working Group (TNC-WG) released its initial IF-MAP (Interface for Metadata Access Points) specification. This open standard extended the TNC architecture for network security to support standardized, dynamic data sharing through Simple Object Access Protocol (SOAP) exchanges among a wide variety of networking and security components. The initial version of IF-MAP enabled integration of network security functions such as network access control (NAC), remote access, intrusion detection, endpoint profiling, behavior monitoring, data leak detection, etc. With several suppliers shipping IF-MAP support in their products, a number of end users have successfully piloted and/or deployed production systems based on IF-MAP 1.1. But in the last two years, customers and suppliers have discovered new uses for IFMAP. Because the protocol is highly extensible, they’ve been able to apply its publish and subscribe semantics to fields such as cloud computing, Industrial Control Systems and SCADA security, physical security, and more. To enable and ease new applications of IF-MAP, TCG has now separated IFMAP into a base protocol (IF-MAP 2.0) and a set of network security metadata (IF-MAP Metadata for Network Security 1.0). Now innovators can employ the base protocol without having to worry about the network security aspects. Because MAP servers are metadata-independent, they can be used for any application. Innovators only need to develop IF-MAP client code for their application. Products with IF-MAP support currently available from suppliers include the Great Bay Beacon endpoint profiler; Juniper Networks Unified Access Control (UAC) and SSL VPN appliances (SA); Infoblox DHCP Server and Orchestration Server; Insightix BSA Business Security Assurance suite; Lumeta IPsonar active network discovery solutions; Hirsch Electronics Velocity Physical Access Control System and Byres Security Tofino industrial security gateways. Following Fig. 1 describes MAP role in TNC v1.4 architecture. Here Metadata Access Point acts as a MAP sever, supporting IF-MAP protocol and real-time ‘‘meta-database’’. While all the other elements interacting with MAP server using IF-MAP protocol to publish, search or subscribe metadata, are classified as MAP clients, which include firewall, IDS, AAA, VPN gateway, GPS, badge readers and so on. Of course, MAP scheme can extend beyond TNC architecture, such as integrated into SCADA system, only if there are network elements supporting IFMAP protocol [5]. It is really a big challenge to coordinate all the security and safety elements in the SCADA system.

430

C. Deng et al.

Fig. 1 IF-MAP scheme used in TNC architecture

• Security and safety infrastructure is complex, heterogeneous, and usually distributed. • Large real-time data flows between infrastructure components. • Needed for coordination between Sensors, Flow Controllers, PDP, physical safety assets etc. • Components often interested in different patterns and events. • Timely routing and reliability of delivery of metadata is critical for coordination. • Coordination data is loosely structured and changes frequently. MAP is specifically designed to fit the security coordination use case. MAP models all system elements into identifiers (all objects are represented by unique identifiers), links (Connote relations between pairs of objects), and Metadata (attribute containers attached to Identifiers or Links). All identifiers and links exist implicitly, but have no meaning until metadata is attached to them. IF-MAP operations and data types are represented as XML documents. Here in Fig. 2, identifiers are represented by ovals, metadata is represented by rectangles, and links are represented by lines connecting identifiers. It describes all the data related with a ‘‘access request’’ originated by an ‘‘identity’’ using a ‘‘device’’ with ‘‘MAC-address’’ and a leased ‘‘IP-address’’, through ‘‘switch device’’ and ‘‘PDP device’’, authorized with a ‘‘capability’’. All MAP clients publish their own related metadata to MAP server, which in turn integrated them into such like a paradigm, supported other element’s subscription and search operation.

Architecture for Secure SCADA System Based on IF-MAP Scheme

431

Fig. 2 Paradigm of IF-MAP data model

3 IF-MAP in SCADA System Industrial control systems such as SCADA that control critical infrastructure operation has evolved from original isolated systems running proprietary control protocols using specialized hardware and software into nowadays more and more interconnecting systems running standard control protocols using commercial hardware and COTS. The Smart Grid is an excellent example of industrial systems connecting using Internet protocols. In this case, power plants and the electrical distribution grid are connected to the internet. Unfortunately, an unprotected system on a network is vulnerable to attack by hackers. Even industrial systems that are connected internally but not connected to the Internet still use Internet protocols or others that are even more vulnerable. This makes it relatively easy for someone to hack those systems. As a result, industrial controls systems and the Smart Grid environment can benefit from IF-MAP. IF-MAP can ensure that the only device on the network that can control a particular critical load such as a chemical pump is the device that is supposed to control that pump. IF-MAP provides rapid information sharing and notification of problems; it is an ideal match for industrial control systems where time is of the essence. As a possible approach to integrate IF-MAP into SCADA system, a MAP server should exist in control system LAN, which is responsible for storing published metadata and responding clients’ search, subscribe and poll operation. Infoblox’s

432

C. Deng et al. Enterprise Network

HMI

Worksation Internet VPN Concentrator Web Server Remote Workstation

DHCP

Data Histrorian

IDS

SCADA Control Access Network

Directory Sever

MAP Server

Posture Validation server

SEM

Audit Server

AAA

SCADA Control Network

Data Server

Worksation

PLC

HMI Communication Network

RTU SCADA Server

Physical Safety

Web Server

Fig. 3 Framework of IF-MAP enabled SCADA system

IBOS (Infoblox Orchestration Server) and Juniper Network’s IC Series Unified Access Control Appliance have been available as a MAP server, while IRON project from FHH and OMAPD project support open-source MAP server. Then MAP clients can be any entity in the control system, as long as they support IFMAP. In fact, MAP client may include IDS, Firewall, SEM/SIM, DHCP, physical safety devices and so on. ARUBA, Great Bay, Juniper, Lumeta, Tofino have provide related products as MAP client. While there also exists several opensource IF-MAP client developed in language of JavaScript, Perl, and C/C ++ [9]. Thus we have designed a framework of SCADA system integrated with IFMAP scheme, and implemented a prototype to demonstrate how effective it can be as to coordinate with every IF-MAP-ware entity to improve security protection of SCADA system. Figure 3 describes this framework, in which MAP clients are embedded with an IF-MAP protocol enabled client agent. To entity in legacy SCADA system, this IF-MAP protocol enabled client agent can be implemented as an independent End box working together with host entity; while in new designed system, entity can be designed with IF-MAP protocol enabled client agent from the ground.

Architecture for Secure SCADA System Based on IF-MAP Scheme

433

Considering the trend of interconnection of enterprise network and SCADA control network, we equip MAP server in SCADA control access network, which is responsible for analysis of all the access requests from enterprise network (or even from Internet) to SCADA control network. When there is a new access request from a relative new agent (periodically), AAA server in access control network will analyze this agent according to security policy, based on Directory Server, Posture Validation Server, and Audit Server. Actually, this architecture forms a Trusted Network Access, as the TCG TNC v1.4 specification demonstrated. In this architecture, AR means access requestor from enterprise network and internet. PEP (Policy Enforcement Point) is the switch with firewall. PDP (Policy Determination Point) stands for AAA server and following Directory Server, Posture Validation Server, and Audit Server. MAP server is a new added server. MAP clients can be all the Sensor and Flow Controller in the scope of whole architecture, like IDS, DHCP, SEM, AAA, Data Historian, wireless access point and so on. Details about how to establish a trusted SCADA system is out of scope of this paper, which will focus on integration IF-MAP protocol with SCADA system and some use cases. We can also refer some advance in the related research topics, like a Scalable Trusted Software Dissemination System [10], to help the collection of proof of system status, Trusted Computing Dynamic Attestation Using a Static Analysis based Behavior Model [11], to collecting dynamic proof of system status, Integrated management platform for seamless services provisioning in converged network [12], to converge multi-system information to get integrated cooperation, and a secure communication in integrated heterogeneous wireless networks [13] to enhance the communication between wireless access point and central control point.

4 Some Use Cases Our new IF-MAP-based SCADA security system allows corporate-wide security policy to be easily deployed for SCADA devices that simply don’t know what security is. It lets the information driving these policy decisions be dynamically determined from a large number of real-time sources, like Active Directory servers, SCADA equipment positions or network security events. Use Case I: AAA server cooperates with IDS AAA server can leverage abnormal activity information discovered by IDS for real-time adjustment of access control policy. First, AAA establishes a subscription session with MAP server by IF-MAP protocol, to subscribe all the potential access requestor’s abnormal activity information. In the same way, IDS establishes a publication session with MAP server. During the running of SCADA system, IDS works to detect intrusion. Given an intrusion actually being discovered, IDS publish this information in an XML format file to MAP server. MAP server then relay this message to all the interesting party, including AAA server. Now AAA

434

C. Deng et al.

server knows there bursts an intrusion based on a specific entity or from such a platform. Then AAA server informs switch to deny any new access request from this entity or platform, to prevent any potential undermining activity to control system. Use Case II: Firewall cooperates with Physical Safety By IF-MAP scheme, traditional industrial physical safety equipment can work together with new information security equipment, like physical badge reader cooperating with firewall. The way to establish communication session with MAP server is the same as Use CaseI. Physical badge reader publishes metadata about employee who has passed the physical badge reader using their eID card to enter control center. When establishing a new control session, firewall searches the eID in the control session is or is not included in the eID list that physical badge reader has published. Then firewall can determine whether to allow or deny this control session establishment. Thus, this is a novel way to integrate physical safety and network security into a coordination fashion for control system protection.

5 Conclusion IF-MAP schema, as this paper demonstrates above, can bring a new sight of pervasive security and unified security, only if more and more network entity began to adopt IF-MAP-ware equipment. Based on IF-MAP sophisticated adoption, many more novel applications can be surely developed. Until now, Juniper and Infobox Corporation have developed MAP server, and Arcsight, ARUBA, LUMETA, have already IF-MAP-supported productions as MAP client. Acknowledgments This project work is sponsored by the National High Technology Research and Development Program of China (863 Program), under Grant No. 2009aa01z426.

References 1. Stuxnet (2010) A breakthough, http://symentac.com/connect/blog/stuxnet. Accessed 29 Nov 2010 2. IF-MAP Binding for SOAP specification version 2.0 (2010) http://trustedcomputinggroup.org . May 2010 3. IF-MAP Metadata for Network Security version 1.0 (2010) http://trustedcomputinggroup.org. July 2010 4. Tofino, Byres Security teams with Juniper Networks to Secure SCADA with Tofino, http:// tofinosecurity.com/blog/tags/tofino 5. TNC-RSA-2010 workshop, TNC everywhere, Pervasive Security 6. Eric Byres, Scott Howard, Metadata Access Point (IF-MAP), New Standards Protecting Old Technology, Oct 2010 7. Clifford N (2010) Understanding trust and security in SCADA systems. In: Second workshop on advances in trusted computing (WATC’10), Tokyo, Japan, Nov 2010

Architecture for Secure SCADA System Based on IF-MAP Scheme

435

8. Okhravi H (2010) Trusted_and_High_Assurance_Systems. Dissertation, University of Illinois at Urbana-Champaign 9. Open source IF-MAP project, http://www. ifmapdev.com 10. Chuan D, Lin Y, Linru M, Yuan C Towards a practical and scalable trusted software dissemination system. JoC 2(1):53–60 11. Li T, Yu F, Lin Y, Kong X, Yu Y Trusted computing dynamic attestation using a static analysis based behaviour model. JoC 2(1):61–68 12. Kryvinska N, Van Thanh D Strauss C (2010) Integrated management platform for seamless services provisioning in converged network. IJITCC 1(1):77–91 13. Xie B, Kumar A, Zhao D, Reddy R, He B On secure communication in integrated heterogeneous wireless networks. IJITCC 1(1):4–23

Research on the Model of the Survivable Storage System (SSS) and Its Architecture Xiao-yuan Yang, Jing-jing Wang and Xiao-feng Chen

Abstract In view of the present inconsistent understanding of the model definition of survivable storage network, a seven-tuple model definition and a kind of architecture of SSS were given, taking the characteristics, such as wide-open style and high intricacy, of the actual network environment into account. In addition, some detailed analysis to the key elements of the model and the modules of the architecture was made separately. Our research can provide guidelines for design and construction of SSS, besides, it can also provide basic methods for evaluating the survivability of a storage system. Keywords SSS

 Seven-tuple model  Architecture

1 Introduction Since 1993 Neumann et al. proposed the definition of network system survivability until now, research on the survivable storage has been lasted for nearly two decades. However, there are lots of issues of network system survivability still X. Yang (&)  J. Wang (&) Key Laboratory of Network and Information Security of CAPF, Engineering College of CAPF, Xi’an 710086, People’s Republic of China e-mail: [email protected] J. Wang e-mail: [email protected] X. Yang  X. Chen (&) Key Laboratory of Computer Networks and Information Security of the Ministry of Education, Xidian University, Xi’an 710071, People’s Republic of China e-mail: [email protected]

James J. (Jong Hyuk) Park et al. (eds.), Computer Science and Convergence, Lecture Notes in Electrical Engineering 114, DOI: 10.1007/978-94-007-2792-2_42, Ó Springer Science+Business Media B.V. 2012

437

438

X. Yang et al.

need further and more researches. Yang et al. [1] found that there are only a few of the relevant thesis, which about the survivable storage, have studied the problem of the definition of the network systems survivability by using the methodology proposed in paper [2]. Furthermore, they drew a conclusion that a unified, standardized definition of network systems survivability has not been formed in the academic world. In the outcomes already achieved in the researches till now, Knight Research Group of the university of Virginia proposed the formalized definition of the network survivability, which is respectively a four-tuple model [3] and a six-tuple model [4]. But they didn’t give the genericity of the system survivability. An eight-tuple formalized definition of the survivable system was proposed in paper [2]. However, it didn’t take the time factor into account. In paper [5], the time factor was considered, then a seven-tuple formalized definition and its application examples were given. But the key elements such as actual work environment of the network, different users’ requirements and so on. Furthermore, a five-tuple formalized definition was put forward in paper [6], combining with the thoughts of intrusion masking and defense-in-depth. It’s a pity that the author didn’t pay attention to the characteristics, open-wide style and high intricacy, of the practical network environment. What’s more, the indication of the work environment, the analysis of survivability requirements and the vague trust relationships between the nodes didn’t considered in its model definition. Therefore, the model definition of SSS is worthy of further study, and its corresponding architecture also needs to be developed and perfected. A seventuple formalized definition of SSS and its architecture are given in this paper based on the previous research outcomes. Besides, both of the definition and architecture are analyzed respectively.

2 A Seven-Tuple Model of SSS 2.1 The Characteristics of the Open Network Environment At present, the bounded network is no longer the mainstream. The open large-scale network system which has no boundary has become the major data service object. Compared with bounded network environment, open large-scale storage network basically has the features as follows: 1. The number of the storage nodes is huge and changing. Besides, faults are unavoidable for the storage nodes; 2. The storage environment always be threatened and attacked. Even some devastating disasters will happen; 3. The network is highly distributed, and there is no one, neither any organizations or any individuals, have the entire management authority;

Research on the Model of the SSS

439

4. Each node doesn’t fully know the situation of the whole network or the other nodes; 5. The nodes are not all completely credible. The mutual-operations between different nodes are decided by the protocol between them.

2.2 The Seven-Tuple Model Definition Focus on the characteristics of the open network environment, here we give the seven-tuple model definition of SSS as follows, based on the five-tuple model definition [6] which was proposed in 2008 by Zhang. Definition 1 A survivable storage system can be defined as a 7-tuple. The Definition of the seven-tuple \ E, S, D, C, F, A, R [. E is the specific description of the storage system’s work environment, including the sets of storage nodes (servers) and users, resources distribution and the system or the users’ requirements or needs, etc. S is the set of services which are provided for users by storage system. D is stands for the technologies and algorithms of data processing, such as the data dispersal algorithm used in the process of data shares storage. C is the strategy of choosing storage nodes. F is the set of different system-fault types, including the disposal methods. A is the protocols of access control and authentication between storage nodes (servers) and users. R is the protocol of data-reconstruction. In the definition above, the elements which belong to the tuple model interact with each other, so as to realize the survivability of the data services in the storage system. In addition, the importance of some elements should be stressed under control, according to the practical requirements of the system and users.

2.3 The Elements Analysis 2.3.1 Work Environment E In the storage system, E is the description of its work environment, including the set of user-nodes, the set of server-nodes, practical natural conditions of the environment, software and hardware configuration, the specifications of system and users’ requirements and needs, etc. All these factors can be expressed by different sets, of which there are many concrete factors. Good description of the work environment is helpful for the designer to achieve an overall and detailed analysis for the features of the storage system, but also helpful to refine the requirements definition of the system and users accurately. At the same time, good description is conducive to carry on

440

X. Yang et al.

targeted tests of the storage system, providing better basis so as to realize its survivability further.

2.3.2 Services Provided by System S Form the service-orientated perspective, the system mainly provides data services to its users, such as accessing, deleting, updating the files and so on. The data files can be text, image, audio, video and other multimedia files. If S ¼ fs0 ; s1 ; . . .; sn gis the set of all the services can be provided by the storage system, each element si ði  0; i 2 Z Þin S stands for one kind of service provided by it. Because of the users’ different identities in reality, their corresponding services have different priorities, which denoted as G ¼ fg0 ; g1 ; . . .; gm g. Each element gi ði  0; i 2 ZÞin G stands for the priority of one certain user’s service provided by the system. For every service provided by the system, there is a particular probability to it according to the user’s requirement or need for the survivable system and the priority of it. Supposing the probability is denoted as P : jpi 2 Rj0  pi  1; i  0; i 2 Z j, and R stands for the real number. So the reliability of the certain service ended can be defined as Pi. P is one of the standards to judge whether the system realizes its survivability. From the perspective of system maintenance, the services are composed by a series of non-basic and basic services [7]. When the system is attacked, it needs to suspend some non-basic services, release the resources occupied by them, and start some appropriate services to replace them. So as to weaken the influences brought by the attacks, provide uninterrupted services to the users, and maintain the system’s survivability.

2.3.3 Technologies and Algorithms of Data Processing D In the open network environment, the users always take care of the quilities of the data services much more. In order to provide timely and sustainable services, we should first process the mass data files, then stored them distributely in the netwrok. At present, the data processing algorithms usually adopted are as follows: (1) Backup and data-mirror Backup is the most basic and direct method [8]. Remote backup can ‘save the life’ of the data files at the critical moment. However, it still has disadvantages of information leak and occupying too big storage space. If combining backup with secret sharing and erasure code, etc., we can enhance the survivability of data services greatly. (2) Secret sharing scheme Secret sharing scheme is proposed by Blakley [9] and Shamir [10] first, including threshold secret sharing scheme and non-threshold secret sharing scheme, which are all common methods to protect data files used in the research of SSS.

Research on the Model of the SSS

441

Threshold secret sharing scheme has the advantage that once some shares have been modified, the probability of obtaining semantically meaningful information is very small if these modified shares participate in the recovery process. Take (k, n) threshold secret sharing scheme for example, the original secret only can be recovered by any k or more than tðm  tÞ participants sharing their secret shares. And the secret couldn’t be recovered by less than k participants. In addition, we always adopt short-share secret sharing scheme to adapt the mass ocean-store. If consider the unreliability of the secret distributor and the share keepers, we may adopt the verifiable secret sharing schemes or other distributed authentication schemes, so that the user can find out which nodes’ shares have been modified. Thus he can avoid obtaining shares from these nodes and start updating promptly. (3) Erasure coding First, erasure coding divides the data file into m segments. Then it transforms the m segments into nðm  nÞ shares, in which any tðm  tÞ shares can restore the original data file. Compared with the threshold secret sharing scheme, erasure code more emphasize the data file’s reliability. Till now, there are lots of data processing algorithms based on erasure codes, such as Information Dispersal Algorithm (IDA) [11] proposed by Rabin, which is very classical. Above all, when we design a data processing algorithm, it is necessary to consider the efficiency of data distributed storage, but also prevent the information from leaking or losing. In the research of SSS, it is an important problem that designing a safe and high-efficiency data processing algorithm, combining with the knowledge of complexity theory and information theory.

2.3.4 Strategies of Choosing Storage Nodes C Data services are provided together by the storage nodes in the distributed storage system, so all the services relate to the problem of nodes selection [6]. The key point of data storage is the selection of storage nodes, and the strategy of nodes selection should take the factors such as nodes’ reliabilities, whether the operating systems have the same structure, geographical environment, hardware configuration, whether cross different networks, and the remaining available storage space etc. into account comprehensively. In addition, during the process of providing data services, the storage network should change constantly the node selection strategy according to the quality of services requested and the number of updated share as have errors. What’s more, four common strategies of storage nodes selection were proposed in paper [12]. And it obtained a conclusion through experiments of the survival rate and space utilized rate. Namely, the first strategy, choosing nodes randomly, is the best storage strategy. Combining with the actual situation of the network to choose the appropriate storage nodes, so as to reduce the burden of updating shares, improve the efficiency and provide the protection for continuous and timely data services.

442

X. Yang et al.

In the storage system of network, besides the text files, the plentiful multimedia files, such as images, are also very important. Through the conclusion above, in paper [13], we put forward a new kind of image survivable storage scheme combing with hyperchaotic. Because there are four pesudo-random sequences fX; Y; Z; U g in the hyperchaotic system, and after quantification they can be denoted as fx; y; z; ug. In order to realize the image survivable storage, this scheme first uses the x-sequence to scramble the rows and columns of binary-layers of the image so as to achieve its encryption. Then it uses the y-sequence to divide the encrypted image into a random number of sub-blocks, uses the z-sequence to choose the sub-blocks randomly, and uses the u-sequence to choose the storing nodes randomly. At last, all the sub-blocks of the image are stored respectively in the random selected storing nodes. The experiment simulations in our paper [13] can prove this scheme can fully realize choosing storing nodes randomly, providing the security and the disater recovery of the stored images at the same time. In addition, some optimization analysis were also done by two different cases. What’s more, according to the conclusions of optimization analysis given, not only can the server determine how many subblocks should be divided into for an image, but also can judge the storage network whether meets the basic survivable requirements.

2.3.5 System Fault Types and Disposal Methods F In the research of survivability, the faults in the system are always divided into three types [6]: crash faults, symmetric faults and Byzantine faults. Crash faults generally mean faults of the hardware or the communication lines, leading the data be transmitted abnormally. Symmetry faults are not easy to be discovered by users. Although the normal service provided by the system still can be carried out continuously, the data provided is malicious or invalid information, because the data has been modified by the behavior of attack or intrusion. Byzantine faults stand for random faults, namely the behavior of the nodes which have faults is random. Thus they can be regarded as software faults, including the disruption of the normal services brought by the attacker’s intrusion. Because when symmetric faults happen, the data is modified, and the user couldn’t obtain accurate information, which is similar to the situation that the system couldn’t provide normal data restore services. Therefore, in the storage system, the symmetry faults are same as Byzantine faults, which means the faults can be divided into two types directly: Byzantine faults and crash faults. In SSS, if (k, n) threshold secret sharing scheme is adopted to data dispersal, then the user must send requests of read to k storage nodes so as to restore the data. Supposing there are only two types of faults in SSS, and there are t nodes which have faults, of which there are c crash faults and b Byzantine faults. Then, the worst situation is that the t nodes have faults are all included in the nodes which the user has sent request of read to. Thus, the accurate information cannot be recovered. At this moment, the user should re-choose t nodes from the rest

Research on the Model of the SSS

443

n-k nodes, and send access request again. Therefore, for this kind of system, we must request n  k  t [6]. Besides, the situation of the system adopting erasure codes is similar to the one adopting threshold secret sharing schemes. In the open network environment, malicious attacks, natural disasters, accidents, computer viruses, the ageing hardware of the node itself and so on all may lead the storage nodes go wrong in the system. Ideally, the mechanism of intrusion or fault detection should be able to forecast the possibility of the fault which will happen in the storage system, and be able to repair the system or eliminate the possibility before the fault happens. However, such kind of technique still