Higher Education Institutions and Learning Management Systems: Adoption and Standardization

Higher Education Institutions and Learning Management Systems: Adoption and Standardization Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal

Senior Editorial Director: Director of Book Publications: Editorial Director: Acquisitions Editor: Development Editor: Production Editor: Typesetter: Print Coordinator: Cover Design:

Kristin Klinger Julia Mosemann Lindsay Johnston Erika Carter Myla Harty Sean Woznicki Jennifer Romanchak Jamie Snavely Nick Newcomer

Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com Copyright © 2012 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Higher education institutions and learning management systems: adoption and standardization / Rosalina Babo and Ana Azevedo, editors. p. cm. Includes bibliographical references and index. Summary: “This book provides insights concerning the use of learning management systems in higher education institutions, to increase understanding of LMS adoption and usage”--Provided by publisher. ISBN 978-1-60960-884-2 (hardcover) -- ISBN 978-1-60960-885-9 (ebook) -- ISBN 978-1-60960-886-6 (print & perpetual access) 1. Internet in higher education--Case studies. 2. Education, Higher--Computer-assisted instruction-Case studies. 3. Web-based instruction--Design. I. Babo, Rosalina, 1967II. Azevedo, Ana, 1977LB2395.7.H55 2012 378.1’7344678--dc23 2011022151

British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.

Editorial Advisory Board António Andrade, Universidade Católica Portuguesa, Portugal Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Schiopoiu Burlea, University of Craiova, Romania Luís Borges Gouveia, University Fernando Pessoa, Portugal Paulo Coelho Oliveira, Instituto Superior de Engenharia do Porto, Portugal Demetrios G Sampson, University of Piraeus, Greece Steve Wheeler, University of Plymouth, UK

List of Reviewers Kamla Ali Al-Busaidi, Sultan Qaboos University, Sultanate of Oman António Andrade, Universidade Católica Portuguesa, Portugal Judy van Biljon, University of South Africa, South Africa Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Burlea, University of Craiova, Romania Nicholas Caporusso, Institute for Advanced Studies, Italy Lai Yiu Chi, The Hong Kong Institute of Education, Hong Kong Thomas Connolly, University of the West of Scotland, UK Dorota Dżega, West Pomeranian Business School, Poland José Manuel Mesa Fernández, University of Oviedo, Spain Robert W. Folden, Texas A&M University-Commerce, USA Jose Albors Garrigos, Universidad Polytecnica de Valencia, Spain Luís Borges Gouveia, University Fernando Pessoa, Portugal Malinka Ivanova, College of Energetics and Electronics, Bulgaria Arpan Jani, University of Wisconsi, USA Alexandros Karakos, Democritus University of Thrace, Greece Ray Kekwaletswe, Tshwane University of Technology, South Africa Arne Wolf Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany Atik Kulakli, Beykent University, Turkey

Jack Lee, The Chinese University of Hong Kong, Hong Kong Carla Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Lourdes Moreno, Universidad Carlos III de Madrid, Spain Muhammad Abdul Mugeet, Aga Khan University, Pakistan Lino Oliveira, Escola Superior de Estudos Industriais e de Gestão, Portugal Paulo Oliveira, Instituto Superior de Engenharia do Porto, Portugal Wieslaw Pietruszkiewicz, SDART Ltd, UK Mário Pinto, Escola Superior de Estudos Industriais e de Gestão, Portugal Ricardo Queirós, Escola Superior de Estudos Industriais e de Gestão, Portugal Marina Ribaudo, University of Genova, Italy Sandra Ribeiro, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, Escola Superior de Estudos Industriais e de Gestão, Portugal Lorenzo Salas-Morera, University of Córdoba, Spain Demetrios G. Sampson, University of Piraeus, Greece Anthony Scime, State University of New York, USA Errikos Ventouras, Technological Educational Institution of Athens, Greece Steve Wheeler, University of Plymouth, UK

Table of Contents

Foreword............................................................................................................................................... xv Preface.................................................................................................................................................. xix Acknowledgment...............................................................................................................................xxiii Section 1 Generalities and Perspectives Chapter 1 General Perspective in Learning Management Systems.......................................................................... 1 Robert W. Folden, Texas A&M University-Commerce, USA Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness......... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course............................ 50 Thomas Connolly, University of the West of Scotland, UK Carole Gould, University of the West of Scotland, UK Gavin Baxter, University of the West of Scotland, UK Tom Hainey, University of the West of Scotland, UK Section 2 Implementing and Evaluating Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs........................................................... 75 Paul Lam, The Chinese University of Hong Kong, Hong Kong Judy Lo, The Chinese University of Hong Kong, Hong Kong Jack Lee, The Chinese University of Hong Kong, Hong Kong Carmel McNaught, The Chinese University of Hong Kong Hong Kong

Chapter 5 ANGEL Mining..................................................................................................................................... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems.........116 Kamla Ali Al-Busaidi, Sultan Qaboos University, Oman Hafedh Al-Shihi, Sultan Qaboos University, Oman Section 3 Trends and Challenges Chapter 7 A Comparative Study on LMS Interoperability................................................................................... 142 José Paulo Leal, CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós, CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal Chapter 8 Mobile Learning Management Systems in Higher Education............................................................. 162 Demetrios G. Sampson, University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas, University of Piraeus & Centre for Research and Technology Hellas, Greece Chapter 9 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires................. 178 Dimos Triantis, Technological Educational Institution of Athens, Greece Errikos Ventouras, Technological Educational Institution of Athens, Greece Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility..........199 Lourdes Moreno, Universidad Carlos III de Madrid, Spain Ana Iglesias, Universidad Carlos III de Madrid, Spain Rocío Calvo, Universidad Carlos III de Madrid, Spain Sandra Delgado, Universidad Carlos III de Madrid, Spain Luis Zaragoza, News Service, Radio Nacional de España, Spain Chapter 11 The Technological Advancement of LMS Systems and E-Content Software..................................... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK

Section 4 Case Studies Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education..................................... 247 Rosalina Babo, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, NID-RH, ESEIG, Portugal Carla Teixeira Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira, ISEP, Portugal Ricardo Queirós, KMILT, ESEIG, Portugal Mário Pinto, KMILT, ESEIG, Portugal Chapter 13 LMS Adoption at the University of Genova: Ten Years After............................................................. 271 Maura Cerioli, University of Genova, Italy Marina Ribaudo, University of Genova, Italy Marina Rui, University of Genova, Italy Chapter 14 Effective Use of E-Learning for Improving Students’ Skills............................................................... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain Chapter 15 Strategies of LMS Implementation at German Universities................................................................ 315 Carola Kruse, Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan, Technische Universität Braunschweig, Germany Arne Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany Compilation of References................................................................................................................ 335 About the Contributors..................................................................................................................... 360 Index.................................................................................................................................................... 370

Detailed Table of Contents

Foreword............................................................................................................................................... xv Preface.................................................................................................................................................. xix Acknowledgment...............................................................................................................................xxiii Section 1 Generalities and Perspectives Chapter 1 General Perspective in Learning Management Systems.......................................................................... 1 Robert W. Folden, Texas A&M University-Commerce, USA In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective. Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness......... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the

varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness. Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course............................ 50 Thomas Connolly, University of the West of Scotland, UK Carole Gould, University of the West of Scotland, UK Gavin Baxter, University of the West of Scotland, UK Tom Hainey, University of the West of Scotland, UK Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web2.0 technologies provided by the chosen LMS. Section 2 Implementing and Evaluating Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs........................................................... 75 Paul Lam, The Chinese University of Hong Kong, Hong Kong Judy Lo, The Chinese University of Hong Kong, Hong Kong Jack Lee, The Chinese University of Hong Kong, Hong Kong Carmel McNaught, The Chinese University of Hong Kong Hong Kong Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University.

Chapter 5 ANGEL Mining..................................................................................................................................... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation. Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems.........116 Kamla Ali Al-Busaidi, Sultan Qaboos University, Oman Hafedh Al-Shihi, Sultan Qaboos University, Oman Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’ supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS.

Section 3 Trends and Challenges Chapter 7 A Comparative Study on LMS Interoperability................................................................................... 142 José Paulo Leal, CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós, CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal A Learning Management System (LMS) plays an important role in any eLearning environment. Still, the LMS cannot afford to be isolated from other systems in an educational institution. Thus, the potential for interoperability is an important, although frequently overlooked, aspect of an LMS system. In this chapter we make a comparative study of the interoperability features of the most relevant LMS in use nowadays. We start by defining a comparison framework, with systems that are representative of the LMS universe, and interoperability facets that are representative of the type integration with other broad classes of eLearning systems. For each interoperability facet we categorize and identify the most representative remote systems, we present a comprehensive survey of existing standards and we illustrate with concrete integration scenarios. Finally, we draw some conclusions on the status of interoperability in LMS based on our study. Chapter 8 Mobile Learning Management Systems in Higher Education............................................................. 162 Demetrios G. Sampson, University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas, University of Piraeus & Centre for Research and Technology Hellas, Greece Learning Management Systems (LMS) are widely used in Higher Education offering important benefits to students, tutors, administrators and the educational organizations. On the other hand, the widespread ownership of mobile devices has lead to educational initiatives that investigate their potential as the means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This mainly aims to support the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions, towards to a truly constant and instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. In this book chapter, we address the issue of designing mLMS for Higher Education by studying and applying the W3C Mobile Web Best Practices 1.0 to a widely used existing LMS, namely, the Moodle. Chapter 9 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires................. 178 Dimos Triantis, Technological Educational Institution of Athens, Greece Errikos Ventouras, Technological Educational Institution of Athens, Greece The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems

(LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, provided that the new method proves its equivalence to traditional methods that might be considered as standard, such as constructed response (CR) tests. Research efforts to that direction are presented. Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility..........199 Lourdes Moreno, Universidad Carlos III de Madrid, Spain Ana Iglesias, Universidad Carlos III de Madrid, Spain Rocío Calvo, Universidad Carlos III de Madrid, Spain Sandra Delgado, Universidad Carlos III de Madrid, Spain Luis Zaragoza, News Service, Radio Nacional de España, Spain Currently, the great majority of institutions of higher education use Learning Content Management Systems (LCMSs) and Learning Management Systems (LMS) as pedagogical tools. In order to make these systems accessible to all students, it is important to take into account not only educational standards, but also standards of accessibility. It is essential to have with procedures and well-established method for evaluating these tools, so in this paper we propose a method for evaluating the accessibility of LCMSs and LMS based on a consideration of particular accessibility standards and other technological and human aspects. The method application is for all LMS, in order to illustrate the effectiveness of the evaluation method, we present a case study over the widely-used LMS Moodle . In the case study, the accessibility of Moodle is evaluated thoroughly from the point of view of visually-impaired persons. The results obtained from the case study demonstrate that this LMS is partially accessible. The evaluation shows that the tool provides poor support to the authors of accessible educational contents. Chapter 11 The Technological Advancement of LMS Systems and E-Content Software..................................... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK This chapter will present the practical aspects of Learning Management Systems adoption by describing this process from the perspective of evolution, observed for LMS and e-content software at West Pomeranian Business School. The chapter will address issues and found solutions relating to LMS deployment and evolution, noticed during the management of e-learning studies. In its first part, chapter will explain the requirements for different types of studies and how they influenced the shape of LMS systems. In the following sections, the chapter will analyze different technologies and software used in the e-learning process. This analysis will also describe how efficiently use the functionality of e-learning software in relation to the users’ requirements. The last part of chapter will present SPE - SDART Presentation Engine, being an innovative e-learning presentation engine, developed in form of Rich Internet Application, to overcome the limitations observed for the previously used presentation engines.

Section 4 Case Studies Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education..................................... 247 Rosalina Babo, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, NID-RH, ESEIG, Portugal Carla Teixeira Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira, ISEP, Portugal Ricardo Queirós, KMILT, ESEIG, Portugal Mário Pinto, KMILT, ESEIG, Portugal The Internet plays an important role in higher education institutions where Learning Management Systems (LMS) occupies a main role in the eLearning realm. In this chapter we aim to characterize the Internet and LMS usage patterns and their role in the largest Portuguese Polytechnic Institute. The usage patterns were analyzed in two components: characterization of Internet usage and the role of Internet and LMS in Education. Using a quantitative approach, the data analysis describes the differences between gender, age and scientific fields. The carried qualitative analysis allows a better understanding of students’ both motivations, opinions and suggestions of improvement. The outcome of this work is the presentation of the Portuguese students’ profile regarding Internet and LMS usage patterns. We expect that these results can be used to select the most suitable digital pedagogical processes and tools to be adopted regarding the learning process and most adequate LMS’s policies. Chapter 13 LMS Adoption at the University of Genova: Ten Years After............................................................. 271 Maura Cerioli, University of Genova, Italy Marina Ribaudo, University of Genova, Italy Marina Rui, University of Genova, Italy The last two decades have seen the spread of LMS among schools, universities, and companies to augment the traditional teaching process with ICT and network technologies. This chapter presents the process leading to the adoption of a Moodle based LMS at the University of Genova in the last decade. By analyzing the data collected from the LMS logs and from questionnaires proposed both to students and teachers, we found out that the needs of the stakeholders are largely limited to resource sharing and organizational support, satisfactorily provided by the current service. Further improvements could be achieved by the introduction of a policy encouraging or forcing the teachers to use the provided LMS. A project on instructional design and, as a case study, the evolution of some of the courses involved in it are also presented. Though the redesign of such courses has improved their results, the impact on the overall organization of the degree program has been negative. We infer that this is due to the excessive freedom the students enjoy in taking their exams in Italy.

Chapter 14 Effective Use of E-Learning for Improving Students’ Skills............................................................... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain The educational system promoted by the European Higher Education Area advocates the introduction of new teaching methodologies in order to improve students’ skills as well as their knowledge in the subject areas they are studying. In response to this, new teaching strategies were implemented in Industrial Engineering and Software Engineering degree courses. The main goal of the project was to improve students’ skills in areas including problem-solving, information management, group working and the acquisition of writing and speaking skills, by means of e-learning tools. In addition to implementing the new strategies, a set of assessments including surveys, forum activity analyses and group tutorial evaluations were also carried out. The combined use of these techniques proved a very useful way of improving the students’ general skills and knowledge, especially in terms of design methods and organisation and planning ability and in general academic performance. Chapter 15 Strategies of LMS Implementation at German Universities................................................................ 315 Carola Kruse, Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan, Technische Universität Braunschweig, Germany Arne Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany In Germany, a learning management system (LMS) has become an everyday online tool for the academic staff and students at almost every university. Implementing an LMS, however, can be very different depending on the university. We introduce some general aspects on the strategies at German universities on how to implement an LMS. These aspects are mainly influenced by two main approaches, the top-down and bottom-up approach, which determine the decisions and actions on different levels at the university. In order to show how the strategies are carried out, we are presenting three case studies from universities based in the German federal state of Lower Saxony. We are going to reveal that both approaches play a part in each strategy, however differently weighted. It becomes clear that networking and collaboration plays a crucial role, not only concerning the technical development of the LMS software but also in organisational and educational terms. Compilation of References................................................................................................................ 335 About the Contributors..................................................................................................................... 360 Index.................................................................................................................................................... 370

xv

Foreword

Learning Management Systems (LMS) are now ubiquitous in institutions of higher education. This has occurred very rapidly with adoption being widespread but with little standardization. LMS’s were first used to support delivery with some communication between teachers and learners, but use has now been extended to support learning activities in innovative and diverse ways. They are also being used to increase student engagement and to track student progress – a vastly different approach to the early years of pushing resources to students. Adoption of LMS’s started with experimentation by a few with small systems. Familiarization facilitated wider adoption until eventually the “institution-wide” adoption of large commercial systems became common. Recent developments have extended the range of options from a few large commercial systems to a wider selection of open source, adaptable and specialized systems. In the intervening years from early adoption to now, higher education institutions have also gained valuable expertise in selecting, implementing, using and evaluating technologies to support learning and teaching some of which has been gathered in the chapters of this book. The authors have focused on a number of areas including: implementation strategies; use of learning management systems and other eLearning technologies; technical developments; evaluation; adoption and acceptance; and supporting skills. Chapter 1 takes a broad historical focus, reflecting on the background of eLearning from the very early days of teaching machines and computer assisted instruction through to correspondence courses and video conferencing. The role of information and communication technologies in supporting higher education processes, including teaching and learning, is explored leading to the overarching concept of learning management systems and how these support different modes of learning. The major conclusion reached is that pedagogy should drive the development and use of an LMS. Chapter 2 argues that learning and knowledge is facilitated by social interactions, implying that communication should be a key component of any eLearning system. The author argues that an LMS should facilitate social interactions at a number of levels independent of temporal or geographic constraints or the context that the knowledge activity takes place in. The communication should be facilitated by whatever means are available independent of specific LMS characteristics. The use of Web 2.0 technologies to support learning in a vocational setting is explored in chapter 3. The authors posit that despite the uptake of eLearning technologies in higher education generally, there is less evidence of uptake in the vocational sector. They aim to answer questions about whether technology can supplement the hands-on approach of vocational training, in particular the use of web 2.0 technologies such as wikis and forums. They present a case study the outcomes of which suggest that although there is potential for educational technologies to offer great benefits for vocational training, there is still much work to be completed in this area.

xvi

Chapter 4 investigates the use of LMS usage logs to facilitate teaching and learning. This is not an uncommon activity in commercial settings and, in an eLearning situation, can reveal valuable information about how the LMS is being used. The authors explore how the data can be analyzed to better inform their understanding of the online activities and, as a result, inform and improve the eLearning strategies that support institutional, faculty and departmental use of the LMS. In a similar vein, chapter 5 explores usage patterns in a specific LMS, ANGEL. Here data mining is used to explore LMS feature use. Using machine learning techniques it is possible to predict: the courses will use the ANGEL system most effectively based on course specific attributes; the interaction between features or sets of features that impact on usage; and those features that are most commonly used. This leads to a set of results that can be used by the institution to inform future decision making regarding feature selection within an LMS, the design, selection and implementation of an LMS, as well as identifying areas requiring additional training. Chapter 6 further explores usage, investigating critical factors that influence the acceptance and use of an LMS by instructors. The author suggests that although LMS survival is determined by instructor and student use, adoption is initiated by the instructor’s acceptance and subsequent use. Through a comparison of instructors’ acceptance of technology, LMS characteristics and organizational characteristics for the acceptance of eLearning, a model of overall acceptance and use is developed in a distance education setting. The study also provides insights into what additional support is needed in situations where computer use and Internet literacy is not high. In chapter 7 the issue of LMS interoperability with other systems in educational institutions is explored. The authors point out that although the LMS is directed at supporting learning it cannot be isolated from other systems in the institution. Two systems have been selected for the comparison representing a significant market share. A number of facets were selected for the comparison, using currently accepted standards, including: system communication with operational environment; learning content management; and academic management. The overall conclusion is that LMS interoperability leaves a lot to be desired. Standards relating to communication and content are relatively well developed but significant work still needs to be done in the area of interoperability of academic management. The advent of mobile technologies and the impact on course management systems in higher education is explored in chapter 8. The authors address the issue of designing such a system in the context of the Moodle LMS, specifically they investigate the application of W3C Mobile Web Best Practices 1.0. A framework is designed for a server-based mobile version of Moodle that follows the W3C guidelines. The enhancing of electronic examinations is explored in chapter 9. The authors explore an extension to multiple-choice questionnaires which allow for novel grading methodologies to be employed. They suggest that simple positive scoring rules and mixed-scoring rules introduce bias for a number of reasons. However the paired scoring method introduced here overcomes some of the shortcomings of these other methods. The authors do emphasis that the initial workload associated with constructing the question bank for multiple-choice questions is high, but overall it is concluded that the enhancements suggested here add to the value of examination tools within an LMS. With the widespread adoption and use of LMS’s in higher education, the authors of chapter 10 highlight the need for these systems to be accessible to all students. An evaluation framework is developed to evaluate the accessibility of LMS’s based on particular accessibility standards as well as other technological and human criteria. The framework is tested using the widely used LMS, Moodle, using the perspective of visual-impairment. A number of findings are reported, not least of which is that Moodle does not meet accessibly standards fully.

xvii

In chapter 11, the authors describe the legal requirements of using eLearning so support distance learning in higher education in Poland. They explore the implications of the regulations and how this has shaped LMS use. They compare the educational processes required of blended- and e-learning pathways and how different technologies can support aspects of those pathways. Extensions to the LMS that are required to support learning needs are discussed. The authors also point out that there are implications for the development of materials (e-content) which increase the cost of production on a number of levels. Finally the authors present an eLearning presentation engine that overcomes some of the difficulties associated with producing e-content. A comparison of Internet and LMS use is undertaken in chapter 12. The authors propose that more attention should be given to users’ Internet skills when developing LMS strategies since most LMS’s are web-based and most students are already adept at navigating the Internet before starting to use the LMS. The results support this proposition but also indicate that consideration should be given to using collaborative platforms such as web 2.0 technologies in place of email since many students are more familiar with using wikis and blogs for example. These outcomes can inform the selection and implementation of suitable digital pedagogical processes and tools that meet both teacher and student needs as well as informing the development of eLearning policies. Chapter 13 presents the process used by one higher education institution to determine an appropriate institution-wide LMS. The institution had a history of pockets of use and innovation to base the selection on. A subsequent evaluation of the adopted system has shown that use of the system is limited to resource sharing and organizational support, the most commonly used activities in the previous collection of systems. The authors suggest that the introduction of an eLearning policy could be instrumental in enhancing and extending the use of the LMS more fully. However they have identified that any extensions of use are beyond the current capacity of the system and support services, a catch-22 situation given the imperative of supporting the modern student digitally. A case study of using eLearning strategies to improve students’ generic skills is presented in chapter 14. Problem-solving, information management, group work and communication skills are the focus of this study. A combined strategy incorporating discussion boards, group tutoring, collaborative learning and peer assessment were implemented together with a number of assessment regimes including surveys, online activity analysis and group evaluations. These have resulted in improved student performance as well as improved perceptions of the accessibility of teachers, even though this is online. The study found that teacher participation is a key factor in motivating students to engage with learning activities as well as to lead discussions. The concluding chapter, chapter 15, explores different strategies that can, and have, been used when implementing an LMS. Two main influences are whether a top-down or a bottom-up approach is adopted. These determine the different levels at which key decisions are made. The authors present three case studies which demonstrate the different approaches. All three use a blend of the two approaches but in different mixes. All institutions eventually implemented the same system despite the different foci. Regardless of strategy it seems the key to success includes good networking and communication throughout the implementation process. Jo Coldwell Deakin University, Australia

xviii

Jo Coldwell joined Deakin in 1997 where she is currently Associate Head (Teaching and Learning) in the School of Information Technology. Before joining Deakin she gained a wealth of experience in both academia (in Australia) and industry (in both Australia and the UK). Jo was eLearning Manager for the Faculty of Science and Technology for a number of years during which Deakin University undertook the first institution-wide implementation of a major learning management system (WebCT Vista). During this time she was intimately involved in the deployment at University, Faculty and teaching levels. Since 2000 she has taught extensively online and her research interests lie in a number of areas associated with engaging tertiary teachers and learners in and with technology.

xix

Preface

E-Learning plays a significant role in education, and its importance increases day by day. Learning environments can take a myriad of distinct forms. Learning Management Systems (LMS) emerge as an important platform to support effective learning environments. According to Wang and Chen (2009), “an LMS employs a range of information and communication technologies to offer an online platform over the Internet, where a whole course can be planned, facilitated and managed by both the teacher and the learner”. In their work it is presented the main functions of some of the LMS nowadays available for educational purposes such as: learning material management, discussion forums, group emailing, audio conferencing, video conferencing, text chat, and whiteboard and synchronous document sharing. For Watson and Watson (2007), the term LMS is used to describe different educational computer applications. LMS is the framework that holds all sides of the learning process, including skills gap analysis. It is responsible to deliver and manage the infrastructural content, to identify and assess individual and organizational learning or training goals, to follow the process in order to reach those goals, and to collect and present data for supervising the learning process of an organization as a whole. With the rising of Web 2.0 and Web 3.0, learning environments are also overflowing Learning Management Systems and Institutions’ boundaries. Learning Management Systems are used all over Higher Education Institutions (HEI) and the need to know and understand its adoption and usage arises. On the one hand, there are different institutional cultures and characteristics and, on the other hand, there are several distinct LMS tools. Considering this it is expected to find out distinct experiences in the adoption and usage of LMS. The richness of each of the experiences can help the worldwide community to better understand how LMS are being used. The most used LMS according to a survey (Babo & Azevedo, 2009) answered by 51 universities from 19 different countries in 5 continents, were Moodle (Moodle, 2009), Blackboard / WebCT (Blackboard, 2009), and Sakay (Sakay Project, 2009). In that study several other LMS were referred such as ItsLearning, Desire2Learn, Claroline, METU Online, Chisimba, High Learn, Formare, Learning Space, First Class, Dokeos, eCollege, Class Fronter, KEWL. The results can be seen as an evolution. In the past years, the proprietary platforms were the most used but currently there is an increase of open source free platforms usage (Bradley et al., 2007). Consequently, there are not many studies regarding the usage level of such tools, concerning students, teachers, tools functionalities, usability, and the entire technological environment. Generally, both proprietary and open source free LMS provide several functionalities, such as, electronic distribution of course syllabi, grades and teachers feedback to students, ability to post hyperlinks to websites, forum for the exchange of ideas, wikis which allows students to swap ideas and information on projects, chat rooms for real time discussion, facilitating emailing and

xx

messaging among the participants (teacher/students, students/students), facilities for students to submit work assignments electronically, the means to administer quizzes and texts online (Janossy, 2008). It is frequent to observe that despite LMS on HEI is offered and usage stimulated, only a few of those functionalities are adopted, either by teachers, or by students. LMS are a powerful technology that has not achieved its full potential yet. As far as we know, understanding the actual aspects of LMS usage in HEI is an issue that is not sufficiently explored on research. Consequently, this is an interesting aspect to be explored and studied. The primary objective of this book is to provide insights concerning Learning Management Systems on Higher Education Institutions. The book aims to increase understanding of LMS adoption and usage providing relevant academic work, empirical research findings and an overview of LMS usage on Higher Education Institutions all over the world. The target audience of this book is composed of Education government members, Higher Education Managers, researchers, academicians, practitioners and graduate students in every field of study. LMS are not limited to a specific academic area being a trend and a new learning approach in any scientific field.

BOOK STRUCTURE This book includes fifteen chapters divided into four sections, namely, LMS Generalities and Perspectives, Implementing and Evaluating, Trends and Challenges, and Case Studies. It counted with the collaboration of researchers from 40 different universities and companies from 35 countries. Despite of the overall quality of the received proposals it was not possible to include all of them. As editors and after serious consideration during the review process, supported by our reviewers’ team and by the Editorial Advisory Board, we chose the best chapters in order to achieve the proposed goals of this book and those chapters which better fits the main focus of the book. In the first section Generalities and Perspectives Robert Folden very well understood the need of a general view of all the aspects related with the Learning Management System issue. One of the problems with much of the Scientific Literature is the assumption that all readers have the same background of the writers. This author very well assumes that is rarely true presenting the readers with the necessary foundations to go further in the field to be studied in depth. This is the chapter that any editor would appreciate to open with a book. Chapter 2 lies upon the premise of the asynchrony of the learning and knowledge sharing process which is a “human-to-human process that happen independent of time and space”. A reflection is presented towards “social awareness to determine the appropriateness of a LMS tool” considering asynchrony and ubiquity through all the process.Uncommonly a social concerning contributes from a technological university author. Both chapters are an excellent treatise of LMS in education. In the line of the two previous chapters, Chapter 3 provide us with an insight of the Distance Education evolution and the evaluation of the use of web 2.0 technologies offered by a chosen LMS in an educational context. Furthermore an interesting case study was developed by University of the west of Scotland and hereby presented. The use of e-learning in vocational courses is well explored in this chapter. Adoption and evaluation are two important and related issues regarding LMS that are presented in section 2. LMS store users’ logs in specific internal databases. These logs contain an immeasurable

xxi

wealth that can and should be used to evaluate LMS usage, in order to help Higher Education decision makers to take better decisions regarding LMS policies. Paul LAM, Judy LO, Jack LEE, and Carmel NAUGHT present, in chapter 4, an interesting study developed in the Chinese University of Hong Kong. The study took place during three years, between 2007 and 2009. The authors define three levels of analysis for the e-Learning activities, namely Popularity, Nature, and Engagement. Using these levels, it was possible to become aware of the different types of LMS usage and to define strategies in order to align its usage with the policies of the institution for e-Learning. Also using users’ logs analysis, but with a different even still very interesting perspective, Tyler Swanger, Kaitlyn Whitlock, Anthony Scime, and Brendan Post study, developed in The College at Brockport, State University of New York, is presented in chapter 5. The study was developed during the academic years of 2007 and 2008. Three data mining tasks, namely classification, clustering, and association, are implemented in order to extract useful knowledge and to obtain meaningful insights on LMS evaluation. This chapter refers to a new and interesting data mining application. In chapter 6, Kamla Ali Al-Busaidi, and Hafedh Al-Shihi present a model that intend to explain instructors’ acceptance of LMS. Despite that LMS usage depends on both students and instructs, it is up to the instructor to start the process and thus it is fundamental to understand which factors affect instructors’ acceptance and consequent use of LMS. The presented model is a valuable contribution in this direction. LMS usage in higher education is gaining momentum each day. As a consequence, new trends and challenges arise. In section 3, these issues are explored. José Paulo Leal & Ricardo Queirós explore, in chapter 7, some issues concerning LMS interoperability. In order to analyze and compare some of LMS interoperability features, a framework was developed and tested using Moodle and Blackboard. The framework defines two facets for LMS interoperability, exploring the main related issues in a stimulating, methodical, and efficient manner. Chapter 8 focuses on the use of mobile devices to access LMS supported courses. Demetrios Sampson & Panagios Zervas present a device developed in order to allow the deployment of LMS courses through online devices. This is advantageously achieved by means of the implementation and validation of a mobile version of Moodle that conforms to guidelines proposed by the World Wide Web Consortium. In chapter 9, Dimos Triantis & Errikos Ventouras contribute with the presentation of an interesting grading scheme applied in multiple-choice questionnaires. The presented grading scheme intends to prevent students from guessing the correct answers and thus develop a fair grading system. The proposed methodology is compared with more traditional ones in a real situation, bringing good insights to this issue. In chapter 10, the important theme of LMS accessibility is introduced. Lourdes Moreno, Ana Iglesias, Rocio Calvo, Nuno Delgado, & Luis Zaragoza, evaluated Moodle LMS, studying in detail accessibility concerns regarding visually impaired users. A new method was developed that leads to a better perceptiveness, which can conduct to the definition of better policies and practices. Dorota Dzega & Wieslaw Pietruszkiewicz presents, in chapter 11, a new technological solution designed to address some specific necessities of a higher education institution. Those necessities are also general necessities of a vast majority of higher education institutions. The referred solution is presented as an innovative extension of Moodle LMS, and consists of a new layer in the eLearning platform, offering additional advantages. The case Study section presents four case studies among Portuguese, Spanish, Italian and German universities. While the first three chapters present deep case studies whose main concern is the better understanding how LMS are being used, the German case study focuses in adoption strategies for LMS.

xxii

Chapter 12 characterizes the students Internet Usage and their LMS usage patterns in a Portuguese University. Chapters 13 and 14 case studies present long-lasting experiments and observation made in Spanish and in an Italian Universities. Chapter 13 presents the effort made by Genova University over the last 10 years in order to adopt “ICT educational support”. Next chapter relates the adoption of an elearning system which involved new teaching strategies and take into consideration student’s workload. The last chapter discusses the taken approaches – Top-Down or Bottom-up- of three German universities regarding the LMS implementation process. Not all the universities have had the same approach. Nevertheless the overall goals were achieved.

REFERENCES Babo, R., & Azevedo, Ana (2009). Learning Management Systems usage on Higher Education Institutions. In Proceedings of 13th IBIMA Conference - Knowledge Management and Innovation in Advancing Economies: Analyses & Solutions (pp. 883-889). Blackboard (2009). Blackboard. Retrieved July 30, 2009 from http://www.blackboard.com/ Bradley, M., Carter, J., Fitzsimons, D., Graham, J., Hurlbut, N., Marshall, D., et al. (2007). Learning Management System Evaluation Report. Executive Summary, Humboldt University. Janossy, J. (2008). Proposed Model Evaluating C/LMS Faculty Usage in Higher Education Institutions. Paper presented at MBAA Conference, Chicago, IL. Moodle (2009). moodle.org - Open-source Community-based Tools for Learning. Retrieved July 30, 2009 from http://moodle.org/ Sakay Project. (2009). Sakai Project Home. Retrieved July 30, 2009 from http://sakaiproject.org/portal Wang, Y., & Chen, N. S. (2009). Criteria for Evaluating Synchronous Learning Management Systema: Arguments from the Distance Language Classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773 Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-0070023-y Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal

xxiii

Acknowledgment

We have learned a great deal from those who lead us along our academic career and gratefully acknowledge our debt to them, especially Professor João Álvaro Carvalho. Our students have contributed in a fundamental way to our work with their editorial assistance, namely: Daniela Gonçalves, Joana Oliveira and Maria Duarte. To all the members of the Editorial Advisory Board, to the reviewers and to the authors, whose names are published in this book and who have assisted us one way or another, we feel very much grateful. We would like to express our thanks to the Publishing team at IGI Global for their expert support and guidance. This book is dedicated to our respective families. Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal

Section 1

Generalities and Perspectives

1

Chapter 1

General Perspective in Learning Management Systems Robert W. Folden Texas A&M University-Commerce, USA

ABSTRACT In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective.

INTRODUCTION Before one begins an extensive study of any topic, it is to their advantage to view the topic in a general fashion. It is important to look at the historical development of the content, as well as, some the theoretical underpinnings of the subject. This helps to develop a healthy perspective for viewing the information that will be studied in depth. It can also lead one to consider those areas of greatest interest for future research. Beginning DOI: 10.4018/978-1-60960-884-2.ch001

with a birds-eye view, you will not initially see the fine detail of any individual topic or aspect of the material of interest, but you will gain an understanding of the directions that impacts have come from and being able to better understand where the subject is likely to go.

eLEARNING ROOTS In understanding any system, it is important to understand its roots (see Figure 1). As we look at the past we can understand the present and pos-

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

General Perspective in Learning Management Systems

sibly project the future (Rose, 2004). Without this knowledge, we may never understand the present and not be able to speculate effectively on where we are going (Rose, 2004). To actually understand the foundation of learning management systems, you must begin with a totally different domain of knowledge; that of psychology; most notably, educational psychology (Holmberg, 2005). One must also look at the developments that have occurred technologically (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Wagner, N., Hassanein, K., Head, M., 2008). Where we are today is an outgrowth of where we have been and it is necessary to understand that path if we are to formulate a good sense of where we are going. There are multiple generations that we have come through (Taylor, 2001).

Programmed Learning/Teaching Machines At the beginning of the twentieth century, a group of psychologists were concerned with conditioning as an explanation of behavioral adaptation. They were generally referred to as ‘behaviorists’. They believed that all behavior (learning) could be explained by the concept of conditioning. ‘Learning’, as they saw it, could be accomplished by controlling the use of stimuli and rewards, both positive and negative. An outcome of this process was the development of programmed learning tools (Rose, 2004). Initially these tools were in the form of booklets that allowed the controller to manage the stimulus by applying the appropriate reward and thus produced the desired behavior/learning. These booklets were later moved to a mechanical device or teaching machine (Rose, 2004) (Keegan, 2002). The material was presented in very small steps that were referred to as a frame. The student was then presented with a blank to fill in after which they were provided with the correct answer. In their original conception they were not officially graded or ranked, but the student worked through the material. Based upon student performance, the

2

same material would have been repeated or new material presented. This form of instruction was very linear in nature and mastery was the end goal (Baggaley, 2008). Others provided some form of grading or required mastery before the student was allowed to progress. See http://www.greenchameleon.com/gc/blog_detail/weve_been_imagined/ for pictures showing students using a selection of these machines.

Computer Assisted Instruction When computers came on the scene in the 1950’s (Watson, W. & Watson, S., 2007), they became the teaching machines and the process was referred to as computer assisted instruction (CAI) (Rose, 2004). The foundation of this learning was individualized instruction (Rose, 2004). The theory was that individual students needed to learn at his pace and in his way. This system was also referred to as an Integrated Learning System (ILS) (Rogers, L. & Newton, L., 2001) (Underwood, 1997).The instructor ensured the proper design of materials so logical organization allowed the student to move through the material in an appropriate manner. This process required that each student had access to a computer for the appropriate amount of time for learning to occur. Originally students used ‘dumb terminals’ attached to mainframe computers. Each student could access his or her own files with the results stored in a centralized database. In the 1980’s and 1990’s intelligent computers in the form of PC’s assumed this role (Keegan, 2002) (Eteokleous-Grigoriou, 2009). These computers were eventually connected through local area networks and had the software stored on a centralized server. Students worked in a networked environment with the instructor moving about the classroom to help the students over the difficult portions and to keep the students on task.

General Perspective in Learning Management Systems

Figure 1. eLearning roots

Computer Managed Instruction One of the early goals of computer assisted instruction was to free the instructor from the instructional process and enable her to become a coach for the students (Rose, 2004). Therefore, course systems were designed to allow the student to progress through a set of materials according to a plan established by the instructor (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999) (Underwood, 1997). To track the student’s progress and provide for a record keeping process, computer managed instruction (CMI) became available (Rose, 2004) (Underwood, 1997). This generally included

more than one course and allowed the students to move forward in the curriculum according to her individual time frame. Many of the CMI systems provided remediation modules that allowed the students to repeat instruction in concepts that they had not sufficiently mastered on previous attempts through the material (Underwood, 1997). Student scores, number of attempts, and other pertinent information were stored in a system database for the instructor or course administrator to view (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991) (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999) (Underwood, 1997).

3

General Perspective in Learning Management Systems

Correspondence Course Learning (CCL) Beginning in the mid-nineteenth century, students were able to study in remote locations using correspondence courses (Gaspay, A., Dardan, S., & Legorreta, L., 2008) (Homberg, 2005). These text-based programs allowed the students to receive written information and a series of questions or projects that tested or reinforced the desired learning. In most cases the students worked independently (Taylor, 2001). Instructor involvement came in the form of written communication to students after they had submitted their work for grading (Holmberg, 2005). In the 1970’s and 1980’s, these courses were enhanced with audio or video recordings to provide the student with some content that was not easily communicated by words alone. In most cases the students and instructors were not in the same location, making this a form of distance learning (Keegan, 2002). This was an attempt to enable students who could not afford, for one reason or another, to travel to the instructor to receive an education. Again, this was basically individualized instruction, but it was not as individualized as the Programmed Learning, CAI, or CMI.

Instructional Television Fixed Service (ITFS) Instructional Television Fixed Service was developed beginning in the 1950s and greatly expanded in the 1970’s and 1980’s as a course delivery platform (Saba, 2000). The students participated in classes that were often remote from the instructor. There was usually an instructor present with one group of students while the others were at one or more remote sites. In many cases this was two-way video and audio, but was mostly one-way video with two-way audio. The audio in those cases was done over an audio bridge. This allowed the students to see the instructor and interact in much the same was as they would in a

4

contained classroom environment. This approach used broadcast television to send the video and the instructor audio (Gaspay, A., Dardan, S., & Legorreta, L., 2008). Because of its use of the television broadcast medium, it tended to be quite expensive. In the 1980’s and 1990’s this was enhanced and somewhat replaced by the sending of videotapes to the students to complete their coursework (Taylor, 2001). It was a blend between correspondence courses and ITFS. While this helped to bring the cost down, it did not provide what either the instructor or the student preferred and so did not do well in the long run.

Video Conferencing (VC) This approach gave way in the 1980’s and 1990’s to video conferencing systems (Taylor, 2001). These conferencing systems provided two-way video and audio links between multiple locations. In most cases this was compressed video and an audio bridge (Baggaley, 2008). While the students could see and hear, the quality often left much to be desired. Over time, the systems improved and use increased. They were hampered by the inability of systems to communicate with other manufacturers’ systems. Theoretically, they all complied with a uniform set of standards that should have made communication possible, but each manufacturer included proprietary algorithms that made it virtually impossible. During the latter part of the decade, there was more cross-collaboration, but it was never an effective method of allowing remote students to participate in a classroom.

CIRCLE OF FOCUS Classroom Based Learning Programmed learning, CAI, and CMI focused on improving classroom teaching. They were basically built upon the model of behaviorism which emphasized individualized instruction

General Perspective in Learning Management Systems

with instructor interaction. The focus was upon improving the learning of students in a contained environment with synchronous learning. This can be seen as allowing technology to find a place in the classroom. While there was talk about replacing the instructor, it was mostly a changing of the role of the instructor. Instead of the instructor being the direct dispenser of knowledge, the instructor became the facilitator/coach in the learning environment.

Distance Based Learning Correspondence course learning, video conferencing, and ITFS focused on distance education (Guri-Rosenblit, 2005). They were attempting to solve the problem of meeting the needs of students who could not or would not participate in a selfcontained classroom (Wagner, N., Hassanein, K., Head, M., 2008) (Wagner, N., Hassanein, K., Head, M., 2008). With students distributed over great distances (Guri-Rosenblit, 2005), there was little or no interaction with one another. Their only source of interaction was with the instructor or the instructor-assigned mentor/proctor. While ITFS and video conferencing theoretically allowed for some interaction, it rarely occurred because everything generally came through the instructor and that tended to stifle any student to student interaction. In some cases, distance-based learning was mated with CAI to form a sort of a hybrid approach. The students would complete computer simulations or drill and practice but also participate in video conferences to receive input from their instructor. Methodically, distance education has adapted by using the ever increasing improvements in the information technology to meet the needs of the students and improve the interaction between and among students and faculty (Baggaley, 2008).

INFORMATION COMMUNICATION TECHNOLOGY (ICT) While all of this technology development within education was going on, information communication technology was developing rapidly (Garrison, 2000). The development of the mainframe computer allowed companies and schools to automate business knowledge acquisition, storage, and processing. It led to many improvements in the functioning of the business side of the educational enterprise. Computers began to invade every aspect of business and then began to spread throughout the lives of individuals. As a result, it is nearly impossible to find a major device that does not possess at least a rudimentary computer. Also, individuals often depend on computers to allow them to access information and share information in an open and free fashion (Hamuy, E. & Galaz, M., 2010). Most individuals find it so difficult to carry on a normal life without computer interaction that we are in the process of making wireless internet a ubiquitous factor in our lives (Gaspay, A., Dardan, S., & Legorreta, L., 2008).

Networking During the 1980’s and 1990’s personal computers became networked with one another so that they could share information across platforms. They were able to present information in a graphical manner. This provided for a more enriched environment for the delivery of instruction (Rose, 2004) (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991). The systems could blend audio and video in such a manner as to appear seamless. They also allowed for greater interactivity with the students (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991). Because they were networked, students could now work on any available computer to continue the work that they had begun previously. Instructors could view individual student work unobtrusively, providing feedback as necessary. Students could participate

5

General Perspective in Learning Management Systems

in elaborate collaborative environments to develop their individual learning. As the computer continued to improve in capability, the range of activities that the students could participate in continued to expand.

Internetworking In the mid-1990’s we saw the development of internetworking with the introduction of the World Wide Web (WWW) (Keegan, 2002). This allowed networks to communicate with other networks far removed from the original source in a reliable fashion (Rose, 2004). This whole development coincided with the development of the graphical user interface (GUI) and the use of media rich environments for the delivery of information. Technology was developing at a very rapid pace, with processors able to handle larger workloads at much faster rates than ever before. Storage space was cheaper and cheaper to provide (Gaspay, A., Dardan, S., & Legorreta, L., 2008). Sound and video delivery systems could provide greater enrichment for less expense. These developments allowed the designers to enrich their information delivery, ignoring most of the limitations of the past (Taylor, 2001). Course material could be asynchronously delivered to many individuals in an individualized manner or synchronously delivered to large masses at one time or in some other combination (Rose, 2004) (Wang, Y. & Chen, N., 2009). As the Internet became ubiquitous, the barriers to delivery of information fell away, allowing for ubiquitous access to the information (education) provided (Taylor, 2001). Students began using computers more and more in their everyday lives. They became ever more comfortable with computer delivery of information in text, video, and audio. This familiarity made it easy to move educational delivery to this platform (Keegan, 2002). Students were able to use the Internet to seek out and obtain information on their own with or without the guidance of an instructor/mentor

6

(Rose, 2004). The technology could fade into the background and allow the course material to move to the forefront, regardless of the overall purpose of the information delivery. The nature of the information delivered was more a matter of perspective than one of substance.

E-Learning/Virtual Learning This ability to move education materials electronically around the world in both synchronous and asynchronous manners created the ability to develop a classroom irrespective of time, space, and proximity (Miller, T. & hutchens, S., 2009). Even with VC and ITFS, you needed to be in fairly specific locations to participate in class activities (Guri-Rosenblit, 2005). While Programmed Learning, CAI, and CMI could be done remotely, they usually were done with the students in one location connected to the same network and using the same hardware system (Baggaley, 2008) (Guri-Rosenblit, 2005). Today, all of those limitations seem irrelevant (Dillenbourg, 2008). Operating on the platform of ICT, educational institutions are delivering all levels of education and training over the Internet (Hamuy, E. & Galaz, M., 2010) (Ozkan, S., Koseler, R., & Baykal, N., 2009). Corporations seek to reduce their training expenses by providing training over the Internet rather than sending their employees to far off places with the related expenses. Even conferences have moved to the Internet to provide similar types of experiences at the attendees’ desktops rather than requiring them to attend the conference at some exotic location. These are also recorded and made available in an asynchronous manner for those who could not or would not attend when the event was going on. ICT is more than just a delivery mechanism; it has become a shaper of what we do (Rose, 2004). It reaches into every area of our lives and has touched nearly every individual on the face of the earth (Guri-Rosenblit, 2005). As the devices that access and share the information have become

General Perspective in Learning Management Systems

ever smaller, they have transformed more and more of our lives (Taylor, 2001). They have become the medium for developing communities far and wide. Individuals can find out even the most mundane or inconsequential pieces of information in a split second (Garrison, 2000). They can also share the most intimate details of their lives with the world just as quickly. These tools also allow instructors and students to share with one another in near-real-time, no matter where they are located. Information is not limited to text or even text and still images. It is possible to share full motion video and audio in very high quality over very great distances. These videos can be of real events or simulated reality. This has opened the way to virtual reality to function in an educational environment. This pervasiveness of ICT is forcing academic institutions at all levels to move their education offerings to an electronic platform to some extent (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Eteokleous-Grigoriou, 2009). It does not require, at this point, that everything be done in an electronic medium, but it does mean that some of it must (Taylor, 2001). This push is coming from multiple sources; from almost every stakeholder group that is involved in the educational process. That pervasiveness also means that instructors have a deeper well of knowledge about their students than ever before (Eteokleous-Grigoriou, 2009). While electronic media allow a degree of anonymity, it also opens up a greater breadth of information about each participant to all of the other participants in the educational process (Dillenbourg, 2008).

INFORMATION COMMUNICATION TECHNOLOGY AND LEARNING MANAGEMENT SYSTEMS For this discussion, we will focus on academic institutions and the systems that are involved there. While they are much the same as those of

other businesses, this focus will enable a clearer correlation to learning management systems in the context that we are viewing them. We need to remember that nearly all of these ICT systems existed in paper form before they appeared in electronic form. Most of what was done in the early systems mirrored what had been done in paper forms. The basic purpose of moving these systems to digital format was to automate the redundant activities in order for the users to focus on those aspects that could not be automated, thereby creating performance improvement (Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, J., 2008) (Jones, 2004). As the systems have developed, they have increasingly moved into areas that were not possible to do or were easily done in print form (Eteokleous-Grigoriou, 2009) (Cradler, 2008) (see Figure 2).

Finance Information Systems (FIS) The first ICT system to be developed was the finance information system. This system allowed enterprises to track income and expenses (Paulsen, 2002) and use the information to make informed decisions. It is sometimes referred to as an accounting information system (AIS). It has been defined as: A system “ . . . that processes financial transactions to provide (1) internal reporting to managers for use in planning and controlling current and future operations and for non-routine decision making; (2) external reporting to outside parties such as to stockholders, creditors, and government agencies.” (Answers.com, accessed on 08-18-2010) Information was generally input using batch files and outputs were done in much the same way. As technology improved, it was possible to do input during real time and to schedule major outputs during the off hours so as not to degrade the performance of the system. These systems generally are comprised of three major components; the database and its front end, the control system, and the reporting system. The database stores all of the essential information and

7

General Perspective in Learning Management Systems

Figure 2. Information Communication Technology (ICT)

provides the input screens (front end) that allow the users to input the information. That information may be added to the database in real time or stored in a temporary ledger to be added at a more efficient time. Generally, if it is stored in a temporary ledger, it is added to the database at the close of business and the information will be available for reports after that time. The databases also allow for locking of the records so that they cannot be changed in any fashion without creating a record of the change (a part of the control system). The control system allows for the ability to develop appropriate controls on the system and users to ensure the integrity of the information contained in the system. The reporting system allows for the processed information to be displayed in an appropriate manner for the decision makers and other interested parties.

8

Faculty Information System (FIS) Faculty information systems are basically a human resource information system (HRIS) that allows an institution to maintain information on nearly everything that needs to be tracked or analyzed about faculty (current and former) and those who are applying for positions. It is able to be maintained by professionals in human resources and by the faculty themselves in order to maintain the most accurate and up to date information. Not only does it maintain information on the classes taught by each faculty member, but their academic credentials and relevant experience as well. It provides a place to maintain a record of their scholarly activity and their service involvement. These systems also allow for the publication of documents necessary for the faculty to perform their duties in an efficient and effective manner.

General Perspective in Learning Management Systems

Initially these systems did not communicate with any other systems, but more and more these systems are integrated with all of the information systems of an institution so that information only needs to be input one time.

Library Information System (LIS) The initial library information system focused on the card catalogue and the record of where an individual item resided. It was basically a large inventory tracking database for library holdings. It served to automate what a librarian was responsible to do in order to maintain control of the assets of the library. As more and more assets became electronic in nature, it also began to be a repository of those items and became then a digital library system. This system now makes it possible for patrons to access the resources of the library without having to be physically present. It also enables libraries to share their resources with one another in a seamless fashion; enabling their patrons to have access to a much broader set of materials than would be possible otherwise. While LISs served to just automate library processes in the beginning, they have now entered into the process of transforming the way that the library delivers information and the way that patrons access information. With LISs, libraries have moved from being islands of information to becoming portals to the information stored in the network of libraries worldwide.

Student Information System (SIS) Student information systems provide the ability to enter student information into a database that will provide an electronic grade book, student course schedules, and other student-related data needs for a school, college, or university (Paulsen, 2002). These systems grew out of the need to automate the various record keeping responsibilities of an academic institution. They have grown to the point of being able to provide advanced informa-

tion to the decision makers on par with enterprise resource planning systems or the HRIS systems. It is rare for an institution of any size to not have some form of an SIS system. While many of these were developed in-house initially, most are commercially available today. In the K-12 market, these systems are being outsourced to a hosted environment allowing for many other enhancements to be included in the SIS system. Most of the commercially available systems today provide the ability to integrate with all of the other ICT systems functioning in an academic institution.

Learning Management System (LMS) In a simple way learning management systems involve the application of ICT to the process of education, but it is a bit more complicated than that (Dillenbourg, 2008). An LMS provides the structure for the delivery and management of instructional content, while providing assessment of individual and organizational learning goals, tracking of progress toward those goals, and providing the data necessary for the management of the learning process of the institution as a whole (Watson, W. & Watson, S., 2007) (Kim, S. & Leet, M., 2008) (Rogers, 2001) (Perry, 2009). In today’s educational environment ICT is often involved in all types of learning systems (see Figure 3). The term was first used to refer to the management system of the PLATO K-12 learning system (Watson, W. & Watson, S., 2007). Today, there are basically four broad system designs used, as seen in the accompanying diagram. These designs are determined by the pedagogical focus of the education enterprise.

Traditional Learning Historically, this form of learning has been the staple for hundreds of years, from the very founding of higher education institutions. Students meet together in real time and in a specified location with the instructor present. This instructor cen-

9

General Perspective in Learning Management Systems

Figure 3. Learning system models

tered approach is the predominant modality of instruction in higher education. ICT can be used to enhance the educational experience with the use of technology delivering the instruction and maintaining student records. The instructor is the primary determinant of what use will be made of ICT in the delivery of instruction and to some extent to its use in the completion of assignments required for the instruction. Most information will be supplied by an instructor directly or in preprinted materials, but may be supplemented with various forms of ICT in the classroom. The students may utilize ICT outside of the classroom to produce class assignments, submit class assignments, or locate information for class, but the primary delivery of instruction is done in a face to face mode. This instruction is delivered synchronously.

10

Distance Learning Distance education dates from the mid nineteenth century. Students and instructors are separated by time, location, or both in this model of instruction (Gaspay, A., Dardan, S., & Legorreta, L., 2008). The materials may be delivered to remote locations via print or ICT, but this form of instruction does not preclude the use of remote classrooms. This form of instruction may be done synchronously or asynchronously. The emphasis of this mode of instruction is the separation of the student from the instructor and not the delivery mechanism (Gaspay, A., Dardan, S., & Legorreta, L., 2008). It does not matter that the separation is physical or temporal, just that there is some separation between the two participant groups. This mode of instruction does not focus on the delivery mechanism or the technological involvement. The

General Perspective in Learning Management Systems

focus may be the individual or a group (Keegan, 2002). Early forms of this modality focused on individual instruction (correspondence courses, etc.), with technology enabling group instruction beginning in the 1980s. This form of instruction was not highly valued until the advent of open universities in the 1970s. Most distance education in the United States is group focused, while most distance education in Europe is individual focused (Keegan, 2002). Regardless of the focus, most materials are of the pre-prepared variety. Instructors may utilize materials that were prepared by another or even by a group of others. The teaching materials may be produced up to ten years before the student interacts with them in a learning environment. Furthermore, the institution that created the materials may not be the institution that is awarding credit for the course completion. There exist clearing houses for courses that can be combined to create distance learning programs of study.

Blended Learning In the blended learning mode, parts of the instruction are delivered in a traditional format while other parts of it are delivered using ICT (Gaeta, M., Orciuoli, F., & Titrovato, P., 2009). It is this blending of the delivery modality that attempts to use the strengths of both formats to enhance the educational experience (Dillenbourg, 2008). ICT is more than just an adjunct to the process, but the students and instructor are required to utilize ICT in order for learning to occur properly. This form requires that parts of it are synchronous and parts are asynchronous. One form of blended learning is called an integrated learning system (ILS) in which networked computers or terminals with a management system monitors and records student performance results and distributes learning modules based upon those results (Dillenbourg, 2008) (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999). These are truly digital systems and can be used over a dis-

tance; they are most generally used as an adjunct to traditional learning. The primary function of these systems is to remediate performance deficits in basic skills. This form of blended learning is much more like the programmed learning roots from which all of this grew. These systems contain a management system that controls the flow of data between the other components, curriculum content that provides the tutorial, practice and assessment modules, and the student record system that maintain registration and performance information on every student enrolled in the system (Rogers, 2001). These are not usually integrated with the ICT systems of the institution. Their structure mirrors the structure of learning management systems without the broad connectivity found in those systems. Some examples of ILS are CentraOne, IntraLearn, Lyceum, and Silicon Chalk, Odyssey, and Plato (the same company that originated the name learning management system).

E-Learning E-Learning had its beginnings in late 1994 or early 1995 and really soared after 1996 when the first modern content management system was developed. In E-Learning all of the instruction is online using ICT to deliver the content. By 1998, it was viewed a mature field of distance education. Michigan Department of Education defines this as “A combination of structured, sustained, integrated, online experiences accessed via a telecommunications network.” This may involve synchronous and asynchronous delivery of instruction. It blends aspects from all of the other models into a model that is not just ICT enhanced, but one in which it would not occur without the use of ICT. In today’s implementation of E-Learning, ICT is transformative and not just supportive of the process of education (Gaspay, A., Dardan, S., & Legorreta, L., 2008). In some circles, E-Learning is used to refer to all learning that occurs through the mediation of ICT, but I use the term more formally to refer to structured/

11

General Perspective in Learning Management Systems

purposeful learning that utilized ICT for its development, delivery, and management in such a way that the instruction is transformed by the medium in which it occurs. It is much similar to the distinction between learning that occurs through independent effort of the student and that which is accomplished through the collaborative effort of instructor and student. Learning happens in both situations, but in the latter, it is substantially different than it would be otherwise because of the nature of the delivery medium. While a learning management system could be utilized in some fashion in each of these models, it is in E-Learning that it is an absolute necessity. ICT in all of these models could be referred to as learning support systems. The problem is that there are no universally accepted definitions for these learning systems that will serve to distinguish them from other ICT systems used in the educational process. For this reason, the terms are used haphazardly to refer to various systems or parts of the system as if they were the same.

COMPONENTS OF A LEARNING MANAGEMENT SYSTEM In a nutshell, LMS is the software that automates the administration and delivery of learning (Watson, W. & Watson, S., 2007) (Baturay, 2008) (Rogers, L. & Newton, L., 2001). It is the overarching ICT that provides all of the functions necessary to provide learning in a digital format at a distance (Wang, Y. & Chen, N., 2009). This does not preclude the ability for it to also serve in a blended learning environment, but it must be able to provide that learning where there is spatial, temporal separation or both. The system may have any number of components, but we will look at the major components that one should expect to find in a modern learning management system. It is important to remember that the technology is generally learning theory independent and can be utilized to function in most any theoretical

12

system (Jones, 2004). However, one must consider theoretical orientation when considering the appropriateness of any learning management system (Wang, Y. & Chen, N., 2009). While this seems pretty straight forward, there is much confusion as to what is meant by the various systems and components of systems. Some authors state that CMSs are LMS that are used by academic institutions while LMS are used in industry (Hamuy, E. & Galaz, M., 2010) (Daniels, 2009). Course management systems are also confused with content management systems, which both use the same acronym, CMS (Daniels, 2009). Course management systems and virtual learning environments (VLE) are basically the same thing with terms differing by the region in which they occur (Daniels, 2009). Course management system is the term used in North America and VLE is the term used in Europe (Daniels, 2009). Content management systems are generally systems used in industry to publish information to various audiences to support their work processes. A related term is learning content management system. It refers to the system that delivers the specific portions of learning content to the learner during the use of the LMS. We will not be referring to content management systems apart from learning and so CMS will always refer to course management systems in this chapter.

Course Management System (CMS) The CMS provides the linkage with all of the other systems involved in the learning endeavor. It is truly the brains of the whole system. This system allows the appropriate individuals to add or remove courses, to sequence courses within a curriculum, add students to the course, assign instructors to individual courses or sections of courses, to monitor other processes of the system, etc. (Watson, W. & Watson, S., 2007) (Daniels, 2009). This is probably the most frequent system confused with LMS. Many individuals believe that is all an LMS does and so what is the difference. To do its

General Perspective in Learning Management Systems

job, this system must interface with the FIS and SIS systems. It is also often linked with the LIS system. These linkages are often rudimentary and not well established. This lack of strong linkage can cause considerable problems to the end users and the institution alike. When it is functioning well, it is like a good traffic officer seeing that everyone and everything gets to where it needs to be in the most efficient manner.

Learning Content Management System (LCMS) The primary purpose of the LCMS is to develop, store, organize, and distribute multimedia content to support the delivery of E-Learning (Watson, W. & Watson, S., 2007). This over arching system has many subsystems that allow it to perform its duties. Most of these occur in the background and the users do not see them or even need to. Course content authoring is one of these background activities. While the instructor uses this system to design the course and upload the appropriate content, they really don’t need to understand what is being done to make it happen. Once the material is stored, usually in the form of learning objects (Watson, W. & Watson, S., 2007), it needs to be accessed at the appropriate time to support student learning (Watson, W. & Watson, S., 2007). These processes need to function effectively as purposed or they will negatively impact the quality of the learning delivered.

Collaborative Learning System (CLS) The CLS is the system that allows the use of the newer aspect of Web delivered content (Web 2.0). This system provides the tools for email communication, discussion groups, newsgroups, instant messaging, blogs, bookmarking, notice board, search tools, etc. Not all LMS have these capabilities, but these and many more are managed by the CLS. This is the part of the LMS that changes the most rapidly. Many times the capabilities in this

system are accomplished by third party apps that are added to the system piece meal. This process can create some very unusual difficulties if the system is not managed effectively. This is also the area where mobile connections are usually made and managed. It is within the CLS that many instructors will develop the social learning that should be part of any learning program that seeks to meet the needs of a diverse student population. However, just using collaborative tools without the requisite understanding and application of social learning theory will not produce the desired results. The importance of this area is supported by belief by some educational practitioners that only 20% of learning occurs based upon formal instruction and that the other 80% is produced by informal instruction that occurs when students interact in social environments involving each other and the instructor.

Assessment Management System All forms of assessment are managed from within assessment management system. This is also where the grade book is located. Assessments could be exams, homework, projects, etc. They could involve student submissions, as well as student participatory activities. The purpose of this subsystem is to ensure that proper delivery and recording of results for all assessment items is maintained. The system should provide the instructor with the tools necessary to assess student performance on the appropriate measures. It should allow the instructor the ability to design these measures with some degree of flexibility. The assessment management system should enable the instructor to provide the student with adequate and timely feedback on their performance so that the student will benefit from the learning and be able to improve the desired outcomes of the learning experience (Watson, W. & Watson, S., 2007).

13

General Perspective in Learning Management Systems

LEARNING MANAGEMENT SYSTEM STAKEHOLDERS There are basically four stakeholder groups that are directly involved in LMS (see Figure 4), but there is a fifth that is peripherally/externally involved. The four primary/internal stakeholders are students, faculty, administrators, and IT staff (Wagner, N., Hassanein, K., Head, M., 2008). The peripherally involved stakeholder group is that group who will hire the students once they complete the education using the system or are involved in accrediting the education institution to be able to offer a degree or credential. Each of these groups holds slightly differing sets of wants and needs, but they also share some commonalities.

Students Students are the ultimate consumers of the LMS output. Generally speaking they are either undergraduate or graduate students of the educational institution. They may be enrolled in one or more courses delivered by the LMS. While there are a number of motivations for their using an LMS, some of the primary ones are access to an education, convenience, and, for a few, because they prefer this mode of learning (Wagner, N., Hassanein, K., Head, M., 2008). Some will be using an LMS because there is no other way to complete the program of study that they desire. In a sense, then, this relates to access. Modern students have grown up with an increasing involvement with technology in their day to day lives. Few, if any, know of a time when they did not have access to the internet to further their contact with one another and with the information that is available from a broader perspective, especially in the western world. Most students today have an extensive background with ICT and use it on a daily basis. This technological background of these students will impact what they expect from any delivery system (Wagner, N., Hassanein, K., Head, M., 2008). It will also play a major role in how they

14

are able to utilize the system to improve their educational outcomes and to meet the goals that they set for themselves in the process, as well as the goals that may be set by the faculty and administrators.

Faculty Instructors, more than anyone else, determine the nature of the E-Learning experience (Wang, W. & Wang, C., 2009). Depending on their theoretical foundation, they play a larger or smaller role determining the educational experience of the students. Generally, they are the arbitrators of whether or not the students will participate in group activities, independent study, or synchronous events, to just name a few ways in which they control the E-Learning experience. If their theoretical persuasion is behaviorist, they may design instruction in such a way as to move the student in a structured manner to the desired outcome, but if they are constructivists, they will allow the students to determine how they will achieve the end result. These are two of the many facets of their role in the instructional process. Ultimately, E-Learning creates many changes for the instructor (Wagner, N., Hassanein, K., Head, M., 2008). Their role is transformed regardless of their theoretical persuasion, because the technology and distance of the students will force that change somewhat. Delivery of information, motivation of students, interaction with students, assessment, etc., will all differ because of the medium of delivery. Instructors will require a differing level of technological sophistication depending upon the level of technical support available, but all will need some level of technological knowledge to be able to function in this environment (Wang, W. & Wang, C., 2009). Instructors in the past spent a great amount of their time in knowledge creation, either in the form of original research or by consuming that research (Jones, 2004). With E-Learning, they will be spending more time in the development

General Perspective in Learning Management Systems

Figure 4. LMS stakeholders

and delivery of learning materials than they have historically done (Wang, W. & Wang, C., 2009). Some studies have indicated that they may spend twice as much time in this process, even with the aid of support staff. This shift in time commitment will result in a shift in the role and expectation of instructors in the future.

IT Staff IT staff are the individuals who have traditionally only been involved in supporting the business side of the institution. If they were involved in educational delivery, it was to support systems that were used in traditional classrooms. They now are involved in systems that must be kept

operational 24/7 (Jones, 2004). These systems are mission critical, but are used by personnel who often times do not understand the intricacies of their function. While the end users will have some technical expertise, they will not generally be technologists, but will have some strong expectations from the systems that they use. This will increase the pressure on the IT staff to provide the services desired. IT staff will have to better understand the educational theories within which the system will function in order for the system to perform according to the needs and expectations of the users (students and faculty). It will no longer be sufficient for IT staff to just understand technology, but they will have to also be cognizant of

15

General Perspective in Learning Management Systems

the environment in which the technology will be implemented (Jones, 2004). It will also force the staff to include the end user in the design process from the very beginning and throughout the lifecycle of the product. These will be different roles for the typical IT staff.

Administrators Administrators in traditional institutions are primarily seeking to create easier access to their institution and remove geographic barriers to student participation (Wang, W. & Wang, C., 2009). In this manner, they are increasing their student population and expanding their academic offerings to a wider market. They are also trying to remain responsive to the market trends and the desire of their targeted market (Jones, 2004). They are attempting to expand the institutional system while maintain fiscal responsibility. In the process of expanding their delivery modalities, they want to maintain the quality of their course offerings so as to not destroy their brand image. This often requires that they need to pay attention to the broader issues of the Environment (the outer ring of our diagram). While the other stakeholders focus on a few of the other stakeholders, this group must pay attention to and balance the needs and concerns of all of the stakeholders. They also must focus on the future needs of the institution by establishing the strategic plans for technological innovation.

Environment The environment is the hodge-podge of others that have an interest in the outcomes of the eLearning endeavor (Jones, 2004). They encompass the various accreditation bodies that must certify the quality of the instruction offered. They are primarily concerned that minimum standards are met that all courses are comparable regardless of the modality of offering. While there are a number of accrediting bodies that focus on distance

16

learning/E-Learning, all accrediting bodies must certify some E-Learning offerings through institutional members. Another group that plays a major role in this process is the potential employer of the students in E-Learning programs. They want to ensure that students have received an adequate education irrespective of manner in which it was delivered. They want the education to provide content that is relevant to the work environment and that meets minimum quality standards. They also may become involved by supporting their current employees as they pursue further education to the benefit of the employer. In this manner they are also concerned with student access. The funders of education are an often overlooked stakeholder group. These are the entities that provide various aspects of the funding for educational institutions. They may be public funders in the form of government agencies or they could be private funders who provide grant funding or give to foundations that support the educational programs of an institution. They may be the parents who are funding their child’s education. Members of this group are primarily concerned that the funds will be utilized to provide a quality education to the students and ensure a respected outcome from their investment. All of these various stakeholders interact with one another seeking to have each other’s needs faced and resolved in the E-Learning effort. None of them stand isolated from the others and individually do not have a preeminent role to play in the eLearning offering. If one is ignored or their needs and desires are not met, it creates the potential for the eLearning effort to fail.

MODELS OF LEARNING MANAGEMENT SYSTEMS The various LMS models differ according to the selection of LMS elements (see Figure 5) and how they are applied to the system and therefore

General Perspective in Learning Management Systems

to the users of that system. While there are many elements that could be utilized to discriminate one LMS from another, we will focus on three. These are the primary elements that influence most decisions concerning the system of choice. The accompanying diagram identifies the three broad elements, but one must bear in mind that most systems are not just one or the other, but are in fact somewhere on a continuum between the extremes within each element.

Source In-House Initially all LMS were developed in-house and were dependent upon the support of the institutional programmers. Most of these systems possessed limited functionality and they were difficult to modify or improve. A few of those systems, like the PLATO LMS, were later transferred to businesses that have either used them in proprietary products or marketed them as proprietary systems. Most in-house developed systems, therefore, are no longer in existence, although there are some who are advocating a return to the in-house developed systems as a way to provide quality at a more affordable cost.

Proprietary Proprietary systems comprise the largest block of systems in operation today. While they all claim to provide the functionality needed by the institution, they are by no means identical. They differ on some very important factors and it would behoove the using institution to choose carefully. While they do differ on the basis of functionality, most provide a basic set of functions along the lines discussed in course management systems. They may or may-not interface with academic administrative systems or an online library system, for instant. They may include some Web 2.0 functionality, but it may be limited in utility. There is constant change within the industry, with companies coming and going by either leaving the industry or

being absorbed by one of the competitors. Some examples of these would be: Blackboard (which recently acquired WebCT Manager and Angel), Desire2Learn, ECollege, & .LRN. There are sites where one can compare these systems in order to decide which one is right for you. With these systems, you can either have a hosted environment, where some provider supplies the hardware and software to make the system work or you, or you can host it yourself on your own servers. There are many pros and cons to each approach and should be considered carefully.

Free/Open Source These systems are supported by a network of programmers around the globe who provide updates to the systems on an almost constant basis (Pan, G. & Bonk, C., 2007). Many of these systems are free to own, but are also often available through a supplier who will provide for the installation and support of the system. They can also be available as a hosted option. They provide a great deal of flexibility to the decision making process. They have traditionally been a niche market, but with Moodle gaining in popularity, they are gaining a larger market share. The basic premise of these systems is what is commonly known as Linus’s Law: Given enough eyeballs, all bugs are shallow (Pan, G. & Bonk, C., 2007). They assume a selfcorrecting mechanism in the peer review process that open-software engenders. This further pushes the costs of these products lower and encourages users to expand their functionality. The development of these products promotes common standards that establish those that are most beneficial to the market. This process also encourages the development of expertise in the broader market arena by encouraging many individuals to be involved in the development of the product.

17

General Perspective in Learning Management Systems

Figure 5. Elements of a learning management system

Time Time refers to whether the student and instructor are involved concurrently or separated by time, in other words, synchronous or asynchronous (Wagner, N., Hassanein, K., Head, M., 2008). Most LMS are designed to be operated in an asynchronous mode. In this mode, an instructor prepares the learning content and interacts with the students mostly on a one to one basis. The interaction does not occur in real (concurrent) time, but with time lapses. This creates the possibility of instructor-student disconnect as one has to wait for a period of time for the other to respond to the inputs. As the system supports a more dispersed student population, possibly spanning the globe, this becomes more of a necessity or least more

18

easily justified. However, there are systems that focus on a synchronous delivery mechanism, such as LaunchForce and LeanLine. These systems put the instructor and students online at the same time. These systems may involve, audio only, audio and video, whiteboard, program share, and other such technologies. In this mode, there is much more the virtual classroom experience with the ability to engender the collaborative involvement of all of the students and instructor in the learning exercises. Collaboration is possible in both synchronous and asynchronous modes, but is more natural in the synchronous environment. Some systems allow for both modes of delivery to be blended together. Such systems as ECollege, Embanet, Jones e-education, and LUVIT eLearning are thought of as total solutions. They

General Perspective in Learning Management Systems

allow delivery in the most effective time frame for the individual course. Many of these systems allow for the content that is delivered synchronously to be recorded and then delivered in an asynchronous fashion. There are tools that often can be utilized with other asynchronous systems to allow for the delivery of portions of the class in a synchronous mode.

Pedagogy Pedagogy is concerned with how a course will be delivered in instructional terms. It will affect the type of interaction that may be involved in the learning process, such as, student to content, student to instructor, and student to student (Hamuy, E. & Galaz, M., 2010) (Wang, Y. & Chen, N., 2009). In the student to content, the student interacts with instructional information, either provided through the course system or accessed by other means, textbook, online, etc. In the student to instructor mode, the student interacts with the content expert or experts to retrieve the needed learning content. In the student to student mode, the students will interact with one another to retrieve information or perform learning activities in a collaborative manner. Pedagogy will also determine the nature of the course structure(see Figure 6); will the course allow the student to move around and choose the order in which material is accessed and work is completed or must they move through in a predetermined orderly sequence (Dabbagh, 2005). Will the student be required to achieve mastery of the material to some predetermined level or will the student construct their own learning with the instructor serving as a mentor/guide? Pedagogy plays a fundamental role in the determination of the nature of the learning experience, whether the learning constructor recognizes the role or not. Pedagogy involves three broad aspects in its structure (Dabbagh, 2005). Each of these aspects plays some role in the impact of pedagogy on the learning process. Pedagogical constructs provides

the foundation out of which the others flow (Dabbagh, 2005). From the constructs will flow the pedagogical strategies that determine the teaching methods that are used in the program. From the strategies one will determine the pedagogical tools that will be need in instructional delivery. It is in this manner that pedagogy will determine the nature of the LMS used in a specific setting.

PEDAGOGICAL CONSTRUCTS These are the basic formulation of the teaching/ learning process (Dabbagh, 2005). They are the models or the components of the models that will guide the development and delivery of instruction (Sabry, K. & Barker, J., 2009). They arise out of one’s understanding of cognition and knowledge. They provide the mechanism for putting theory into practice (Dabbagh, 2005). Some major examples include: behaviorist, cognitivist, and constructivist. While these are not the only models used, most others grew out of them.

Behaviorist Behaviorist pedagogical systems will provide a structured/programmed approach to the teaching/ learning process. This pedagogical system sees the mind as a ‘black box’ that is not knowable and so is not a concern. No attention is paid to the internal processes of learning, in fact, many would say internal processes don’t exist, but this pedagogical system focuses on the measurable outcomes of learning. These outcomes therefore equal learning and rewards are the motivators that support the production of desirable outcomes. Learning is linear and structured. Theorists in this paradigm are Watson, Thorndike, Skinner, and Pavlov.

19

General Perspective in Learning Management Systems

Figure 6. Pedagogical structure

Cognitivist

Constructivist

The cognitivists focus on how the mind processes and uses information to produce learning. They are interested in the mental structures and processes that are necessary to explain human behavior (Dabbagh, 2005). They pay greater attention on the learner’s thoughts, beliefs, attitudes, and values in explaining the learning outcomes (Ardichvili, A. & Yoon, S., 2009). They are more focused on the learners’ differences and seek to accommodate them by varying the instruction material and processes. To them learning is an individualized process. Group activities are relatively unimportant and rarely addressed. They seek to provide organizational structure that allows the learner the ability to move through the material in a highly individualized fashion that ties the new learning to existing informational structures. Theorists in this paradigm are Gagne, Briggs, and Bruner.

Constructivists built off the cognitivistic view that the mind is more than a ‘black box’ responding to stimuli. It instead focused on the processes involved in learning, seeing those processes as internal and, therefore, not visible directly (Homberg, 2005). Constructivists see learning as an active process that works within a context to produce (construct) knowledge (Dabbagh, 2005) rather than just acquiring it. Learners must be actively involved in both acquiring and processing information subjectively (Baggaley, 2008). As learners actively process the information, they will gain a deeper understanding of the content (Ardichvili, A. & Yoon, S., 2009). The learner needs to be encouraged and enabled to search for the knowledge or solve problems on their own rather than provided the content or solutions (Ardichvili, A. & Yoon, S., 2009). All knowledge

20

General Perspective in Learning Management Systems

is seen as social in origin, developed while engaged in activities (projects), mediated by tools (Lytras, M. & Pouloudi, A., 2006). Instructors in this tradition will provide the learners with realworld simulations, collaborative experiences with others, or by providing them with the knowledge and ability to access knowledge at the time that it is needed (Ardichvili, A. & Yoon, S., 2009). Theorists in this paradigm are Dewey, Piaget, Vygotsky, and Bruner. These models/constructs are almost never used solely. They each will often find a place in some part of the E-Learning program. Most practitioners will have a preferred approach to the design and delivery of instruction, but will generally use some of the less preferred methods within their program to meet the needs of the widest number of students or to overcome areas of difficulty in the instructional program.

PEDAGOGICAL STRATEGIES The application of pedagogical constructs to the teaching process leads to three broad pedagogical strategic systems (Dabbagh, 2005). Each system seems to lean toward a specific set of educational methodologies/tools. There is an overarching perspective that guides the choices that are made by these educators in the development of the learning program. This produces the framework for E-Learning (see Figure 7).

Behavioral/Cognitive Strategy The Behavioral/Cognitive framework/strategy promotes facilitated learning. The teacher provides the guidance and direction that is needed for the student to learn. The primary educational methods/tools will be lectures, presentation, textbooks, and teacher directed discussions. The teacher is the source of information that leads to knowledge acquisition in this process. The process is very linear and focuses on measureable

outcomes. Practitioners may or may not pay any attention to individual differences or care much about the intellectual processes that are involved in developing the learning outcomes.

Constructivist Strategy The constructivist framework/strategy focuses on individualized learning. These practitioners see the process of learning as very interactive with the student interacting with the teacher and with the learning materials. The student plays the major role in the learning outcome. Students determine what is and is not important to learn. The teacher moves from the ‘sage on the stage to the guide on the side’ in this process. Learning activities and materials must be somewhat fluid in this process, because it is not possible to know what the student will need before the student needs it. The primary educational methods/tools used will be case studies, self-instructional materials, and questions and answers. The pace of learning will be dictated by the student and their desires.

Social Constructivist Strategy The social constructivist framework/strategy focuses on collaborative learning (Huang, S. & Yang, C., 2009). Learning is seen as a social endeavor and requires the collaboration between and among all participants. The teacher will very often be just one of the learners in this process. The group will determine the nature of the learning. Learning activities will vary with the class involved; the students will bring many of the materials from their own search for answers. The primary educational methods/tools used will be case studies, problem based learning, discussion forums, teamwork, and seminar sessions. The emphasis will be upon the group participation in the learning process with the group playing a major role in the decisions about what will transpire in the process. This is a very interactive strategy requiring consistent feedback from the teacher and fellow students. Learning is

21

General Perspective in Learning Management Systems

Figure 7. Pedagogical frameworks

seen as a process rather than an end. Control rests in both the teacher and students within a context of continuous communication.

PEDAGOGICAL TOOLS Your pedagogical strategy will determine the features, pedagogical tools, that your LMS will need to provide (Dabbagh, 2005) (Sabry, K. & Barker, J., 2009). If your strategy is a behavioral/cognitive framework, you can probably get by with a simple course management system that uses prepackaged course material to deliver instruction in a linear fashion with teacher graded outcomes measures. You will need a simple email communication

22

system to allow the students to ask questions of the teacher and allow the teacher to respond to those questions. You would also benefit from having a document delivery mechanism and a means for the students to submit their work and take exams. Most systems available today provide these features and more. If your pedagogical strategy is in a constructivist framework, you will need the entire aforementioned, but also some means for the student to navigate through the material individually and to set personal learning objectives. They will also need to be able to acquire their own learning materials to provide for their individualized learning plan. There will need to be flexibility in the assessment of learning outcomes to provide

General Perspective in Learning Management Systems

for the individualized nature of those outcomes. You will want to provide for simulations, blogs, message boards, etc. to allow the students to be able to construct their own learning activities and provide mechanisms for easy feedback from the teacher/coach. If your pedagogical strategy is a social constructivist framework, you will need everything already mentioned, but also include mechanisms for multi-way communication so that all participants can communicate with each other in a free and open manner. To be most effective, you will need to provide for both synchronous and asynchronous communication modalities. Students will need to work individually and in groups; as much as possible, the groups should be self selected.

FUTURE RESEACH DIRCTIONS Too much of research focuses on finding something new or defending a current stance on a subject. We need to have researchers who are willing to integrate the findings of others into some sort of a coherent system. This should involve the gathering of information from related disciplines so that we can improve learning management systems by using the knowledge from all appropriate sources. If we become too enmeshed in our limited view of the world, either by just looking at what is being produced in distance learning publications or by looking at only what is being done in our limited geographical sphere, we will not be able to meet the transformations that will occur in the future. We also need to focus on those aspects that will improve the quality of what we do.

CONCLUSION Learning management systems, while a modern phenomenon, have very deep roots in a broad array of disciplines. Each of these disciplines is still functioning today and produces impacts

upon LMS and their use within the educational environment. The nature of that impact is influenced by pedagogy. Pedagogy will drive what the LMS will have and how it will be used in the delivery of E-Learning. In most cases, teachers will not stick with one pedagogical framework, but will use aspects of all of them at some point in the delivery of learning. Students will benefit from this eclectic use of pedagogy. We have only scratched the surface of what is involved in the function of an LMS based on pedagogical decision making. The better the teacher understands the pedagogy underlying the decisions being made, the more effective the LMS will be in meeting those needs. Many LMS implementations fail because there was not a clear understanding of the pedagogical needs of the teachers using the system (Dillenbourg, 2008). This chapter has only lightly touched on the topics presented. It is not intended to be an exhaustive treatment of any of the subjects. It is hoped the readers will continue to pursue those topics of interest and to be aware of those of lesser interest. In this manner, we should all be able to understand the changes that have occurred and be prepared for those that will come.

REFERENCES Ardichvili, A., & Yoon, S. (2009). Designing integrative knowledge management systems: Theoretical considerations and practical applications. Advances in Developing Human Resources, 11(3), 307–319. doi:10.1177/1523422309337593 Baggaley, J. (2008). Where did didtance education go wrong? Distance Education, 29(1), 39–51. doi:10.1080/01587910802004837 Baturay, M. (2008). Analysis of a learning management system model. E-Journal of New World Sciences Academy, 3(3), 464–477.

23

General Perspective in Learning Management Systems

Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L. (1999). Design and delivery of integrated learning systems: Their impact on student achievement and attitudes. Journal of Educational Computing Research, 21(4), 475–486. Cradler, J. (2008). Research on e-learning. Learning and Leading with Technology, 3(5), 54–57. Dabbagh, N. (2005). Pedagogical models for e-learning: A theory-based design framework. International Journal of Technology in Teaching and Learning, 1(1), 25–44. Daniels, P. (2009). Course management systems and implications for practice. International Journal of Emerging Technologies & Society, 7(2), 97–108.

Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K. (1991). A comprehensive study of the effects of an integrated learning system. Indiana State University. Guri-Rosenblit. (2005). ‘Distance education’ and ‘e-learning’: Not the same thing. Higher Education, 49, 467–493. doi:10.1007/s10734004-0040-0 Hamuy, E., & Galaz, M. (2010). Information versus communication in course management system participation. Computers & Education, 54, 169–177. doi:10.1016/j.compedu.2009.08.001 Holmberg, B. (1995). Theory and practice of distance education, 2nd edition. London, UK, and New York, NY: Routledge.

Dillenbourg, P. (2008). Integrating technologies into educational ecosystems. Distance Education, 29(2), 127–140. doi:10.1080/01587910802154939

Holmberg, B., & Ortner, G. E. (1991). Research into distance education. Frankfurt am Main, Germany: Peter Lang.

Eteokleous-Grigoriou, N. (2009). Instilling a new learning, work and communication culture through systemically integrated technology in education. Systems Research and Behavioral Science, 26, 707–716. doi:10.1002/sres.983

Homberg, B. (2005). The Evolution, Principles and Practices of Distance Education (Vol. 11). Hagen, Germany: FernUniversität, ZIFF.

Gaeta, M., Orciuoli, F., & Titrovato, P. (2009). Advanced ontology management sytem for personalised e-learning. Knowledge-Based Systems, 22, 292–301. doi:10.1016/j.knosys.2009.01.006 Garrison, R. (2000). Theoretical challenges for distance education in the 21st century: A shift from structural to transactional issues. International Review of Research in Open and Distance Learning, 1(1), 1–17. Gaspay, A., Dardan, S., & Legorreta, L. (2008). Distance learning through the lens of learning models: New outlets for innovation. Review of Business Research, 8(4), 53–62.

24

Huang, S., & Yang, C. (2009). Designing a semantic bliki system to support different types of knowledge and adaptive learning. Computers & Education, 53, 701–712. doi:10.1016/j. compedu.2009.04.011 Jones, D. (2004). The conceptualisation of elearning: Lessons an implications. Studies in Learning, Evaluation, Innovation and Development, 1(1), 47–55. Keegan, D. (2002, November). The future of learning: From eLearning to mLearning. ZIFF Papiere 119. Hagen, Germany: FernUniversitat, Ziff. Kim, S., & Leet, M. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24, 284–294. doi:10.1111/j.1365-2729.2007.00260.x

General Perspective in Learning Management Systems

Lytras, M., & Pouloudi, A. (2006). Towards the development of a novel taxonomy of knowledge management systems from a learning perspective: An integrated approach to learning and knowledge infrastructures. Journal of Knowledge Management, 10 6), 64-80. Miller, T., & Hutchens, S. (2009). 21st century teaching technology: Best practicess and effectiveness in teaching psychology. International Journal of Instructional Media, 36 3), 255-262. Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating learning management systems: Adoption of hexagonal e-learning assessment model in higher education. Transforming Government: People, Process and Policy, 3(2), 11–130. Pan, G., & Bonk, C. (2007). The emergence of open-source software in North America. International Review of Research in Open and Distance Learning, 8(3), 1–17. Paulsen, M. (2002). Online education sytems in Scandinavian and Australian universities_A comparative study. International Review of Research in Open and Distance Learning, 3(2), 1–14.

Sabry, K., & Barker, J. (2009). Dynamic Interactive Learning Systems. Innovations in Education and Teaching International, 46(2), 185–197. doi:10.1080/14703290902843836 Taylor, J. (2001). 5th generation distance education. 20th ICDE World Conference on Open Learning and Distance Education (pp. 1-11). Dusseldorf, Germany. Underwood, J. (1997). Integrated learning systems: Where does the management take place? Education and Information Technologies, 2, 275–286. doi:10.1023/A:1018677616969 Wagner, N., Hassanein, K., & Head, M. (2008). Who is responsible for e-learning success in higher education? A stakeholder’s analysis. Journal of Educational Technology & Society, 11(3), 26–36. Wang, W., & Wang, C. (2009). An empirical study of instructor adoption of Web-based learning systems. Computers & Education, 53, 761–774. doi:10.1016/j.compedu.2009.02.021

Perry, B. (2009, June). Customized content at your fingertips. Training & Development, 29–31.

Wang, Y., & Chen, N. (2009). Criteria for evaluation synchonous learning management systems: Arguments from the distance language classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773

Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, J. (2008). Evaluating learning support systems usability an empiracle approach. Communication & Cognition, 41(1 & 2), 143–158.

Watson, W., & Watson, S. (2007). What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-007-0023-y

Rogers, L., & Newton, L. (2001). Integrated Learning Systems -- An ‘open’ approach. International Journal of Science Education, 23(4), 405–422.

ADDITIONAL READING

Rose, E. (2004). “Is there a class with this content?” WebCT and the limits of individualization. The Journal of Educational Thought, 38(1), 43–65. Saba, F. (2000). Research in distance education: A status report. International Review of Research in Open and Distance Learning, 1(1), 1–9.

Abel, R. humes, L., Mattson, L., McKell, M., Riley, K., & Smythe, C. (2007). Achieving Learning Impact 2007. August 2007. Retrieved from Http://www.imsglobal.org/learningimpact2007/ li2007report.cfm. Bates, A. (1995). Technology, Open Learning and Distance Education. London, UK: Routledge.

25

General Perspective in Learning Management Systems

Castells, M. (2000). The Information Age: Economy, Society and Culture: Vol. I. The rise of the Network Society. Oxford, UK: Blackwell. Cooper, P. (1993). Paradigm shifts in designed instruction: From behaviorism to cognitivism to constructivism. Educational Technology, (May): 12–19. Cristea, C. (2010). Education and media in the postmodern pedagogy. Petroleum – Gas University of Ploiesti Bulletin. Education Sciences Series, 62(1A), 102–106. Elias, T. (2010). Universal instructional design principles for Moodle. International Review of Research in Open and Distance Learning, 11(2), 110–124. Freishtat, R., & Sandlin, J. (2010).. . Educational Studies, 46, 503–523. Garrison, D. R. (1990). An analysis and evaluation of audio teleconferencing to facilitate education at a distance. American Journal of Distance Education, 4(3), 13–24. doi:10.1080/08923649009526713 Green, N., Edwards, H., Wolodko, B., Stewart, C., Brooks, M., & Littledyde, R. (2010). Reconceptualising higher education pedagogy in online learning. Distance Education, 31(3), 257–273. do i:10.1080/01587919.2010.513951 Gregor, S., & Benbasat, I. (1999). Explanations from intelligent systems: Theoretical foundations and implications for practice. Management Information Systems Quarterly, 23(4), 497–530. doi:10.2307/249487 Hinostroza, J., & Mellar, H. (2001). Pedagogy embedded in educational software design: Report of a case study. Computers & Education, 37(1), 27–40. doi:10.1016/S0360-1315(01)00032-X

26

Hulsmann, T. (2004). Guest editorial – Low cost distance education strategies: The use of appropriate information and communication technologies. International Review of Research in Open and Distance Learning, 5(1), 1–14. Keegan, D. (Ed.). (1993). Theoretical principlesof distance education. London, UK, and New York. NY: Routledge. Ley, T., Kump, B., & Albert, D. (2010). A methodology for eliciting, modellling, and evaluating expert knowledge for an adaptive work-integrated learning system. International Journal of HumanComputer Studies, 68, 185–208. doi:10.1016/j. ijhcs.2009.12.001 Lockwood, F. (1995). Open and distance learning today. London, UK, and New York. NY: Routledge. Mackenzie, O., & Christensen, E. L. (Eds.). (1971). The changing world of correspondence study. University Park, PA: Pennsylvania University Press. Moore, M. G. (2003). Network systems: The emerging organizational paradigm. [Editorial]. American Journal of Distance Education, 17(1), 1–5. doi:10.1207/S15389286AJDE1701_1 Moore, M. G., & Clarke, G. C. (Eds.). (1989). Readings in principles of distance education (pp. 29–37). University Park, PA: The American Center for the Study of Distance Education. Moore, M. G., & Kearsley, G. (2005). Distance education: A systems view. Belmont, CA: Wadsworth Publications. Ozkan, S., & Koseler, R. (2009). Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation. Computers & Education, 53, 1285–1296. doi:10.1016/j.compedu.2009.06.011

General Perspective in Learning Management Systems

Paulsen, M. (Ed.). (2003). Online education and learning management systems: Global e-learning in a Scandinavian perspective. Oslo, Norway: NKI Forlaget. Sellar, S. (2009). The responsible uncertainty of pedagogy. Discourse: Studies in the Cultural Politics of Education, 30(3), 347–460. doi:10.1080/01596300903037077 Sewart, D., Keegan, D., & Holmberg, B. (Eds.). (1983). Distance education: International perspectives. London, UK: Croom Helm (Routledge). Skinner, B. F. (1968). The technology of teaching. New York, NY: Appleton-Century-Crofts. Watson, D. M. (2001). Pedagogy before technology: Re-thinking the relationship between ICT and teaching. Education and Information Technologies, 6(4), 251–266. doi:10.1023/A:1012976702296 Zawacki-Richter, O., Backer, E., & Vogt, S. (2009). Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning, 10 6), 21-50.

KEY TERMS AND DEFINITIONS Behaviorism: The theory or doctrine that human or animal psychology can be accurately studied only through the examination and analysis of objectively observable and quantifiable behavioral events, in contrast with subjective mental states. Blended Learning: The use of both classroom teaching and on-line learning in education.

Cognitivism: A theoretical approach in understanding the mind using quantitative, positivist and scientific methods that describes mental functions as information processing models. Constructivism: A theory of knowledge (epistemology) that argues that humans generate knowledge and meaning from an interaction between their experiences and their ideas. E-Learning: Learning that is done at a distance using information communication technologies for delivery. Information Communication Technology: Consists of all technical means used to handle information and aid communication, including computer and network hardware as well as necessary software. Instructional Television Fixed Service: A band of twenty (20) microwave channels available to be licensed by the U.S. Federal Communications Commission (FCC) to local credit granting educational institutions. Pedagogy: Refers to strategies of instruction, or a style of instruction. Programmed Learning: A learning methodology or technique first proposed by the behaviorist B. F. Skinner in 1958. It has three elements: (1) it delivers information in small bites, (2) it is self-paced by the learner, and (3) it provides immediate feedback, both positive and negative, to the learner. Video Conferencing: A set of interactive telecommunication technologies which allow two or more locations to interact via two-way video and audio transmissions simultaneously.

27

28

Chapter 2

Knowledge Sharing in a Learning Management System Environment Using Social Awareness Ray M. Kekwaletswe Tshwane University of Technology, South Africa

ABSTRACT The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness.

INTRODUCTION This chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms, as

learners share knowledge. The chapter argues that to be able to design LMS that ought to enable social interaction among learners not confined to particular places, we must first understand how learners interact and the tools they use. In this chapter, learners use social awareness to determine

DOI: 10.4018/978-1-60960-884-2.ch002

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Knowledge is not a fixed commodity, but a function of our interactions with external resources including tools, media, and other humans (Ryder & Wilson, 1997). This suggests that human knowledge transforms as people socially interact with others and the surrounding environment. Consequently, the chapter is premised on the notion that knowledge is created and transferred through the dynamic interactions among individuals and between individuals and their environments (Nonaka, 1994). Thus, knowledge sharing is social and sensitive to context. It is inferred from the notion of knowledge creation, sharing or transfer that knowledge can be perceived to transform when context and social presence awareness interact. In this regard, context and presence awareness influences the interaction and the problems that could be solved and how they are solved. In this chapter, social awareness is synonymous with context and social presence awareness. Although a great number of studies (e.g., Shariq, 1999; Polanyi, 1966; 1958) have shown that knowledge creation and transfer is essentially a human-to-human process or an outcome of social interaction (Nonaka, 1994), the relationships or roles of context and social presence awareness as catalysts for knowledge sharing and transformation in a learning environment has not been explored. This chapter aims to contribute to that effect. This chapter is about exploring and understanding how, through a learning management system environment, a learner uses social awareness to leverage personalized knowledge sharing. It reveals the actual nature of ubiquitous learning through social interaction where awareness of context and social presence is argued to be the underlying process of the activity. The chapter is on how varied forms of communication for knowledge sharing in an LMS learning environment are an outcome of social interaction coordinated by social awareness. Social awareness is synonymous

with awareness of context and social presence. Context is understood as the situation in which a learner or a group of learners find themselves. Accordingly, context is defined as any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application (Dey and Abowd, 1999). Social presence is re-defined and understood to be the mediated presence of another learner who could provide personalized on-demand knowledge support for a learning problem as the learner traverses varied learning contexts. Learners in contact universities come from varied social backgrounds, with diverse languages and cultures. To this point, one of the prevailing educational challenges is that of providing personalized academic support to under-prepared learners (Jaffer et al, 2006). Awareness of the social environment and social resources is, therefore, fundamental to the provision of personalized academic support to a learner. Learning is made ease when a learner has consistent awareness of context and presence of social resources (Kekwaletswe, 2009; 2007). Ubiquitous personalized knowledge support refers to the provision of context sensitive and anywhere, anytime help as learners traverse varied locations. Learners use awareness of context and social presence as a means to access ubiquitous learning support, interpret and adjust their knowledge – sharing what they know with others through social interaction. The chapter, thus, focuses on the peer-to-peer interaction and the learning environment. The social interaction whose outcome is transformed knowledge and provision of support is location and time independent. Since sharing learning experiences is a ubiquitous phenomenon, learners continuously use awareness of the environment and awareness of available social resources they can draw upon to facilitate knowledge consultation (Kekwaletswe, 2009: 2007). This chapter is about the advancement of the human-centric approach to knowledge creation

29

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

and sharing through enhanced person-to-person interaction – where context and social presence awareness is of vital significance to how learners create, use, and share knowledge. The chapter uses the contextual inquiry research method to understand how a learner goes about sharing knowledge, and the tools they use. The practical contribution of the chapter is the understanding of learning management systems environments and how learners use social awareness to model their actions for the provision of personalized academic support. The rest of the chapter is as follows: firstly, the background to the research problem is given; secondly, the theoretical foundations for the chapter are articulated followed by the research methodology; lastly, the chapter discusses, with examples of empirical evidence, how social awareness is used to leverage knowledge sharing in the learning environment. The chapter is then concluded.

BACKGROUND TO THE RESEARCH PROBLEM The practical relevance of the chapter is towards enriching personalized on-demand academic support through social awareness. To this point, this chapter focuses on LMS communication tools appropriated by awareness of learners’ context. The ideal social awareness for knowledge sharing is one that is sensitive to the background of a learner (social context includes culture and language), arranging these aspects to provide immediacy on the available social network, independent of the learner’s location and task at hand (Kekwaletswe and Ng’ambi, 2006). The focal point of the chapter is therefore how social awareness is used to support personalized social interaction for a South African university learner as s/he traverses varied learning contexts. The objective of this chapter is not to confirm previously established premises and theories but to find out, through engagement with learners, how they, as they traverse varied learning

30

locations, interpret and use social awareness for interaction whose outcome is knowledge sharing. I argue that the effectiveness of a knowledge sharing within a learning management system is fundamentally affected by the social interaction and the context in which a problem-driven learning activity takes place. In the chapter, learning through a learning management system environment is defined as any sort of learning and knowledge sharing that happens due to social awareness when the learner is not at a fixed, predetermined location – in varied learning contexts. An LMS learning environment conveys ubiquitous learning that is not confined to specific locations, and is time independent. Context aware ubiquitous learning is meant to support learning by identifying a learner’s surrounding contextual environment and social presence to provide a rounded and seamless learning experiences. In a contact university, there tend to be disproportionate access to available social resources between when learners are attending scheduled or formal classes and when they are away from scheduled classes (Kekwaletswe and Ng’ambi, 2006) especially as they move away from campus locations. The challenge for universities is that instructors and tutors are not always available to provide learners with ubiquitous support as learners move away from formal locations or contexts. The alternative for these learners is to consult with a knowledgeable peer who, for the most part, shares a background. There are three types of contexts within which a learner is mobile and for which a learner needs a learning management system and social awareness as a medium to transform knowledge. The following three learning contexts have been discussed by Kekwaletswe and Ng’ambi (2006).

Formal Learning Contexts These contexts represent formal structures – such as scheduled lectures and laboratory sessions – in which a learner’s behaviour and action is shaped according to the university class timetable. Inter-

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

action in these spaces is usually one-way from instructor to learner using English as the official language of instruction, although learners often interact with each other using their own diverse languages. In the formal learning context, the instructor delivers a lecture and a learner either takes notes or is given a handout. Even though learners are invited to ask questions, there is little time to assimilate the material and meaningfully engage with the learning materials (Ng’ambi, 2004). Learning and knowledge seeking action is, therefore, mostly passive. In this context, social presence is usually availed through wired PCs in the computer labs and the face-to-face presence of tutors, instructors and peers.

Semi-Formal Learning Contexts These contexts represent informal spaces on campus used by learners, usually while waiting for the next lecture to start or after it finishes. They include the library, cafeteria, mingling areas and walk-in laboratories. As learners begin to reflect on the previous lecture and skim through the learning materials, questions begin to arise for which clarifications are required (Ng’ambi and Hardman, 2004). The challenge a learner is faced with is how to find an instructor or tutor who is available for immediate or on-demand consultation. Most instructors schedule consultations and are often unavailable for ad hoc interactions. The dilemma for most learners is that the consultation periods are limited and not always suitable (Ng’ambi, 2004). Consequently, the learner’s alternative is to find the nearest socially present knowledgeable peer or class-mate who can provide social support (Kekwaletswe and Ng’ambi, 2006). In the semi-formal learning context social presence is still availed through wired PCs in the computer labs and face-to-face presence of peers. Since the environment is on campus, where most learners meet and come for formal classes, a learner is still very much aware of the available social network.

Informal Learning Contexts Although it is difficult to be explicit on the characteristics of an informal learning context, these contexts include working during after-hours or weekends at university residences or private homes. In these environments, a learner usually uses his or her mother tongue to consult with peers (Kekwaletswe, 2006) or may write down questions to ask the instructor when they do get into the formal context (Ng’ambi, 2004). There are three things that must be known to provide ubiquitous personalized learning and knowledge support to a mobile learner in an informal learning environment: a) Knowledge about the location of a learner so as to help identify the potential knowledgeable peer; b) the preferred language of a learner in which he or she is likely to be conversant, and c) the awareness of a peer’s social presence and contexts – including location and situation (Kekwaletswe, 2006).

The Research Problem One of the prevailing educational challenges in the new South Africa is that of providing personalized academic support to under-prepared learners (Jaffer et al., 2006). In a contact university (as opposed to distance learning university), where students attend formal lectures and scheduled laboratory sessions, there tends to be inconsistency in social presence or access to available social networks for academic support between when learners are on campus and off campus (Kekwaletswe, 2006). Based on empirical evidence, I argue that in a learning environment that has a learner population with diverse social backgrounds and languages, social awareness of peers with a shared background is fundamental in the provision of personalized academic and social support. I distinguish between a learner being on campus in a formal context and being on campus or off campus in informal contexts.

31

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Since knowledge sharing and learning tasks are not confined to particular locations but are carried across different learning contexts, I argue that learning and social resources available to learners ought to move with a learner. I use the term social resources pragmatically to mean knowledgeable peers, tutors, instructors and the awareness of context and presence (Kekwaletswe, 2006; Kekwaletswe and Ng’ambi, 2006). In this regard, the problem is that of ensuring that the quality of resources available to learners remains consistent for supporting a knowledge transforming task or action regardless of time and location of a learner (Kekwaletswe, 2006). In other words, learners should have a consistent social awareness of presence and context. The argument is that when learners solve and engage in a learning task – problem based learning – they bring prior knowledge and experience to the social interaction situation, where the knowledge transformation outcome is influenced by social awareness. Thus, learning management systems ought to leverage this. But first, we need to understand how LMS communication tools are and could be used. In conventional interaction with the learning activity, learners have an impoverished mechanism for providing social awareness of the available social network. Consequently, personalized social support is denied when learners do not have the opportunity to access a consistent social network (ibid.). In order to understand social presence and context awareness and their role in learning system environments, we must understand both what the context is and how social awareness can be used to facilitate the ubiquitous learning. An understanding of the context and a sense of social presence enables the learner to model behavior along the expectations or the shared understanding of a social learning community.

32

THEORETICAL FOUNDATIONS This section lays out the epistemological and ontological foundations of the inquiry and reviews the existing literature that informs the chapter. The inquiry is about learning management system environments that support learning and sharing of knowledge, with context and social presence awareness as catalysts for learning actions. The theory of knowledge cannot be divorced from the situations under which knowledge and learning experiences take place and this is reviewed next, followed by a specific focus on learning environments. Social presence and context for interaction in such environments also receive special attention in this section.

Knowledge To understand knowledge and how it is shared in a learning environment as an outcome of social interaction, I unpack the logical development of how knowledge is created, retained, and used. The construction of knowledge requires processing of data into information, new information is then created which is then communicated or transferred outside of the human brain. With that premise, I briefly define data as raw facts that can be shaped and formed to create information. Thus information is data that is given a meaning within a context. Therefore, only data and information can be captured, transferred or stored outside the brain. Knowledge is created from the processing of information, and during this processing, new knowledge can be acquired or created for future use, when more or new information is acquired and processed (Van Beveren, 2002). Knowledge is transformed into information within the brain to be communicated externally through language or demonstration (ibid.. 2002). Language may include different forms of communication such as text, verbal and body gestures. What then constitutes knowledge?

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

There are probably as many definitions and explanations of knowledge as there are theories of knowledge from diverse disciplinary perspectives. Without joining the philosophical debate of what exactly knowledge is, it is encouraging to notice that literature has adopted a pluralistic epistemology, acknowledging that there are many types or forms of human knowledge. The following are only some of the definitions of knowledge. •

•

•

•

Knowledge is an individual’s stock of information, skills, experience, beliefs and memories (Alexander et al., 1991). Knowledge is the stock of conceptual tools and categories used by humans to create, collect and share information (Laudon and Laudon, 1995) Goffman (1978) explains knowledge by asserting that the “making of knowledge is a way of enacting reality, giving existence to things and events, and organizing the world.” Nonaka (1994) deems knowledge to be “justified true beliefs”. The theory of knowledge creation sees knowledge as a dynamic human process of justifying personal beliefs as part of an aspiration for the “truth”.

Generally, scholars of knowledge observe knowledge in two ways: “know how” and “knowthat”. The former is created ‘here and now’ in a specific, practical context and conveyed through analogies and metaphors; the latter is contained in manuals and procedures and oriented towards a context-free theory (Patriotta, 2003).

Knowledge as a Multifaceted Phenomenon The evolution of institutional knowledge has been informed by a wide spectrum of theoretical traditions. Knowledge is a multifaceted phenomenon which has been debated in a variety of disciplinary contexts – from philosophy and sociology, to

social psychology and cognitive science; from economics to management and organization analysis. The breadth and depth of the subject would not allow me to trace a lineage of existing knowledge theories. Nevertheless, cognitive theories have looked at knowledge as a representational phenomenon. Winograd and Flores (1986) point out that “a cognitive being ‘gathers information’ about things and builds up a ‘mental model’ which will be in some respects correct (a faithful representation of reality) and in other respects incorrect. Knowledge is a storehouse of representations, which can be called upon for use in reasoning and which can be translated into language while thinking is a process of manipulating representations” (op. cit., p73). In cognition-action theory, the foundational hypothesis is that action always possesses a cognitive basis which is reflected in the representational activities of the mind. The duality of cognition and action underlies the conceptualization of knowing as a computational activity (Patriotta, 2003). Since human behaviour is always oriented towards a goal, action is a form of problem-solving, where the actor’s problem is to find a path from some initial state to a desired goal state, given certain situations along the way. Accordingly, there is a need for researchers to understand how learners acquire new problem representations for dealing with new problems. A general theme uniting many situated approaches to cognition is a change in the way the personenvironment relationship is envisaged. Rather than a person ‘being’ in an environment, the activities of person and environment are parts of a mutually constructed whole. The inside-outside relationship between person and environment is replaced by a part-whole relationship (Bredo, 1994). Learning is centered around problem-solving and is intricately related to the context; ‘context’ here means understanding (a) the problem’s conceptual structure as well as (b) the purpose of the activity and (c) the social milieu in which it is embedded (Scribner, 1987).

33

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

The above briefly highlighted the multifaceted views on learning and knowledge and how it is created. The highlight is what informs my own definition of knowledge sharing. Having given an overview of what knowledge is and how it may be created, the following section will highlight the role of social awareness in transformational learning. In order to understand social presence and context awareness and the degree of their role in a learning management system environment, we must understand both what the context is and how the social awareness can be used to facilitate the ubiquitous learning.

Social Presence Theory In this section, the concept and Theory of Social Presence is discussed. As I examined the social presence studies and literature, it became apparent that most, if not all, of the studies on the concept employ a positivist approach. Whilst this chapter employed an interpretive approach, it is essential that I highlight the extent to which the theory and concept has been explored in previous research – which in turn informed the redefinition of the concept as used in this chapter. Short, Williams and Christie (1976) asserted that different communication media express varying degrees of social presence based on their ability to transmit nonverbal and vocal information. Thus, they initially introduced the Social Presence Theory as “technical social presence,” defining it as the capacity of the medium itself to present the “salience of the other person in interpersonal interaction” (p65). Two concepts associated with social presence are “intimacy” (Argyle and Dean, 1965) and the concept of “immediacy” (Wiener and Mehrabian, 1968). Intimacy depends on factors such as physical distance, eye contact, facial expression and personal topics of conversation. Immediacy is a measure of the psychological distance which a communicator puts between himself and the object of his communication.

34

This notion of Short et al. (1976) was, however, questioned by communication researchers (Gunawardena & Zittle, 1997; Byam,1995; Walther, 1994) who showed that perceived social presence in mediated interactions varies among participants in the same mediated conversations. That is, many of their research participants perceived mediated discourse as more personal than traditional classroom discussion. The claim by Short et al. (1976) that the quality of the communication media determines its social presence or richness was also disputed by Ngwenyama and Lee (1997) who showed that the communication richness of a media is dependent on who uses the media and how they use it. Gunawardena and Zittle (1997), for example, defined social presence as “the degree to which a person is perceived as ‘real’ in mediated communication” (op. cit., p8). They, like Ngwenyama and Lee, also argued that social presence was as much a matter of individual perceptions as an objective quality of the medium. In a survey to measure students’ perceptions of the social presence of others in a computer conference, Gunawardena and Zittle found that perceived social presence predicted more than half of the variance in students’ satisfaction with the conference. Their results also indicated that students who felt a higher sense of social presence enhanced their computer communication using emoticons to express missing nonverbal cues in textual form. Rourke, Anderson, Garrison and Archer (2001) regard social presence as one of the three fundamental “presences” that support learning, the other two being cognitive presence and teaching presence. Thus, they define social presence as “the ability of learners to project themselves socially and affectively into a community of inquiry” (p50). Rourke et al. identified three categories of social presence indicators – affective experience, cohesive experiences, and interactive experiences – and explored their use in online discussion. Affective experience contain personal expressions of emotion, feelings, beliefs, and values; Cohesive

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

experience are communication behaviours that build and sustain a sense of group commitment, such as greetings and salutations and group or personal reference; Interactive experience are behaviours that provide evidence that others are attending, such as agreement/disagreement, approval and referencing previous messages. There is, evidently, a lack of social presence research in learning management systems environment. Most studies of social presence have focused on the nature of online discussion and accordingly conceptualized social presence as a single construct with an emphasis on perceptions of the presence of peers (e.g., Swan, 2003 & 2002). As noted by Richardson and Swan (2003), there is some indication that instructor social presence may be equally important. Social presence of instructors has been considered in explorations of “teaching presence” (Shea et al., 2003; Anderson et al., 2001). Researchers have demonstrated both that students perceive the presence of others (Picciano, 2002; Gunawardena & Zittle, 1997; Gunawardena, 1995) and that they socially present themselves (Swan, 2003 & 2002; Rourke et al., 2001) in online course discussions. Nevertheless, there is lack of studies – positivist or interpretivist – looking at learners perceiving the presence of others and socially presenting themselves in a learning environment or context. This chapter addresses this shortcoming and it does so by following an interpretive tradition, diverting from the positivist tradition. In the chapter learners use awareness of a social presence for purposes of social interaction whose outcome is knowledge sharing. In this chapter, social presence is defined and understood to be the mediated presence of another learner who could provide personalized on-demand social support for a learning problem as the learner traverses varied learning contexts. Context and context awareness are also fundamental concepts in a learning environment where a learner is not fixed to particular locations. The concepts of context and context awareness are discussed in the next section, in view of the fact that they form the second phenomenon of the chapter.

Context Awareness Context and context-awareness are fundamental concepts in a learning environment where a learner is not fixed to particular locations. The following discussion of context and context awareness studies and literature is intended to show how the concepts are relevant to learning and ubiquitous social interaction. Lonsdale et al. (2003) describe context as a set of changing relationships that may be shaped by the history of those relationships. The figure below gives their hierarchical description of context as a dynamic process with historic dependencies. In Figure 1, a snapshot of a particular point in the ongoing context process can be captured in a context state. A context state contains all of the elements currently present within the ongoing context process that are relevant to a particular learning focus, such as the learner’s current project, or a learning activity. A context substate is the set of those elements from the context state that are directly relevant to the current learning and application focus, that is to say, those things that are useful and usable for the current learning system. Context features are the individual elements found within a context sub-state. Each feature is atomic and refers to one specific item of information about the learner or his/her setting. In implementing context awareness within their architecture, Lonsdale et al. (2003) derive a context sub-state and use the context features contained within it to determine what content might be appropriate for a learner. Context related to the human environment is structured into three categories (Schmidt et al., 1999): •

•

Information on the user; knowledge of habits, emotional state, bio-physiological situations. The user’s social environment; co-location of others, social interaction, group dynamics.

35

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Figure 1. Context hierarchy (Lonsdale et al., 2003)

•

The user’s tasks; spontaneous activity, engaged tasks, general goals.

Context related to the physical environment is also structured into three categories: • •

•

Location; absolute position, relative position, co-location. Infrastructure; surrounding resources for computation, communication, task performance. Physical situations; noise, light, pressure (Schmidt et al., 1999).

An entity needs contextual information to choose between alternative strategies in order to reach its objectives, or to do useful work. Context gives hints about what is or what is not achievable (Rakotonirainy et al., 2000). If the learner’s initial knowledge sharing or transfer objectives are not reachable then they are changed to suit the current context, i.e., the objectives change or actions are taken according to the context to maintain an objective. Abstract context awareness then

36

means being aware of what you can know about your context while concrete context awareness means being aware of what you do know about your context (op. cit.).

RESEARCH METHODOLOGY Empirical Inquiry in Formal Learning Contexts Interactions of learners in formal learning contexts at the University of Cape Town – as a contact university – are usually passive (Ng’ambi, 2004). That is, learners who are mobile hardly need to actively interact with each other in a formal lecture. The assumption is that it is mostly outside the formal contexts that university learners, notably at UCT, begin to interpret and engage effectively with learning materials and therefore need social presence and context awareness about available peers for social support. Since awareness of context and social presence is not much of a need for learners in formal environments, the

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

empirical inquiry does not focus on formal learning contexts. The empirical evidence suggests that social interaction of learners, where they begin to simplify meanings and begin to apply what they have learned in class, happens mostly outside of the formal learning context. The argument is that South African learners, who traverse varied locations, need more personalized academic support through social awareness as they move away from formal learning contexts. In view of this, the empirical study and evidence does not include learning management system environments in formal learning contexts but focuses on semiformal and informal learning contexts. The empirical evidence was gathered at the University of Cape Town campus and residences using contextual inquiry methodology. Contextual inquiry is a field research framework that depends on conversations with users in the context of their work (Holtzblatt and Jones 1994). It is based on ethnography, where the researcher goes into the research participant’s own environment. It is an explicit step for understanding who the user really is and how the work progresses from day

to day (Beyer and Holtzblatt, 1998). Essentially, contextual inquiry consisted of observing learners’ actions and talking with learners in their learning environment while they were engaged in authentic learning tasks. The focus of the contextual interviews and textual interactions was on the learners’ mundane activities, notably problem-driven social interactions related to learning. In Figure 2, learners use social awareness to interact with others who are not in the same location using Web-based mediated communication to send and read emails or chat through instant messaging. Learners interacted with learning resources, posing or responding to questions via a Web-based learning management system. In the figure, learners also use mobile devices such as Portable Digital Assistants (PDAs) to interact via mobile instant messaging, and mobile phones that could be carried around to interact verbally or send short text messages (SMSs). The study began with sixty learners completing a qualitative questionnaire meant to understand communication tools, forms of interaction, social presence and context awareness. The initial sixty

Figure 2. Representation of the research framework

37

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

learners were randomly identified as they traversed the university campus and in libraries and laboratories. There were no specific criteria for selection other than that the mobile learners had to represent or give a true reflection of typical learners at the University of Cape Town. Twenty of the sixty were asked to take part in the Web-based LMS and PDA pilot based on their willingness and commitment to be available for the duration of the study. Since the study was on understanding how university learners communicate, the tools they use and how they exhibit awareness, it was not necessary to determine who should participate or not participate in the study, other than that they should be registered university learners. The WiFi-enabled PDAs were to facilitate the “anywhere anytime” online learning interactions and the opportunistic mobile instant messaging. Eight of the twenty learners were interviewed and observed as they interacted with others in authentic LMS learning environments. Although this study was not on a particular LMS platform, participants also used Vula, a university SAKAI-based online learning environment. The textual interactions were in the form of chats and instant messages. The contextual verbal interviews were recorded using a digital recorder and transcribed. The storylines were in the form of thick descriptions including all the environmental details that could be observed and documented. Pictures capturing the actions and environment situations were constantly taken. The focus of the inquiry was on how learners interact, the tools they use and how awareness is used to support personalized learning. In the next section, the empirical evidence is discussed in terms of how learners interact, the appropriateness of the tools they use and how social awareness manifest in a learning environment. It is worth noting that the aim of the chapter was not to study a specific LMS platform but to study how knowledge is shared through awareness of context and social presence – awareness determines learning actions and the appropriateness of LMS tools. 38

KNOWLEDGE SHARING IN A LEARNING MANAGEMENT SYSTEM ENVIRONMENT USING SOCIAL AWARENESS Socio-cultural theories of mediated learning suggest that what is learned will emerge from the relationship between human action, and the social, cultural, institutional and historical contexts in which action occurs. This makes it essential for us to understand these contexts and activities before we begin to investigate issues of learning (Sutherland et al., 2000). In view of what Sutherland et al. (2000) suggests, it is first important for us to understand how social awareness manifest in a learning environment if we were to design appropriate learning management systems that could leverage knowledge sharing and learning actions. This section discusses how social awareness manifest amongst learners sharing learning experiences. In earlier sections, I noted that one of the prevailing challenges in the new South African higher education is that of providing personalized support to under-prepared learners – who by the very nature of the South African population come from diverse backgrounds and cultures. In most African cultures, it is uncommon for younger people to interact with or question their elders. This phenomenon tends to apply in an educational environment where learners are not comfortable questioning the instructors. The alternative for these learners is then to consult and interact with their close friends and knowledgeable peers to provide academic support. For this reason the inquiry focused on social presence of peers and not “teaching presence”. Consequently, this chapter is about interactions amongst learners (peers) for purposes of knowledge sharing, as they traverse varied learning locations. However, it is worth noting that in South Africa, availability and access to wireless and wired computers, remains a challenge. These resources become even more inadequate as you move away from campus. The

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

study was therefore guided by the three localized learning contexts discussed in section two, which apply to a typical South African contact university. The framework depicts social awareness and presence in three different learning contexts of the mobile learning environment. Social awareness is a mental concept where a peer and a learner become aware of the social network that follows them as they move across the varied learning contexts In Figure 3, a learner is consciously aware of available knowledgeable peers should s/he encounter a learning problem for which s/he needs to consult. By the same token, a peer is consciously aware of the presence of other learners should s/he not be able to address a problem encountered by a learner. That is, a learner and a knowledgeable peer have a consistent social awareness of a social network (social resources) regardless of their location and context. Even though learners interact via Web-based environments, in this study, the social awareness and availability of social resources in

remote locations was mostly mediated by mobile technologies, e.g., mobile phones and PDAs. Knowledge sharing involves two actions: transmission (sending or presenting knowledge to a potential learner) and absorption by the audience (Davenport and Prusak, 1998). If knowledge is not absorbed, it has not been shared. In other words, merely making knowledge available is not sharing. In this regard, social interaction achieved and enhanced by awareness of context and presence is necessary. The research investigated how learners use social awareness – social presence and context awareness – to enable a knowledge sharing interaction. It sought to understand ubiquitous learning where learners support others as they traverse the three learning contexts. Communication or interaction that is mediated by technology is generally grouped into two categories: asynchronous and synchronous. Asynchronous communication occurs between learners independent of time and location. This kind of communication does not need the send-

Figure 3. A framework for mobile learner-to-learner environment (Kekwaletswe, 2006)

39

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

ing and the receiving learners to be “available” concurrently. Examples include leaving a phone voicemail, posting to or reading a discussion board, and sending and receiving email (although this could also be considered synchronous). On the other hand, synchronous mediated interaction is considered a “real-time” experience between two or more learners. Examples of tools that facilitate synchronous communication include telephones, audio-video conferencing software, instant messaging, virtual chat, virtual classrooms, and whiteboards. Asynchronous and synchronous mediatedcommunication can be used in individual or group learning situations, as well as traditional or online learning environments. Although a significant body of research validates the notion that learning is a social act, learners may still acquire knowledge in a mediated interaction. There are several forms of technology mediated communication, such as electronic mail, web-based consultation environments, instant messaging on wired networks and short message service (SMS) on mobile phones. Although they support mobility of a learner and may afford presence awareness, they do not do so in real-time. In the study, there was a need to select a mediating technology that supports mobility of a learner as well as presence awareness of available peers in real-time. Thus, mobile instant messenger (IM) Jabber client for Pocket PCs (PDAs) called iMov messenger (www.jabber.org) was selected. The mobile instant messaging on PDAs supports an immediate on demand formal or informal expressive interaction. Such real-time interaction could be one-to-one or several concurrent dyadic conversations. Mobile IM provides mobile learners with a real-time interactive space to share learning experiences – exchanging textual messages that do not require detailed email-like messages or face-to-face interactions. Dey and Abowd (1999) suggest that contextaware applications look at the who’s, where’s, when’s and what’s (that is, what the user is doing) of entities and use this information to determine

40

why the situation is occurring. In this chapter, the context-aware learning interaction environment supports (who) a learner as s/he engages with the (what) learning materials, (where) whether in semi formal or informal learning contexts (when) anytime through the use of context awareness and social presence mechanisms. In this chapter, social awareness, and not a learning management system, does actually determine why a learning situation is occurring.

Social Awareness in a Learning Environment In semi and informal contexts, learners engaged in a learning activity are able to use implicit and explicit context awareness to increase or decrease the interaction whose outcome is learners “helping” or “supporting” each other. The notion of social interaction presupposes an existence of two or more learners speaking or acting. And also, embedded in this notion is the assumption that a learning community is socially present without which interaction – “help” or “support” – is impossible. The following tables give examples of how learners use implicit and explicit context and social presence awareness to increase or decrease social interaction whose outcome is personalized learning. To help the reader understand how awareness manifests, the examples are structured as follows: what the researcher wanted to know, the learner’s experience and a brief discussion. The experiences are grouped in a table form per research theme and query. Showing the empirical evidence this way was to allow the reader to see where the low level research questions (elements of awareness) that support the main question are addressed. Table 1 shows how learners are conscious or aware of the situation and the context in which they are interacting and sharing experiences with others, e.g., how fast they would need the response, the content of interaction, etc. It also highlights how presence and availability help the decision

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

regarding how to communicate. The different mediating tools offer different opportunities and accessibility. The preceding tables showed just some of the examples of how awareness of context and social presence manifest in a learning environment. In a learning environment, the central activity is knowledge where peers provide personalized social support through social interaction. Learners

acquire knowledge and experience through social interaction, drawing upon available social resources for the solution of practical learning problems. The empirical evidence showed that social awareness of the social resources, the learning task, and the environment does influence the action a learner undertakes – hence the appropriateness of an LMS communication tool.

Table 1. How does a learner share knowledge and learning experiences with others? Researcher’s Intention

Different Learners’ Response / experiences

Researcher’s Comments

How does a learner usually communicate with peers, besides face-to-face meetings, when faced with a learning problem and/or needing help?

…through SMS, …I call or email, …I use my mobile phone, …we do online chat, …I facebook sometimes …mobile phone chat (viamiXit). NB: MiXit is a mobile phone service that allows friends to chat (send/receive text) instantly and in real-time.

Here the learner’s intention is to seek help or share knowledge about a learning problem. He or she picks up an appropriate tool that best mediates the thought.

What is a learner’s reason for communicating using a particular tool?

…Depends on how desperate I am, if desperate I phone, if not I SMS. ...Depends on what I want to say …It depends on the situation at the time …Depends on the seriousness of the situation …Availability of funds …Economic limitations and time available …It depends on where I am at the time that I need to contact them.

The experiences show evidence of the learners’ situation as awareness of context. Location, cost, need and reason to interact are taken as context awareness which influences how learners consult each other, e.g., calling will give immediate response.

How does a learner decide which method of interaction to use?

…With email, it.s efficient because I can phrase the issues better and they can review and reply with specific and thorough answers. …It depends on the issues I need to discuss or message I need to get across. ...How they will respond. …With SMS and calls, the response is faster and I can get help I need in time. …It is the fastest way to get feedback from them. …Whichever will reach the fastest in terms of being seen by the person, and the immediate need. …It.s quick and direct (Mobile phone calls). …They are always with them (peers always carry their mobile phones) ...I know they will always have their cell ...Depends on if they are available, and what will be quicker at that moment. ..If they are available (via mobile phones)

Learners use content of the interaction as context for which a mediating tool is chosen. Learners decide on the method of interaction with peers based on how quickly and fast they need the response. Where a short message is enough for the interaction, they text each other. Issues of time are noted as an influencing factor in picking a mediating tool, noting that most interactions are done on-demand to solve a learning problem instantly. Learners use social presence awareness as an influencing factor for choosing a specific mediating tool.

41

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Table 2. How does location as context awareness influence the knowledge sharing social interaction? Researcher’s Intention

Different Learners’ Response / Experiences

Researcher’s Comments

From which locations is a learner willing to contact his/her peers for a learning task or problem?

…Home, library, hospital …University, basically anywhere on campus ...Only within university premises …Everywhere …Malls and shops …In the lab

Learning tasks and problems are not confined to fixed locations, thus learners could share knowledge about a problem from any location.

If a learner wanted to interact with a peer, how would s/he consider a peer’s present location?

.If they are in a lecture or the library and I have a small issue then I won’t bother them. In essence, I would just take into consideration my need to contact them and their availability in terms of what they’re doing and where they are. .If they were on campus or at the library or maybe at home, I’d feel free to contact them since they will probably be more flexible and relaxed to respond.

Learners use awareness of location to determine if the peer’s location favors a knowledge sharing social interaction.

How does a learner’s knowledge of where a peer is (their location) affect the knowledge sharing social interaction?

..Makes it easier because you would know who is closest to you for help ..It would be great to know if they are nearby, but if they’re not anywhere where I can reach them, then I’ll have to call regardless of where they are. …The closer he or she is, the better, because it will be easier for me to go to her when push comes to shove. …If I knew they were in a club or something, I wouldn’t bother them with school work.

The experiences show that awareness of a peer’s location would determine if it is plausible or would make sense to consult a peer.

Awareness of context and social presence is used as a tool that enables personalized sharing of experiences. An aspect of social awareness includes the location factor and context, which manifests as valuable to how the interaction is influenced. Culture and social background manifest as playing a fundamental role in learning contexts where learners often would rather seek help or support from peers who share a background. A common background allows better understanding and better communication and thus eases up learning and knowledge interactions. Awareness of context also manifests in how an interaction is influenced by a learner’s emotional and physical state, including behaviors and actions of others. Environmental situations also manifest as playing a role in a learner’s knowledge sharing decisions. Social presence of peers manifest as

42

fundamental with the role it plays in the sharing decisions. That is, awareness of a social presence enables an opportunistic learning and knowledge interaction and can also motivate a learner to engage with a learning task.

CONCLUSION This chapter did not directly address a specific learning management system (LMS) platform but addressed awareness and forms of communication that can be encountered as tools of LMS platforms. The chapter argued that to be able to design LMS that would efficiently enable social interaction among learners not confined to particular places, then there is a need to first understand how learners interact and the tools they use. In this chapter,

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Table 3. How does a peer’s activity and emotional state as context awareness influence knowledge sharing social interaction? Researcher’s Intention

Different Learners’ Response / Experiences

Researcher’s Comments

How would knowledge of what a peer is doing (current activities) affect a learner’s interaction decision?

…Issues like a person sleeping or eating may make me wait a while. ..I need to know if they got my full attention ..If sleeping, I will leave them be, if studying will interact with them …Would respect that they are really busy and seek help elsewhere

A peer’s activity influences the decision about whether to consult with that learner or consult with someone else.

How do actions and behaviors of others affect a learner’s decision to learn or share experiences?

..I am motivated by seeing others work and tend to slack when others aren’t working …When others are discussing work, it increases chances of grasping concepts. ...It doesn’t affect me much, because we all have different learning capabilities. . ..I take seriously any feedback or advice from friends who have done courses I’m doing. ..If I am studying and people are noisy, I will not be able to concentrate. If they are serious, I too will be.

Presence of others, that is, seeing what they are engaged with influences the decision of a learner to get involved and do the same. For some, what others are doing does not influence their decision to work

To what extent do emotional states of a peer affect a learner’s actions?

…Very little, unless they’re my friends and I am concerned, or if it’s group work and they’re slacking due to moods. …You know what you can ask and can’t, their moods may limit your questions. ...Lets you know how to approach them

Learners could be sensitive to the mental state of a peer, thus altering the way they consult for help or sharing experiences.

How does the awareness of a peer’s current emotional situation (e.g., s/he is stressed, happy, sad, cheerful, etc.) help in interacting with them?

…Very helpful, you don’t want to interact and learn with a sad guy, so they ought to be happy, cheerful. If anything they can be stressed. …It’s useful in the sense that I can know whether to bother them or not. It makes little difference in group work unless their situation is very serious. .It makes me understand their behavior and the way in which they react to my questions or experience. …It makes me understand them better and assist in determining what things I can or cannot say. …Will try to approach them correctly, depending on what kind of mood they are in?

The experiences of learners suggest that emotional states and activities of peers do influence a social interaction. The social awareness determines how learners approach their peers for a learning purpose. The emotional states could be regarded as signs and rules, thereby altering the context in which a knowledge sharing activity happens. Although these are unwritten signs and rules, learners know not to disturb or pester a peer who is not in the best of emotion and spirit to help with a learning task.

learners used social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Mobile phones, SMS, PDAs, email, instant messaging were in this chapter seen as tools and forms of LMS communication, without having to study a specific LMS platform. In order to understand learning management systems

and the degree of their role in learning, we must understand both what the communication tools are and how the social awareness can be used to facilitate the ubiquitous learning. An understanding of the context and a sense of social presence enables the learner to model behavior along the expectations or the shared understanding of a social learning community.

43

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Table 4. How do environmental situations as awareness of context influence knowledge sharing actions? Researcher’s Intention

Different Learners’ Response / Experiences

Researcher’s Comments

To what extent is a learner aware of his/her surroundings, during a learning activity, and how does that influence his/her decision to interact with peers?

..Temperature, I don’t like studying alone when it’s too cold so I normally chat a lot when it’s cold. ...The time of the day determines when I need to chat or where I need to work. ...Noise, too much movements, then I can’t concentrate on my work. …When I talk I can tell who is an understanding person from a lost one. …When I am with people I am not so aware of my surroundings in the sense that I am when I am alone, however, it is when its silent and cold that I tend to SMS/call people with my problems. ...If I see a lot of other students studying and discussing with their friends, they keep me motivated and I ask them questions.

The environmental context and situations do influence how learners act and how a learning action happens. The temperature during the day or ambience or noise levels may help a learner resort to a different knowledge action. For example, a learner saying that she prefers to do more social interaction than studying in isolation when the temperatures drop. A learner consults with peers, taking advantage of their social presence and surroundings.

Table 5. What is the role of Social Presence Awareness in the knowledge sharing environment? Researcher’s Intention

Different Learners’ Response / Experiences

Researcher’s Comments

How is it important for a learner to know a peer’s availability before he/she decides to interact with them?

Very important For me, it is quite important

Awareness of social presence or a peer’s availability is essential for knowledge interaction decisions

How does awareness of presence influence a learner’s knowledge action?

..It helps me determine who I can get assistance from ..Puts me at ease since I know my friends could help when I get stuck …I know I’m going to get help ..If they are available and willing to help, I will contact them. …I have a choice to pick any depending on who is more knowledgeable on that particular topic. …I know that when my academic questions arise I’ll be answered with certainty ...Then I could reach them whenever I need them ..It makes learning very easy, we could exchange ideas and information anytime and I will know that I am always able to ask for help

Social presence of knowledgeable peers is important for interaction whose purpose is sharing knowledge experiences. Learners are consciously at ease when they know their peers are available to help with a learning task, at anytime. Knowing a peer is socially present (awareness of social presence) gives a learner a sense of having a personalized academic support that “follows” them regardless of a learning problem and location.

How is a learner able to keep the sense of presence during a technology-mediated interaction, particularly when using textbased instant messaging.

..Make the messages as short as possible ..Instant or quick responses …I don’t want to wait for a response to my question, which is why I use IM.

Short, quick and instant responses grab a learner’s attention in a text-based social interaction.

How is a learner able to read emotions of peers during IM interaction and how can s/ he sense if the peer is interested in talking to them or not?

. ..Words, descriptive uninhibited words ... I know when they have something else in their minds . ..One word answers would indicate that he isn’t keen to talk. . ..The level of response and language used (enthusiasm)

Even though they are not in the same location, learners are able to exhibit social awareness levels during a mediated social interaction

44

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

The role of context awareness, and how it is experienced, is significant to ubiquitous social interaction whose outcome is personalized knowledge sharing. The context awareness in the learning management system environment manifests in three categories: 1) Mediating tools or applications as context of the sharing action learners use context awareness to decide on the appropriateness of the mediating tool. For example, if a learner is faced with a time-driven learning task she cannot resolve in isolation then she uses a PDA or a mobile phone to call or text message a peer as opposed to writing and sending an email to a peer. In this situation the learner is aware of the need to get instantaneous response, aware of how best to get the instant support, aware of how best to get social presence, as well as aware of the best mediating tool to exploit. 2) Location of a learner as context of the activity - learners need awareness of where they are themselves and where their peers are located, which is significant in determining the plausibility of a social interaction. 3) Learner’s situation –refers to context awareness dealing with culture and language, the learner’s current activity as well as environmental states. This chapter contributed towards an informed insight on how learners share their knowledge in a learning environment – focusing on LMS communication tools and their appropriateness. The chapter has shown that awareness of context and social presence (synonymous with social awareness) is a useful and significant characteristic for sharing learning experiences and the consequent learning support that occur independent of location and time. When learners socially interact through LMS mediation, they use context awareness to identify an appropriate tool and peer, in order to increase or decrease the knowledge sharing experience. Thus, in a learning environment, the social awareness is important since it influences the learning and knowledge decisions and actions, as the learning contexts change.

REFERENCES Alexander, P. A., Schallert, D. L., & Hare, V. C. (1991). Coming to terms: how researchers in learning and literacy talk about knowledge. Review of Educational Research, 61(3), 315–343. Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2). Argyle, M., & Dean, J. (1965). Eye-contact, distance and affiliation. Sociometry, 28, 289–304. doi:10.2307/2786027 Beyer, H., & Holtzblatt, K. (1998). Contextual design, defining vustomer-centred systems. MA, USA: Morgan Kaufmann Publishers, Inc. Bredo, E. (1994). Cognitivism, situated cognition and Deweyian pragmatism. Philosophy of Education. Retrieved June, 2010, Online at http:// www.ed.uiuc.edu/EPS/PES-Yearbook/94_docs/ BREDO.HTM] Byam, N. (1995). The emergence of community in computer-mediated communication. In Jones, S. G. (Ed.), Cybersociety. Newbury Park, CA: Sage. Davenport, T. H., & Prusak, L. (1998). Working knowledge: How organizations manage what they know. Boston, MA: Harvard Business Review Press. Dey, A. K., & Abowd, G. D. (1999). Towards a better understanding of context and context-awareness. GVU Technical Report GIT-GVU-99-22. College of Computing, Georgia Institute of Technology, Atlanta, Georgia. Goffman, E. (1978). Behavior in public places. Notes on the social organization of gatherings (4th ed.). New York, NY: Free Press.

45

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Gunawardena, C. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2/3), 147–166. Gunawardena, C., & Zittle, F. (1997). Social presence as a predictor of satisfaction within a computer mediated conferencing environment. American Journal of Distance Education, 11(3), 8–26. doi:10.1080/08923649709526970 Holtzblatt, K., & Jones, S. (1994). Contextual inquiry: A participatory design technique for system design. In Schuler, D., & Namioka, A. (Eds.), Participatory design: Principles and practice. Englewood Cliffs, NJ: Prentice-Hall. Jaffer, S., Ng’ambi, D., & Czerniewicz, L. (2006). The role of ICTs in higher education in South Africa: One strategy for addressing teaching and learning challenges. Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa, 10-21 July. Retrieved from http:// emerge2006.net. Kekwaletswe, R. M. (2006). Social presence and context awareness for knowledge transformation in an m-learning environment. In Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa (10-21 July). Retrieved from http://emerge2006.net. Kekwaletswe, R. M. (2007). Knowledge transformation in a mobile learning environment: An inquiry of context and social presence awareness. PhD thesis. South Africa: University of Cape Town. Kekwaletswe, R. M. (2009). Conceptualizing ubiquitous learning through context-aware wired and wireless Web services. In Proceedings of the IADIS Mobile Learning conference, Barcelona, Spain (25 – 28 February).

46

Kekwaletswe, R. M., & Ng’ambi, D. (2006). Ubiquitous social presence: Context-awareness in a mobile learning environment. In Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing, Taichung, Taiwan (90-95). Taichung, Taiwan. Laudon, K. C., & Laudon, J. P. (1995). Information systems: A problem-solving approach (3rd ed.). Orlando, FL: Dryden Press. Lonsdale, P., Baber, C., Sharples, M., & Arvanitis, T. N. (2003). A context awareness architecture for facilitating mobile learning. In Proceedings of MLEARN 2003 (LSDA), London. Ng’ambi, D. (2004). Towards a knowledge sharing framework based on student questions: The case for a dynamic FAQ environment. PhD thesis. South Africa: University of Cape Town. Ng’ambi, D., & Hardman, J. C. (2004). Towards a knowledge-sharing scaffolding environment based on learners’ questions. British Journal of Educational Technology, 35(2), 187–196. doi:10.1111/j.0007-1013.2004.00380.x Ngwenyama, O. K., & Lee, A. (1997). Communication richness in electronic mail: Critical social theory and the contextuality of meaning. Management Information Systems Quarterly, 21(2), 145–167. doi:10.2307/249417 Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14–37. doi:10.1287/orsc.5.1.14 Patriotta, G. (2003). Organizational knowledge in the making: How firms create, use, and institutionalize knowledge. Oxford, UK: Oxford University Press. Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence and performance in an online course. Journal of Asynchronous Learning Networks, 6(1).

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

Polanyi, M. (1958). Personal Knowledge. Chicago, IL: The University of Chicago press. Polanyi, M. (1966). The tacit dimension. London, UK: Routledge & Kegan Paul. Rakotonirainy, A., Loke, S. W., & Fitzpatrick, G. (2000). Context awareness for the mobile environment. Retrieved Oct 11, 2009 from ftp:// ftp.cc.gatech.edu/pub/gvu/tr/2000/00-18r.ps.Z. Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1), 68–88. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2). Ryder, M., & Wilson, B. (1997). From center to periphery: Shifting agency in a complex technical learning environments. Paper presented at the Meeting of the American Educational Research Association, Chicago, IL. Schmidt, A., Beigl, M., & Gellersen, H. W. (1999). There is more to context than location. Computers & Graphics, 23, 893–901. doi:10.1016/S00978493(99)00120-X Scribner, S. (1987). Thinking in action: Some characteristics of practical thought. In Sternberg, R., & Wagner, R. (Eds.), Practical intelligence: Nature and origins of competence in everyday world. Cambridge, UK: Cambridge University Press. Shariq, Z. S. (1999). How does knowledge transform as it is transferred? Speculations on the possibility of a cognitive theory of knowledgescapes. Journal of Knowledge Management, 3(4), 243–251. doi:10.1108/13673279910303998

Shea, P. J., Pickett, A. M., & Pelz, W. E. (2003). A follow-up investigation of “teaching presence” in the SUNY Learning Network. Journal of Asynchronous Learning Networks, 7(2), 61–80. Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. Toronto, Canada: Wiley. Sutherland, R., Facer, K., Furlong, R., & Furlong, J. (2000). A new environment for education? The computer in the home. Computers & Education, 34(3-4), 195–212. doi:10.1016/ S0360-1315(99)00045-7 Swan, K. (2002). Building communities in online courses: The importance of interaction. Education Communication and Information, 2(1), 23–49. doi:10.1080/1463631022000005016 Swan, K. (2003). Developing social presence in online discussions. In Naidu, S. (Ed.), Learning and teaching with technology: Principles and practices (pp. 147–164). London, UK: Kogan Page. Van Beveren, J. (2002). A model of knowledge acquisition that refocuses knowledge management. Journal of Knowledge Management, 6(1), 18–22. doi:10.1108/13673270210417655 Walther, J. (1994). Interpersonal effects in computer mediated interaction. Communication Research, 21(4), 460–487. doi:10.1177/009365094021004002 Wiener, M., & Mehrabian, A. (1968). Language within language: Immediacy, a channel in verbal communication. New York, NY: AppletoncenturyCroft. Winograd, T., & Flores, F. (1986). Understanding computers and cognition: A new foundation for design. Norwood, NJ: Ablex.

47

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

ADDITIONAL READING Bush, M. D., & Mott, J. D. (2009). The transformation of learning with technology: Learner-centricity, content and tool malleability, and network effects. Educational Technology, 49(2), 3–20. Delta Initiative (2009). The state of learning management in higher education systems, report for the California State University System. Dourish, P., & Bellotti, V. (1992). Awareness and coordination in shared workspaces. In Proceedings of ACM CSCW Conference (pp. 107-114). Dourish, P., & Bly, S. (1992). Portholes: Supporting awareness in a distributed work group. Proceedings of the CHI ‘92 Conference on Human Factors in Computing Systems (pp. 541-547). Engeström, Y. (1987). Learning by expanding: An activity theoretical approach to developmental research. Helsinki, Sweden: Orienta-Konsultit. Engeström, Y. (1999). Activity theory and individual and social transformation. In Engeström, Y., Miettinen, R., & Punamaki, R. (Eds.), Perspectives on activity theory. Cambridge, UK: Cambridge University Press. Gardner, C. (2009). A personal cyberinfrastructure. EDUCAUSE Review, 44(5), 58–59. Gobbin, R. (1998). Adoption or rejection: Information systems and their cultural fitness. In H. Hasan, E. Gould, and P. Hyland (Eds.), Information systems and activity theory: Tools in context (pp. 109-124). University of Wollongong Press, Wollongong, N.S.W. Gould, S. J. (1987). Time’s arrow: Time’s cycle: Myth and metaphor in the discovery of geographical time. Cambridge, MA: Harvard University Press.

Gutwin, C., Greenberg, S., & Roseman, M. (1996). Workspace awareness in real-time distributed groupware: Framework, widgets and evaluation. In Proceedings of HCI on People and Computers XI (pp. 281-298). Jonassen, D. H., & Rohrer-Murphy, L. (1999). Activity theory as a framework for designing constructivist learning environments. Educational Technology Research and Development, 47(1), 62–79. doi:10.1007/BF02299477 Lane, L. M. (2009). Insidious Pedagogy: How course management systems impact pedagogy. First Monday, 14(10). Mlitwa, N., & Van Belle, J. P. (2010). A proposed interpretivist framework to research the adoption of learning management systems in universities. Communications of the IBIMA (Vol. 2010). Retrieved from http://www.ibimapublishing.com/ journals/CIBIMA/cibima.html. Morgan, G. (2003). Faculty use of course management systems (Vol. 2). ECAR Research Bulletin. Sclater, N. (2008). Web 2.0, personal learning environments, and the future of learning management systems,” ECAR Research Bulletin (vol. 2008, no. 13). Suchman, L. (1987). Plans and situated actions: The problem of human–machine communication. Cambridge, UK: Cambridge University Press. Taylor, J. (2006). Evaluating mobile learning: What are appropriate methods for evaluating learning in mobile environments? In M. Sharples (ed.), Big issues in mobile learning: Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative: University of Nottingham, UK. Retrieved November 29, 2009, from http://telearn.noe-kaleidoscope.org/ warehouse/Sharples-2006.pdf. Tollmar, K., Sandor, O., & Shömer, A. (1996). Supporting social awareness at work, design and experience. In [New York, NY: ACM Press]. Proceedings of CSCW, 96, 298–307.

48

Knowledge Sharing in a Learning Management System Environment Using Social Awareness

KEY TERMS AND DEFINITIONS Awareness: An understanding of the activities of others, which provides a context for your own activity. Context: Understood as the situation in which a learner or a group of learners find themselves. Contextual Inquiry: A field research framework that depends on conversations with users in the context of their work. Knowledge Sharing: Involves two actions; transmission (sending or presenting knowledge to a potential learner) and absorption by the au-

dience. If knowledge is not absorbed, it has not been shared. Social Awareness: Is synonymous with context and social presence awareness. Social Presence: Defined and understood to be the mediated presence of another learner who could provide personalized on-demand social support for a learning problem as the learner traverses varied learning contexts. Ubiquitous Personalized Support: Refers to the provision of context sensitive and anywhere, anytime support as a learner traverses varied locations.

49

50

Chapter 3

Learning 2.0:

Using Web 2.0 Technologies for Learning in an Engineering Course Thomas Connolly University of the West of Scotland, UK Carole Gould University of the West of Scotland, UK Gavin Baxter University of the West of Scotland, UK Tom Hainey University of the West of Scotland, UK

ABSTRACT Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web 2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web 2.0 technologies provided by the chosen LMS. DOI: 10.4018/978-1-60960-884-2.ch003

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Learning 2.0

INTRODUCTION There has been considerable research into the perceived benefits of eLearning and Learning Management Systems (LMS) within education and it is clear that LMS now play a pivotal role in the delivery of eLearning within many educational institutions. The research literature cites many advantages of eLearning, particularly the convenience and flexibility offered by the (asynchronous) ‘anytime, anywhere, anyplace’ education. However, much less research has been carried out into the use of educational technologies and tools within vocational training environments. The purpose of this chapter is to discuss the impact on learning with the introduction of a LMS into a vocational engineering course. This chapter discusses the pilot implementation of Web 2.0 tools within an LMS and aims to answer a number of questions: (i) Can technology supplement the hands-on nature of vocational training? (ii) Can the use of wikis and forums aid vocational training? (iii) Can the pilot be considered a success? Much of the research in this area has been mainly anecdotal and has not considered the different nature of vocational training with most of the research focusing on the traditional educational environment. This chapter utilises both qualitative and quantitative surveys on the views of trainees and instructors and aims to identify the areas within the training programme where the LMS could be utilised further to aid learning. It also considers the areas where the use of the LMS has not been as successful as anticipated and the reasons for this. The next section of this chapter discusses the literature on LMS, the use of Web 2.0 technologies within education, and ePortfolios. The subsequent sections introduce the research rationale, the case study and an empirical analysis of the pilot implementation. The chapter concludes

with a discussion of the findings and provides some recommendations for the implementation of eLearning within vocational training.

PREVIOUS RESEARCH eLearning can be defined as “… any use of Web and Internet technologies to create learning experiences” (Horton, 2003, pp. 13). eLearning is essentially an evolved form of distance education, which Connolly and Stansfield (2007a) describe through a six-generation model, as depicted in Figure 1. The first generation (the ‘correspondence model’) was provided mostly through paper-based instruction, characterized by the mass production of educational materials. The difficulty with correspondence education has been the infrequent and inefficient form of communication between the instructor and the learners. Further, it was difficult to arrange for peer interaction in correspondence based distance education. The second generation (the ‘multimedia model’) was provided through integrated multimedia such as delivering courses via television or introducing material like audio and video tapes, computer-based learning (CBL) in addition to printed material. The third generation was provided through two-way communications media such as audio/video-conferencing and broadcast technology. The fourth generation of distance education (the first generation of eLearning) is defined as mainly passive use of the Internet, consisting primarily of conversion of course material to an online format, low-fidelity streamed audio/video, and basic mentoring using email. However, the educational philosophy still belongs to the pre-Internet era. The fifth generation of distance education (the second generation of eLearning) uses more advanced technologies consisting of high-bandwidth access, rich streaming media, online assessment (eAssessment) and LMS that provide access to course material, communication facilities, and learner services. The sixth generation of distance education (the third

51

Learning 2.0

generation of eLearning) is a more collaborative learning environment based much more on the constructivist epistemology, promoting reflective practice through tools like ePortfolios, Web 2.0 technologies such as blogs and wikis, online communities, and using interactive technologies such as online visualizations, games, and simulations.

We are also now starting to see the development of mLearning (mobile learning) through devices like PDAs (Personal Digital Assistants), mobile phones and smartphones, and tablet devices. For second and third generation eLearning, LMS support new approaches for people to learn and assist with the delivery but also with the way

Figure 1. Models of distance education (adapted from Connolly & Stansfield, 2007a)

52

Learning 2.0

in which information is presented leading to acquisition of new knowledge (Holmes & Gardner, 2006). eLearning and the use of LMS are now an integral part of most educational institutions with educational technologies witnessing exceptional levels of growth in recent years. To support the growth of LMS, many schools, colleges and universities have invested heavily in up-to-date technology. Also referred to as Virtual Learning Environments (VLEs), ‘learning platforms’, ‘distributed learning systems’, ‘course management systems’ and ‘instructional management systems’, LMS combine a range of course/subject management and pedagogical tools to provide a means of designing, building and delivering online learning. LMS are scalable systems that can be

used to support an institution’s entire set of teaching and learning courses. Although LMS are used extensively within educational institutions globally, their use is relatively low within vocational training companies. One reason for this may be that vocational trainers do not appreciate that some aspects of vocational training may lend itself well to online delivery. There are many different LMS products available, some at considerable cost and others available as open source. Regardless of which product is chosen, most LMS contain similar functionalities as shown in Table 1. As with most things, eLearning has advantages and disadvantages. The research literature cites many advantages of eLearning, particularly the convenience and flexibility offered by the

Table 1. Comparison of LMS functionality Features/Tools

LMS Blackboard ProSites

Moodle

Learnwise

Frog

E-portfolio

N

Y

Y

Y

File up-load

Y

Y

Y

Y

Notice/bulletin board

Y

Y

Y

Y

Course outlines

Y

Y

Y

Y

Assignments

Y

Y

Y

Y

Assessments

Y

Y

Y

Y

Multi-media resources

Y

Y

Optional extra

Y

Evidence gathering

Y

Y

Y

Y

Calendar

Y

Y

Y

Y

Administration tools

Y

Y

Y

Synchronous collaboration tools (video conferencing)

Y

Y

Optional extra

N

Forum/discussion board

Y

Y

Y

Y

Email

Y (Internal)

Y (Internal)

Y (Internal)

Y (Internal)

External links

Y

Y

Y

Y

Student home page

Y

Y

Y

Y

Real-time chat

N

Y

Y

Y

Quiz design

Y

Y

Y

Y

Costs

£6,655 per annum (200 users/licences) £3,152 per annum (100 additional users/ licences

Open source Additional costs if hosting required

Ranges from £3,300 per annum for 1,000 users to £27,500 per annum for 50,000 users

Bespoke system - £26,000 Yearly support - £4,500 Standard package - £22,500 Yearly support - £4,500

53

Learning 2.0

(asynchronous) ‘anytime, anywhere, anyplace’ education (McDonald, 2002), which gives learners time for research, internal reflection, and ‘collective thinking’ (Garrison, 1997). Moreover, the text-based nature of eLearning normally requires written communication from the learner, which along with reflection, encourage higher level learning such as analysis, synthesis, and evaluation, and encourage clearer and more precise thinking (Jonassen, 1996). In addition, eLearning courses also have the capability to present multiple representations of a concept, which allows learners to store and retrieve information more effectively (Kozma, 1987). It is also argued that increased social distance provides a number of distinct advantages to online conferences (synchronous or asynchronous). In written communications anonymity of characteristics such as gender, race, age, or social status can be preserved, which can reduce the feeling of discrimination and provide equality of social interaction among participants. In turn, this can permit the expression of emotion and promote discussion that normally would be inhibited (Gunawardena, 1993). eLearning is not without its disadvantages; for example (Connolly & Stansfield, 2007b): • •

• • •

•

54

Costs may initially exceed more traditional methods; More responsibility is placed on the learner who has to be self-disciplined and motivated; Some learners lack access to a PC/Internet or have difficulty with the technology; Increased workload for both students and faculty; Non-involvement in the virtual community may lead to feelings of loneliness, low self-esteem, isolation, and low motivation to learn, which in turn can lead to low achievement and dropout; Dropout rates tend to be higher in eLearning courses than in traditional face-to-face courses, often 10 to 20 percentage points higher.

Perhaps one of the most damaging criticisms is that some eLearning simply replicates the social organization of traditional education and training and that the potential benefits of eLearning - of personalized and accessible learning experiences - are missed. Taking this on board, this is one of the reasons this research is of particular importance. There is a high chance that the traditional nature of the organisation under investigation may simply deploy an LMS but not utilize it to its full potential.

Learner Expectations and Web 2.0 Web 2.0 is a term often associated with “… the social use of the Web which allow[s] people to collaborate, to get actively involved in creating content, to generate knowledge and to share information online” (Grosseck, 2009, pp. 478). Web 2.0 applications, such as blogs and wikis, are being introduced into LMS such as Blackboard and Moodle, providing students with increased flexibility in terms of how they communicate with fellow students and gain feedback from peers. In industry the use of Web 2.0 applications are being gradually introduced as organisations have begun to realise their potential for the purposes of learning and information sharing. The way in which education and training institutions and industrial organisations facilitate learning and information sharing is being determined by the expectations and prior learning experiences of the individuals within them. Within most learning or working environments today, there is a sense of expectancy that innovative learning and communication channels should already be in place to accommodate the diverse ways in which individuals learn. The aspects of institutional and organisational competitiveness that are closely related to the concept of the ‘knowledge economy’ means that institutions promoting learning via technological means now have an interest in developing and running learning initiatives with the minimum of effort (DeRouin, Fritzsche, & Salas, 2005). These particular factors are having an impact

Learning 2.0

upon the format and delivery of eLearning initiatives within educational and industrial contexts. Web 2.0 technologies are attractive within these contexts as they allow greater student independence and autonomy, increased collaboration and an increase in pedagogic efficiency (Franklin & Harmelem, 2007, pp. 1). For this reason, the use of Web 2.0 technologies could be an invaluable asset to those institutions delivering ‘on-the-job’ training, especially where an element of group work is undertaken. It also facilitates knowledge sharing and promotes learning in an industrial context, as well as taking account of pedagogical issues. This was important to our study given that one part of the vocational training to be delivered was a group-based project.

Social Software and Learning 2.0 It could be argued that the concept of eLearning is being enhanced by the rapid development of ‘social software’. McKelvie, Dotsika, and Patrick (2007) state that “social software is a community driven technology which facilitates interaction and collaboration and depends largely on social convention.” Though social software can be used on an individual basis it is predominately concerned with the notions of open and collective communication, dialogue and the ability to liaise with individuals collectively. Social software is having an effect upon eLearning delivery within education and industry as it has altered the way in which learning is taught and learnt. The use of social software allows the learner to generate knowledge and share their learning experiences on a collective level as well as allowing users to openly reflect upon what they have learnt. eLearning distinguishes itself from social software as it is predominately associated with electronic instruction and is better suited for education and training purposes. Web 2.0 is transforming the way in which people learn as the learning is predominately social and self-directed in nature whereas eLearning is normally associated with

individual learning. The use of social software and Web 2.0 technologies have given rise to the term ‘Learning 2.0’, sometimes referred to as eLearning 2.0, which broadly summarizes all opportunities arising from the use of social media for learning, education or training.

The Pedagogy of Learning 2.0 The interactive and collaborative nature of social software makes it highly suited towards sustaining and facilitating what are known as communities of practice (CoPs) or “groups of people informally bound together by shared expertise and passion for a joint enterprise” (Wenger & Snyder, 2000, pp. 139). In conjunction with the concept of CoPs, the learning theory of social constructivism appears to complement and accommodate the principles surrounding the use and learning benefits associated with Learning 2.0. The constructivist view of learning adopts the stance that learners do not learn individually from one another and stresses the relevance regarding the socio-cultural context of learning. Predominately, social constructivism contends that knowledge is formulated through the processes of social interaction and collaborative learning. Though the concept of social software can, in theory, support a wide range of learning approaches, it is inherently applicable towards the learning theories of social constructivism and CoPs. It has been generally regarded that one of the salient aspects of any CoP is its ability to construct and store collective knowledge in what has been referred to as a ‘shared repertoire of communal resources’ (Wenger, 2000). Additionally, CoPs are most usually distinctly defined by the concepts of collective understanding, mutual engagement and shared repertoire (Wenger, 2000). It could be argued that for this reason, the use of Learning 2.0 could be an invaluable asset to organisations. The examples given below demonstrate the growth of those technologies and tools that facilitate knowledge sharing in the wider community.

55

Learning 2.0

ePortfolios As well as the Learning 2.0 technologies, many educational institutions and organisations are utilising ePortfolios, the electronic version of the paper portfolio. As with LMS, the use of ePortfolios is also growing in popularity, particularly within vocational training. A portfolio can be defined as “a collection of documents relating to a learner’s progress, development, and achievements”; an ePortfolio simply indicates that some or all of the evidence is collected in digital form (Beetham, 2009). EDUCAUSE (2005) defines an ePortfolio as “a valuable online tool that learners, faculty, and institutions can use to collect, store, update, and share information. E-Portfolios allow students to reflect on their learning, communication with instructors, document credentials, and provide potential employers with examples of their work”. ePortfolios are well suited to education, vocational training and working environments as they capture the concept of lifelong learning and support individuals as they progress through school, college, Higher Education, training and employment (Richardson & Ward, 2005). Learners gather learning evidence and define these evidences through a self-reflection process. They attribute their competences to learning products or outcomes and reflect on how they acquired those competences. From a pedagogical perspective this process helps learners to better understand how they learn and helps them to become self-directed learners (Berlanga et al., 2008). ePortfolios can be classified into various types – assessment, presentation, learning, personal development and shared ownership (Curyer et al., 2007) but in reality most are a combination. If a standard approach was adopted for ePortfolios, institutions and organisations could share and exchange ePortfolio data, which could lead to the streamlining of the processes connected to prior learning, with student transitions through courses, and with training that involves either sequential or parallel movement through multiple institutions

56

and companies (Curyer et al., 2007). This could also help to fulfil the concept of an ePortfolio being utilised throughout lifelong learning. Richardson and Ward (2005) carried out an in-depth study of 12 different ePortfolios and found, amongst other things, that no two systems were identical or offered the same range of functions. However, it should be considered that an ePortfolio tends to be chosen on a ‘fit-for-purpose’ basis and often vendors may customise their ePortfolios to suit a particular customer, as is the experience of the authors. Most ePortfolios are driven by the learner; that is, the learner is responsible for the maintenance of the ePortfolio and decides who has access to its contents but in some environments, as in the study under investigation, this may not be desirable, as some aspects of vocational training need to be driven by the instructor, not the learner. Internal examiners and external verifiers often need access to trainee assessment material and that assessment material cannot be amended once verified. In our study, the instructors required a measure of customisation of the ePortfolio, which included allowing multiple assessors and the instructors themselves having overall control of the ePortfolio.

ePortfolio Use in Higher/ Further Education According to JISC (2008) ePortfolios represent the latest in a line of technology-based innovations that are becoming an integral part of the learning landscape in Higher and Further Education. Student ePortfolios developed out of faculty-assigned, print-based student portfolios as far back as the mid-80s. They were typically found in art-related disciplines or those that consisted of substantial written components, such as English studies, and gained greater importance in education during the mid-90s. Student ePortfolios are commonly found in college programmes where teachers use them to provide evidence of competence. This includes

Learning 2.0

communications, maths, business, nursing and engineering to record students’ learning experiences and skill set (Lorenzo & Itellson, 2005). The Open University in the UK has been using ePortfolios as an assessment tool in online courses for many years (Mason, Pegler, & Weller, 2004) and the University of Washington developed an ePortfolio in 2001 to allow students to record their entire educational learning experiences in an organised and integrated manner. A survey carried out by Strivens (2007) found that of those who participated in the survey, 20 institutions (54% of those with an ePortfolio) commented that the ePortfolio was available to all students across the institution. The increase in the use of ePortfolios in Higher Education is further supported by Beetham (2009) who states that work is currently being carried out to integrate LMS, student record systems and ePortfolio tools to provide formative feedback. It appears that educational institutions have embraced the concept of ePortfolios and appreciate their value.

ePortfolio Use in Vocational Training Portfolios (paper-based) have been utilised for many years in vocational training; for example, Austria has been using portfolios in teacher training for the past 12 years and covers topics such as supervision and professional upgrade in vocational education and is regarded as a working portfolio, as examination for teachers is impractical (Dorninger & Schrack, 2007). ePortfolios have been endorsed by some of the major vocational examiners such as City & Guilds in the UK. City & Guilds (2009) undertook a survey of 95 colleges and training providers and found that, although cost savings were significant, the main advantage was the reduction in time taken for candidates to complete their qualification when using an ePortfolio. However, although feedback was positive, only 16% of those surveyed used ePortfolios. Some centres expressed fear over technical glitches, which could result in lost work,

and some resistance to change was also noted. The literature demonstrates that ePortfolios offer a valid way to assess and ensure completion of an individual’s training, and while usage is not pervasive, uptake is growing.

METHODOLOGY The aim of this study was to determine whether eLearning could be used successfully within a vocational training environment for engineering students. More specifically, the objectives of the project were (i) to select suitable LMS and ePortfolio systems, (ii) develop some of the provision in an online format, and (iii) evaluate the success of this provision. This study started in late 2008 and the initial part of the study was completed in May 2010. Empirical data was obtained from a cohort of students going through their Induction programme, which was the first course developed in an online format, and from their final group project before completing their training. In addition, the instructors were surveyed in order to ascertain their views of the LMS and the use of the wiki and forum for the group project. The instructors’ survey also included questions relating to the use of the LMS as a learning tool. For this study, the research philosophy of interpretivism was adopted. Interpretivism is an epistemology that advocates that the researcher must understand the differences between humans in their social roles as actors. This philosophy places emphasis on conducting research among people as opposed to objects, such as cars and houses. Interpretivism stems from the intellectual tradition of phenomenology, which is concerned with the ways in which humans make sense of the world around them (Saunders, Lewis & Thornhill, 2007). According to Whisker (2001), phenomenology encourages both quantitative and qualitative methods. However, it should be considered that quantitative research is more often associated with positivism. McBride and Schostak (1994) state:

57

Learning 2.0

“Where quantitative forms of research, employing questionnaires and sampling procedures attempt to eradicate the individual, the particular and the subjective, qualitative research gives special attention to the subjective side of life...qualitative researchers are more likely to ask how it feels... they focus upon the social construction of such things...”. As such, this study is therefore adopting a mixed methodology with an Action Research and within case study approach as the authors are actively involved in the project. Gerring (2007, pp. 20) defines case study as “the intensive study of a single case where the purpose of that studyat least in part- to shed light on a larger class of cases”. It is hoped that at the termination of the project it will be possible to develop a framework that will aid practitioners in the successful implementation of an LMS in vocational training, as well as promote a rich area for future research. The next section will outline the company under investigation, the type of training delivered and how the chosen technologies were selected.

Case Study: Vocational Training This project came about when University of the West of Scotland entered into a two-year partnership programme with a local training organisation, East Kilbride & District Engineering Group Training Association (EKGTA), to explore how much of a Modern Apprenticeship programme could be delivered online. EKGTA is an employer-led training provider for the engineering industry with charitable status. Established in 1966 and recognised as one of the premier training groups throughout Scotland, it aims to serve the needs of the employer, whilst ensuring candidates have the opportunity to develop the knowledge and skills necessary in employment. The Association specialises in training Modern Apprentices at craft and technician levels, and in basic engineering skills training to national standards. EKGTA provides training in other disciplines, such as, Health & Safety, Computer Based Training, Professional

58

Development but apprentice engineering training is EKGTA’s core business and for this reason the main focus is directed towards that training. A Modern Apprenticeship Programme may involve: • • • • •

A period of training in an approved training centre (off the job). Completion of a Level 2 Vocational Qualification. Attainment of core skills to intermediate one level (minimum). Completion of a National Certificate (day release at FE College). Completion of a Level 3 Vocational Qualification in company.

Level 2 competence involves the application of knowledge and skills in a significant range of varied work activities, performed in a variety of contexts. Some of the activities are complex or non-routine, and there is some individual responsibility and autonomy. Collaboration with others may often be a requirement On the other hand, Level 3 competence involves the application of knowledge and skills in a broad range of varied work activities performed in a wide variety of contexts, most of which are complex and nonroutine. There is considerable responsibility and autonomy, and control or guidance of others is often required. There are six instructors who deliver the practical element of the qualification with the academic element delivered onsite by a local Further Education college. Once the trainees return to their company, advisors visit trainees on average every 12 weeks and oversee the continuation of the training. Management appreciate the advancements in technology that could potentially support and enhance the training programmes offered by the Association, however, as the project has progressed, it has become obvious that this sentiment is not echoed by all staff. Before the project, technology within EKGTA was used primarily for the record-

Learning 2.0

ing of completed assignments (ePortfolio) and the storing of lecture material, and was therefore used far below its potential. In the early stages of the project the pilot of the LMS was to be rolled out within the electrical area of the workshop, but on further discussion it was decided that this would not be an inclusive approach. The decision was taken to implement the LMS throughout the entire organisation in two stages: the Induction programme and the group project, thus including all instructors.

Selecting the LMS In discussion with EKGTA, a number of features were identified that the chosen LMS had to support. Existing students were also consulted. Two informal group forums were set up with ten participants in each. This helped to identify those features that students believed would enhance learning. Almost all students agreed that 24x7 access to learning materials would be advantageous, more use of multimedia, such as video demonstrations and interactive quizzes, with immediate feedback to reinforce learning. To identify a suitable LMS, it was first necessary to carry out desk research on a number of different LMS to identify the functionalities that each one supported and to match these against the company’s and the students’ criteria. A short list of potential systems was drawn up that included Blackboard ProSites, Learnwise, Frog and Moodle, Moodle being the only open source option. The outcome of the research demonstrated that there was very little difference in the functionalities of each, as shown in Table 1. The ability to automatically map student evidence of their work to individual components of assessment was regarded by the instructors as the most important function that needed to be supported by the ePortfolio component as this was their most time intensive activity, however, after much research it became apparent that this function was not readily available with any of the

shortlisted LMS systems and it would be necessary to select a separate ePortfolio system to the LMS, although it was hoped the two could be integrated together in some way. After extensive analysis and consultation with management and staff, Moodle was chosen as the LMS as it was highly modular and provided the same features as the commercial systems but at no cost to the company. The other LMS investigated were more suited to educational institutions with large numbers of students. For this project, the maximum student uptake in one year would be limited to 120 and so the need for a large commercial product could not be justified. In addition, Moodle has a large and growing community with almost 40 million users and over 50,000 sites worldwide at the time of writing (Moodle, 2010). The next stage of the project was to identify a suitable ePortfolio system.

Selecting the ePortfolio Again, a short list of potential ePortfolio systems was drawn up based on instructors’ requirements, which included Learning Assistant, One File, Pebble Pad and Mahara, which is open source and an add-on module for Moodle. EKGTA previously used two ePortfolios, one at Level 2 and one at Level 3, which were not integrated in any way. The ePortfolio at Level 2 gives on-campus students access to all lecture material, standard assessment documentation, and in some instances multimedia. At Level 3 advisors use the Modern Apprenticeship online ePortfolio system. At this level, students are off-campus, therefore regular remote communication between advisors and students needs to be undertaken. The Modern Apprenticeship online ePortfolio did not fulfil some of the criteria required by the instructors and was no longer being fully supported and so was excluded from further consideration. After in-house evaluation, it was concluded that Learning Assistant best suited the requirements. Learning Assistant is marketed as an ePortfolio and eAssessment solution for training

59

Learning 2.0

centres that deliver vocational qualifications. It has been designed specifically to meet the needs of the vocational training environment. Learning Assistant, as well as being fully integrated from Level 1 through to Level 4, is a fully supported system with regular updates to meet the changing needs of the students and any changes to the engineering qualification. In addition, the Learning Assistant vendor could customise the system to support multiple assessors and facilitate the automatic mapping of student evidence of their work to individual components of assessment.

Identifying Priority Features After researching a variety of LMS and ePortfolio systems to identify their generic features and the system(s) that is more suited to the vocational training environment, it was then necessary to survey the six workshop instructors and one training advisor to identify those features that would be regarded as a priority for implementation. This gave an indication of where the end-users see technology enhancing learning and to help in the delivery of training. Table 2 provides details of this survey. The lower the number assigned, the higher importance assigned. The data demonstrates that the top five priority features are: •

• • • •

ability to automatically map student evidence of their work to individual components of assessment; a suitable ePortfolio system; evidence gathering; uploading of assignments; a file upload capability.

These features were subsequently prioritised for implementation.

Implementation In the early stages of the project the pilot of Moodle was to be rolled out within the electrical area of the workshop, but on further discussion 60

it was decided that this would not be an inclusive approach. The decision was taken to implement Moodle throughout the entire workshop, thus adopting an inclusive approach. All trainees go through a week long induction programme - a general induction and a workshop induction. A course was developed and populated with two units: general induction and workshop induction. All lecture material and presentations were uploaded to Moodle and self-assessment quizzes designed to reinforce learning. Learning Assistant was customised by the vendor and installed for students to use and a single sign-on was developed between Moodle and Learning Assistant so students only had to log in once to access both systems. Figure 2 shows an example page in the Moodle system and Figure 3shows the reporting system within the ePortfolio.

FINDINGS AND DISCUSSION In this section, we discuss the findings from the first year of using the LMS for training the engineering students. We start with an evaluation of the results from the Induction programme followed by the group project.

Induction Programme Results A five-day Induction programme was run for a cohort of students in Autumn 2009. Demographic information was collected at the start of Induction and an evaluation questionnaire was distributed at the end.

Demographic Questionnaire 26 participants completed the demographic questionnaire. 25 (96%) of the respondents were male and 1 (4%) of respondents were female. The mean age of participants was 17 (SD = 0.748) with a range from 16 to 19. 17 (65%) of participants started an apprenticeship at EKGTA when leaving school, 9 (35%) of participants did not. The

Learning 2.0

Table 2. Survey results for priority features ePortfolio

Student home page

Course outlines

Assignments

Map evidence to assessment

Multimedia

1

6

4

7

2

2

4

13

16

3

2

14

1

13

8

6

2

10

6

16

4

5

3

1

5

12

13

8

1

7

1

12

3

5

2

4

4

10

5

1

2

8

22

82

53

35

14

46

File up-load

Evidence gathering

Notice board

External links

Quiz design

Forum

3

3

5

8

14

15

7

1

12

11

15

8

9

3

7

12

11

14

2

7

12

8

13

14

4

2

14

11

16

9

6

11

14

7

8

13

7

3

9

12

13

11

38

30

73

69

90

84

Real time chat

Calendar

Email

Admin tools

16

2

6

3

9

10

5

6

15

16

4

5

15

11

10

9

15

10

3

6

15

9

16

10

16

14

15

6

101

72

59

45

majority of the participants (18, 69%) left school in 2009, 6 (23%) left school in 2008 and 2 (8%) left in 2007. The majority of the participants (23, 88%) indicated that they had no other full-time employment in the past while 3 (12%) did. 1 participant indicated that they worked in the hospitality sector. 19 (73%) of participants indicated that they had not attended college prior to attending the training course, whereas 6 (23%) of participants indicated that they had attended college first. The courses

that were undertaken were primarily in the areas of construction (joinery, roofing, brick-laying, etc) and creative industries (art, fashion design, media, etc). These courses were undertaken at intermediate, NC and HNC level. Table 3 shows the standard grade qualifications and Table 4 the intermediate/higher qualifications achieved by the participants.

61

Learning 2.0

Figure 2. Screenshot of Moodle page

Table 3. Standard grade qualifications achieved

Table 4. Intermediate or higher qualifications achieved

Standard Grade Qualification

Number with qualification

Mathematics

21 (81%)

Higher Grade Qualification

Number with qualification

English

24 (92%)

Mathematics

20 (77%)

Physics

17 (65%)

English

15 (58%)

Craft and Design

13 (50%)

Physics

11 (42%)

French

16 (61.5%)

Craft and Design

5 (19%)

Biology

6 (23%)

French

3 (12%)

Geography

13 (50%)

Biology

1 (4%)

Chemistry

10 (38%)

Geography

5 (19%)

Modern Studies

13 (50%)

History

5 (19%)

German

2 (8%)

None

3 (12%)

Spanish

1 (4%)

Modern Studies

2 (8%)

20 (77%)

Others

14 (54%)

Other

Evaluation Questionnaire Students were asked to complete a questionnaire at the end of the Induction programme. 41 participants completed the evaluation questionnaire. Participants were asked to rate their level of interest in the training on a scale of 1 to 4 (1 being a low

62

level and 4 being a high level) for each section of the Induction course, namely Day 1 General Introduction, Workplace Environment, Tools & Maintenance, Business Improvement Techniques, ICT, COSHH & Hand Care, Fire Prevention, Risk Assessment, Electrical Safety, Measurements and Materials. The overall results were generally posi-

Learning 2.0

Figure 3. Screenshot of ePortfolio page

tive indicating that the training was sufficiently interesting for the participants. The area of the training that had the highest level of interest was Electrical Safety (Mean = 3.41, SD = 0.59). The areas of training that received the lowest levels of interest were Business Improvement Techniques (Mean = 2.93, SD = 0.65) and the General Introduction on the first day (Mean = 2.93, SD = 0.80). Participants were also asked to rate whether they found the training sufficiently challenging (1 = lowest, 4 = highest). Electrical Safety was rated as the most challenging (Mean = 3.29, SD = 0.60) while the General Introduction on the first day was rated as the least challenging (Mean = 2.75, SD = 0.71). The overall approachability of the instructors was rated highly by the participants in all areas. The average rating for approachability across all areas was Mean = 3.46, SD = 0.86 with a range of 3.32 to 3.59 out of 4. Participants were asked to rate how much they believed that their knowledge improved in each

of the areas. The area rated as having the highest level of knowledge improvement was COSHH (Control of Substances Hazardous to Health) & Hand Care (Mean = 3.51, SD = 0.60). The area rated as having the least level of knowledge improvement was the General Introduction (Mean = 3.17, SD = 0.68). Participants were also asked to rate how appropriate they considered the duration of the activities to be. Overall the results were not particularly positive. The most appropriate rated duration was for Electrical Safety (Mean = 3.15, SD = 0.76) and the least appropriate rated duration was for the General Introduction (Mean 2.75, SD = 0.74). Participant ratings for the helpfulness of teaching aids were generally positive. The highest rating for teaching aids was in the area of Electrical Safety (Mean = 3.49, SD = 0.51) and the lowest rating was in the Helpful Work Place Environment teaching area (Mean = 3.22, SD = 0.52). Participants were also asked to rate if Moodle helped in each of the teaching areas. Table 5 shows the

63

Learning 2.0

rankings of perceived help Moodle provided in each of the teaching categories. Overall, the results of the evaluation questionnaire were generally positive. Electrical Safety consistently achieved the highest ratings for level of interest, challenge, appropriate duration and perceived help provided by Moodle. The General Introduction on the first day received the highest amount of criticism primarily because of its duration as it was considered by participants to be too long and lacked participation and interaction. Participants provided some of the following qualitative comments regarding the duration of the General Introduction: Day 1 was not that interesting because there was not a lot of participation - it was all listening. I think the 3 day introduction as a whole could be delivered as a participation lesson. The first day was not interesting enough. We were just thrown information and didn’t have enough time to absorb the information. Duration - induction was too long and it could involve the trainees more to make it more exciting.

Participants also provide qualitative points for action and improvement. These primarily included: more interaction, more participation and the reduction of the duration of the General Introduction.

GROUP PROJECT The group project was designed to develop and promote team-working skills that are applicable to real-life working in the engineering industry and is the last assessment the students take before going out into industry. The overall objective of the project is for team members to build a truck within a nominal budget. Each member of the team is assigned a role, for example, project manager, chairman, secretary or quality control. To trial Moodle in the group project, it was decided to introduce a wiki for formal recording of the project, a forum to facilitate communication when not all members of the team were present and an assignment folder to allow candidates to upload all documentation for assessment. Two tests were designed: a pre-test to help identify current knowledge and a post-test to help identify new knowledge and students’ views of the wiki and forum. A total of 43 candidates participated in the

Table 5. Ratings of help provided by Moodle in each teaching category Teaching Area

64

Rank

Mean

Standard Deviation

Risk Assessment

1st

3.56

0.50

Electrical Safety

1st

3.56

0.55

Materials

1st

3.56

0.55

Measurements

2nd

3.51

0.60

Work Place Environment

3rd

3.49

0.51

Tools and Maintenance

3rd

3.49

0.55

Fire Prevention

3rd

3.49

0.55

COSHH and Hand Care

4th

3.46

0.50

Business Improvement Techniques

5th

3.44

0.59

ICT

6th

3.41

0.59

Day 1General Introduction

7th

3.33

0.58

Learning 2.0

group project. The candidates were divided into 6 teams with 7-8 students per team. The findings are discussed below.

If someone comes up with an idea then they can post it at anytime even when team members are not there.

Pre-Project Test

You can have access when you are off and record progress.

33 participants completed the pre-test questionnaire. All of the respondents were male. The mean age of participants was 17.18 (SD = 0.73) with a range from 16 to 19.8. The majority of participants (30, 91%) indicated that they had been involved in group work before, while 3 (9%) had not. The majority (27, 82%) of the participants who had participated in group work before indicated that they took part in the group work at school, 10 (30%) at college and 3 (9%) at work. None of the participants had ever used a wiki before. The majority of the participants (24, 73%) described a wiki as an ‘an online collaboration tool with built in tracking’, 7 (21%) described a wiki as ‘a social networking tool’ and 2 (6%) described it as ‘an online instant messaging tool’. The majority of participants (21, 64%) indicated that they had used an online forum before while 12 (35%) had not. 18 (55%) participants had used an online forum for social use, 5 (15%) had used an online forum for a school project, 2 (6%) had used one for a college project and as part of a work project. 12 (36%) of participants believed that an online forum was ‘a social networking tool’, 14 (42%) believed that it was ‘an online chat room’ and 4 (12%) believed that it was ‘an online collaboration tool with built in tracking’. 23 (70%) of participants believed that the use of a wiki 24x7 would encourage teamwork, while 10 (30%) believed this not to be the case. Those who indicated that a wiki would encourage team work gave some of the following reasons: Allows you to keep every member of the team up to date.

Those who indicated that a wiki would not encourage teamwork gave some of the following reasons: Better to discuss the project in person so that you are able to give them a clearer picture of what is done. The whole team would have to use it to benefit and they were not. 22 (67%) of participants believed that the use of a forum would help open up communication between members while 11 (33%) believed this not be the case. Those indicating that the forum would open up communications gave the some of the following reasons: If one team member is not in one day he/she can then see what his/her team has done that day. It can get everyone involved. 17 (52%) of the participants were not aware of any project management tools, while others listed Excel, Word, PowerPoint, efacts, wiki, graphs, flip charts, CAD, Gantt charts and Moodle as project management tools. The majority (22, 67%) indicated that they had not used any project management tools in the past while 11 (33%) had. The majority of participants (21, 64%) indicated that they thought a Project Route Map to be ‘a tool that helps to identify all areas of the project that need to be considered’, 11 (33%) believed it to be ‘a tool that is used to help plan the project and then consulted on a regular basis’ and 1 (3%) believed it to be ‘a tool that is only used to plan

65

Learning 2.0

the project but then filed away’. Participants were asked to list the project resources that they were aware of, 12 (36%) indicated that they were not aware of any, 3 (9%) indicated that they were aware of Moodle as a project resource; 4 (12%) of participants also mentioned the Internet and books as a project management resource. 18 (55%) of participants believed a budget to be ‘a set amount of money that can be spent on the project from start to finish’, 14 (42%) of participants believed that a budget was ‘an amount of money that should cover all cost of the project from start to finish’ and 1 (3%) believed that is was ‘the amount of money that is set for the first stage of the project but is ‘topped up’ as the project progresses’. 14 (42%) of participants identified the information pyramid as something that ‘allows the presenter to identify between the different types of information ensuring that the most important information is not omitted’, 10 (30%) as something that ‘helps the presenter to divide the information into equal parts’ and 9 (27%) as something that ‘helps the presenter to ensure that the correct information is conveyed during a presentation’.

Post-Project Test 26 participants completed the post-test. Participants were asked how much they agreed with the following statement: ‘everyone participated in the group project’. To get an overall rating of the perception of overall participation, strongly agree was recorded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean rating for perceived participation was 2.69 (SD = 1.19) with a range from 1 to 5. This indicates that the level of perceived participation was not particularly high with the majority of participants either disagreeing 11 (42%) or strongly disagreeing 4 (15%) that everyone was participating in the group activity. 13 (50%) of participants described a wiki as ‘an online collaboration tool with built in tracking’, 8 (31%) described it as ‘a social networking

66

tool’, 2 (8%) described it as ‘an online chat room’ and ‘an online instant messaging tool’. 10 (38%) identified a forum as ‘an online collaboration tool with built in tracking’, 7 (27%) identified it as ‘a social networking tool’, and ‘an online chat room’. 2 (8%) described it as ‘an instant messaging tool’. The majority of participants 15 (58%) said that they did not think that a wiki promoted teamwork, while 10 (38%) believed that it would promote teamwork. Those who indicated that the wiki would promote teamwork gave some of the following reasons: People could catch up on work they may have missed. Helped each other with the difficulties. Those who indicated that the wiki would not promote teamwork gave some of the following reasons: Took too long to complete and was repetitive. We didn’t use it much. Participants were asked how much they agreed with the following statement ‘if the wiki was a compulsory element of the group project all team members would have used it’. To produce a mean score of the expected participation level, strongly agree was recoded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean rating was 2.76 (SD = 1.03) with a range of 1 to 4. This means that approximately 50% of participants would use the wiki if it were a compulsory part of the group project. 13 (50%) of participants believed that the forum helped with communication and 13 (50%) did not. Those who believed that the forum helped with communication gave some of the following reasons: All people can communicate if not present.

Learning 2.0

People could see things they missed. Those who believed that the forum did not help with communication gave some of the following reasons: We didn’t use it much. Easier to talk. Participants were asked to rate their level of agreement of this statement: ‘If the forum was a compulsory element of the project, all team members would have used it more for communicating with those team members who are not present on a full-time basis’. To produce a mean score of the expected participation level, strongly agree was recoded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean score rating of expected participation was 3.23 (SD = 0.95) with a range from 1 to 5. This indicates that the forum was more popular than the wiki in terms of expected participation. Participants were asked to list all of the project management tools that they were now aware of, 6 (23%) of participants mentioned a wiki, 8 (31%) answered ‘none’, 2 (8%) mentioned Gantt charts and 1 (4%) mentioned Moodle. 13 (50%) of participants now identified a Project Route Map as: ‘a tool to help identify areas of the project that need to be considered’, 2 (8%) identified it as ‘a tool used to plan the project and is then filed away’ and 11 (42%) identified it as ‘a tool that is used to identify areas of the project that need to be considered and consulted on a regular basis’. Participants were asked to list all of the project resources that they were familiar with now that the project was complete: 6 (23%) of participants mentioned a ‘wiki’, 11 (42%) said ‘none’ and a small number mentioned talks, PowerPoint, Gantt charts and Moodle. The majority of participants (14, 54%) identified a budget as ‘an amount of money that can be spent on the project to cover all project costs from start to finish’, 11 (42%)

identified it as ‘a set amount of money that can be spent on the project from start to finish’, and 1 participant (4%) identified it as ‘the amount of money that is allocated to material for the project’. 6 (23%) of the participants identified the information pyramid as ‘a tool that helps the presenter to divide the information into equal parts’, 9 (35%) as ‘a tool that helps the presenter to ensure that the correct information is presented during the presentation’ and 11 (42%) as ‘a tool that helps the presenter identify between the different types of information ensuring that the most important information is not omitted’.

Comparison of Pre and Post-Tests 21 participants completed both the pre- and the post-tests. A Wilcoxon Signs ranked test indicated that the increase in knowledge was not significant with regards to wikis (Z = -.890, p = 0.03) and forums (Z = -.816, p = 0.414). However, when combined together, a Wilcoxon Signs ranked test indicated that there was a significant decrease in forum and wiki knowledge (Z = -1.807, p = 0.04). The mean in the pre-test was 1.14 out of 2 (SD = 0.65) and the mean in the post-test was 0.81 out of 2 (SD = 0.68). This was primarily due to a significant reduction in wiki knowledge after the training (Z = -.890, p = 0.03). This is possibly due to the fact that the instructors did not promote the wiki as it was not compulsory and instead suggested that the students used the forum to maintain the project documentation. A Wilcoxon Signs ranked test indicated that the increase in knowledge was not significant with regards to project route maps (Z = -1.000, p = 0.317), the information pyramid (Z = -.378, p = 0.705) and budgets (Z = -1.265, p = 0.206). However, when combined together there was a significant increase in the project management knowledge between the pre-test and post-test (Z = -1.615, p = 0.05). The mean in the pre-test was 1.10 out of 3 (SD = 0.94) and the mean in the posttest was 1.42 out of 3 (SD = 0.87). A significant

67

Learning 2.0

correlation was detected between undertaking group work previously and the knowledge score for budget in the pre-test (r = -.414; n = 21; p = 0.03). There was also a significant correlation between undertaking group work previously and the knowledge score for forums in the post-test (r = -.459, p = 0036). A significant correlation between using project management tools previously and knowledge scores about wikis in the post-test (r = -.539, p = 0.01) was also detected. In addition, there was a significant correlation between using project management tools previously and knowledge about Project Route Maps in the post-test (r = -.408, p = 0.03). While some of these correlations were expected, some of the others were not and these will be tested again with subsequent cohorts going through the project module.

I am keen to start uploading my lectures and restructure my lessons to incorporate Moodle All courses could be delivered online but investment in teaching material, software and manpower would have to be made to progress the project Reinforcement of the tutor lead sessions and the ability to access the information from anywhere in the centre It gives those trainees who are self motivated the opportunity to keep pace or catch Groups were able to share information with team members and use it as a note pad for ideas

It is limited to providing a storage facility for course material with the odd quiz to reinforce training

There was a definite divide between the instructors. Some could identify the support that Moodle and the Web 2.0 tools could bring to the training programme, whist others did not. Part of this could be attributed to the traditional nature of engineering; however, the instructors who were new to the company were more open to new ideas. The students who were working under the supervision of those instructors who promoted the use of the technologies were far more receptive and willing to use Moodle and the Web 2.0 tools. However, the students who were supervised by those instructors who did not encourage the use of the technologies were far more resistant to their use. This demonstrates that there must be enthusiasm and guidance in the use of technology to ensure its successful uptake. It should be noted that these issues were brought to the attention of management by the authors, but management felt they could not force instructors to use the technologies.

The wiki only helped to confuse matters in this short space of time

CONCLUSION

Some of the positive comments made by instructors were as follows:

This chapter set out to discuss the impact on learning with the introduction of a LMS into a

INSTRUCTORS RESULTS Instructors were surveyed to obtain feedback on their opinion of the Web 2.0 tools used during the group project, to gauge their acceptance of Moodle, and to identify areas where they believed the LMS could be utilised further. The course is a practical course and would not lend itself well to online delivery It was long winded and the trainees had to build it themselves, the wiki would be ideal for large academic projects that spans a university course or is used in a notational sense

68

Learning 2.0

vocational training course and to answer (i) can technology supplement the hands-on nature of vocational training, (ii) can the use of LMS and Web 2.0 tools (specifically wikis and forums) aid training, and (iii) can the pilot be considered a success? Much of the work carried out in this sector is hands-on and on-the-job training but it also consists of a substantial knowledge element that would lend itself well to the use of eLearning. From the literature review provided within this chapter, it may be argued that the use of educational technologies for practically-based courses (e.g. vocational training) should be a relatively straight forward process. Early eLearning was used in a practical capacity; however, the case study demonstrates that the introduction of learning technologies into a vocational training course was received with mixed reviews. This could largely be accredited to the traditional nature of engineering training but what was obvious from the data collected and by the authors’ experience in working with the organisation, much of the opposition to the technology use was due to the lack of flexibility and understanding from the instructors. From the analysis it can be seen that the technology was received with a mixed response. The online induction course was well received by the students (and was also accepted by the instructors). Unfortunately, the use of wikis and forums for the project module, while well received by the students was not fully adopted by the instructors, possibly as they were not confident of their knowledge and use of the technologies. As a result, they made the use of the wiki optional and instead asked the students to use the Moodle forum to maintain their project documentation. This caused the students to become confused as to the distinction between forums and wikis, resulting in a decrease in learning. The authors conclude that the research demonstrates that educational technologies could potentially aid the vocational training of apprentices and that this area could offer a rich source of future research. However, it is early days to design a framework that will aid practitioners in

the successful implementation of a LMS into a private vocational training organisation but as the empirical data grows this should become possible. At this stage the authors suggest, tentatively, that eLearning can support and in some instances replace some elements of a vocational programme. The authors acknowledge that the practical element would be very difficult to replace with technology but it could be supplemented through other tools such as multimedia and even games technology. Educational technologies offer the opportunity to open up learning in the vocational training sector, rather than restricting it to the traditional 9-5 scenario. The authors would recommend that those educators who deliver traditional vocational training courses be fully consulted during the implementation process as well as receive the necessary training needed to highlight the potential benefits that technology could bring to the overall course being delivered. If buy-in from students can be obtained and instructors can learn to appreciate the value that could be gained from educational technology then its use could diffuse throughout the vocational training sector.

ACKNOWLEDGMENT This project received financial support from the Knowledge Transfer Partnerships programme (KTP). KTP aims to help businesses to improve their competitiveness and productivity through the better use of knowledge, technology and skills that reside within the UK Knowledge Base. KTP is funded by the Technology Strategy Board.

REFERENCES Beetham, H. (2009). E-portfolios in post-16 learning in the UK: Developments, issues and opportunities. Retrieved October 2, 2009, from http://www.jisc.ac.uk/uploaded_documents/eportfolio_ped.doc

69

Learning 2.0

Berlanga, A. J., Sloep, P. B., Brouns, F., BitterRijpkema, M. E., & Koper, R. (2008). Towards a TENCompetence e-portfolio. [iJET]. International Journal of Emerging Technologies in Learning, 3, 24–28. City & Guilds. (2009). How are centres using ePortfolios. Retrieved November 12, 2009, from http://www.cityand guilds.com/44912.html Connolly, T. M. & Stansfield, M. H. (2007a). From e-learning to games-based e-learning. International Journal of Information Technology and Management, 26(2/3/4), 188-208. Connolly, T. M., & Stansfield, M. H. (2007b). From e-learning to online games-based e-learning: Implication and challenges for higher education and training. In Li, F. (Ed.), Social implications and challenges of e-business. Hershey, PA: IdeaGroup Publishing. Curyer, S., Leeson, J., Mason, J., & Williams, A. (2007). Developing e-portfolios for VET: Policy issues and interoperability. Australian flexible learning framework. Retrieved October 16, 2009, from http://e-standards.flexiblelearning.net.au/ docs/vet-eportfolio-report-v1-0.pdf DeRouin, R. E., Fritzsche, B. A., & Salas, E. (2005). E-learning in Organizations. Journal of Management, 31(6), 920–940. doi:10.1177/0149206305279815 Dorninger, C., & Schrack, C. (2007). E-portfolio’s in education-learning tools or means of assessment? In Proceedings of the International Computer-Aided Learning Conference (ICL2007), Villach, Austria. EDUCAUSE. (2005).Retrieved on February 11th, 2011, from E-Portfolios Web page, URL: http://www.educause.edu/ELI/Archives/EPortfolios/5524

70

Franklin, T., & Harmelen, M. (2007). Web 2.0 for content for learning and teaching in higher education. Joint Information Systems Committee (JISC) report. Retrieved August 8, 2010, from http://ie-repository.jisc.ac.uk/148/1/web2content-learning-and-teaching.pdf Garrison, R. (1997). Computer conferencing: The post-industrial age of distance education. Open Learning, 12(2), 3–11. doi:10.1080/0268051970120202 Gerring, J. (2007). Case study research: Principles and practices. Cambridge, UK: Cambridge University Press. Grosseck, G. (2009). To use or not to use Web 2.0 in higher education? Procedia Social and Behavioral Sciences, 1, 478–482. doi:10.1016/j. sbspro.2009.01.087 Gunawardena, C. N. (1993). The Social context of online education. In Proceedings of the Distance Education Conference, Portland, Oregon. Holmes, B., & Gardner, J. (2006). E-learning concepts and practice. London, UK: SAGE Publications. Horton, W. (2003). E-learning tools and technologies: A consumer’s guide for trainers, teachers, educators, and instructional designers. Indianapolis, IN, USA: Wiley. JISC. (2008). Effective practice with e-portfolios: Supporting 21st century learning. Joint information systems committee. London, UK: JISC. Jonassen, D. H. (1996). Computer-mediated communication: Connecting communities of learners. Computers in the Classroom (158–182). Edgewood Cliffs, NJ: Prentice-Hall, Inc. Kozma, R. (1987). The implications of cognitive psychology for computer-based learning tools. Educational Technology, 27(11), 20–25.

Learning 2.0

Lorenzo, G., & Ittelson, J. (2005). An overview of ePortfolios, Educause learning initiative. Mason, R., Pegler, C., & Weller, M. (2004). Eportfolios: An assessment tool for online courses. British Journal of Educational Technology, 35(6), 717–727. doi:10.1111/j.1467-8535.2004.00429.x McBride, R., & Schostak, J. (1994). What is qualitative research. Retrieved May 7, 2009, from http://www.enquireylearning.net/ELU/Issues/ Research/Res1Ch1.html McDonald, J. (2002). Is “as good as face-to-face” as good as it gets? Journal of Asynchronous Learning Networks, 6(2), 10–23. McKelvie, G., Dotsika, F., & Patrick, K. (2007). Interactive business development, capturing business knowledge and practice: A case study. The Learning Organization, 14(5), 407–422. doi:10.1108/09696470710762637 Moodle. (2010). Moodle Statistics. Retrieved August 25, 2010, from http://moodle.org/stats/ Richardson, H. C., & Ward, R. (2005). Developing and implementing a methodology for reviewing e-portfolio products, The centre for recording achievement (CRA). Retrieved October 10, 2009, from http://www.jisc.ac.uk/uploaded_documents/ epfr.doc Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students (4th ed.). Pearson Education Ltd. Strivens, J. (2007). A survey of e-pdp and ePortfolio practice in UK higher education. UK: Higher Education Academy. Wenger, E. C. (2000). Communities of practice and social learning systems. Organization, 7(2), 225–246. doi:10.1177/135050840072002 Wenger, E. C., & Snyder, W. M. (2000). Communities of practice: The organizational frontier. Harvard Business Review, 78(1), 139–145.

Whisker, G. (2001). The Postgraduate Research Handbook. Basingstoke, UK: Palgrave MacMillan.

ADDITIONAL READING Agostini, A., De Michelis, G., & Loregian, M. (2009). Using blogs to support participative learning in university courses. Int. J. Web Based Communities, 5(4), 515–527. doi:10.1504/ IJWBC.2009.028087 Baxter, G. J., Connolly, T. M., & Stansfield, M. (2009). The use of blogs as organisational learning tools within project-based environments. Int. J. Collaborative Enterprise, 1(2), 131–146. doi:10.1504/IJCENT.2009.029285 Berge, Z. L., & Muilenburg, L. Y. (2005). Student Barriers to Online Learning: A factor analytic study. Distance Education, 26(1), 29–48. doi:10.1080/01587910500081269 Cartelli, A., Maillet, K., Stansfield, M. H., Connolly, T. M., Jimoyiannis, A., Magalhaes, H., & Toland, J. (2008). Identifying and promoting best practice in virtual campuses and e-searning, ED-MEDIA 2008 Conference - World Conference on Educational Multimedia, Hypermedia & Telecommunications, Vienna, Austria (30 June-4 July 2008) (pp. 897-902). Chapman, C., Ramondt, L., & Smiley, G. (2005). Strong community, deep learning: Exploring the link. Innovations in Education and Teaching International, 42(3), 217–230. doi:10.1080/01587910500167910 Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52(1), 141–146. doi:10.1016/j.compedu.2008.07.003

71

Learning 2.0

Connolly, T. M., & Stansfield, M. H. (2007). Developing constructivist learning environments to enhance learning. In N. Buzzetto-More (Ed.), Principles of effective online teaching: A handbook for experienced teachers developing e-learning, 19-38.

Lorenzo, G., & Ittelson, J. (2005). An overview of e-portfolios. D.Oblinger (Ed.), Educause learning initiative paper (pp. 1-28). Retrieved October 18, 2010, from: http://www.sorteoudla.org.mx/ promueve/ciedd/CR/tecnologia/AnOverviewofEPortfolios.pdf

Gerrard, C., Connolly, T. M., & Stansfield, M. (2006). The role of staff development to enhance the integration of e-learning within the HE curriculum, European Conference on e-Learning (ECEL) 2006, 11-13 September 2006, University of Winchester, UK.

Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4), 134–148.

Grippa, F., & Secundo, G. (2009). Web 2.0 projectbased learning in higher education: some preliminary evidence. Int. J. Web Based Communities, 5(4), 543–561. doi:10.1504/IJWBC.2009.028089 Hall, H., & Davison, B. (2007). Social software as support in hybrid learning environments: the value of the blog as a tool for reflective learning and peer support’. Library & Information Science Research, 29(2), 163–187. doi:10.1016/j. lisr.2007.04.007 Illeris, K. (2002). The Three Dimensions of Learning: Contemporary learning theory in the tension field between the cognitive, the emotional and the social. Copenhagen: Roskilde University Press. Kim, S. W., & Lee, M. G. (2007). Validation of an evaluation model for learning management systems, Journal of Computer Assisted Learning, Blackwell Publishing Ltd. Larusson, J. A., & Alterman, R. (2009). Wikis to support the “collaborative” part of collaborative learning. Computer-Supported Collaborative Learning., 4(4), 371–402. doi:10.1007/s11412009-9076-6 Lawless, N., & Allan, J. (2004). Understanding and reducing stress in collaborative e-learning. Electronic Journal of e-Learning, 2(2).

72

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books. Schön, D. A. (1987). Educating the reflective practitioner: Towards a new design for teaching in the professions. San Fransisco, CA: Jossey-Bass Inc. Shin, J., & Bickel, B. (2008). Communities of practice: Creating learning environments for educators (Kimble, C., & Hildreth, P., Eds.). Information Age Publishing. Stansfield, M. H., & Connolly, T. M. (Eds.). (2009). Institutional transformation through best practices in virtual campus development: Advancing e-learning policies organizations. Hershey, PA: IGI Global Publishing. doi:10.4018/978-160566-358-6 Stansfield, M. H., & Connolly, T. M. (2009). An exploration into key issues relating to the adoption of good practices in e-learning and virtual campuses. In Mayes, T., Morrison, D., Mellar, H., Bullen, P., & Oliver, M. (Eds.), Transforming higher education through technology enhanced learning. Higher Education Academy. Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice. Boston, MA: Harvard Business School Press.

Learning 2.0

Williams, J. B., & Jacobs, J. (2004). Exploring the use of blogs as learning spaces in the higher education sector. Australasian Journal of Educational Technology, 20(2), 232–247.

KEY TERMS AND DEFINITIONS Communities of Practice (CoPs): Groups of people informally bound together by shared expertise and passion for a joint enterprise (Wenger & Snyder, 2000, pp. 139). E-Portfolios: A valuable online tool that learners, faculty, and institutions can use to collect, store, update, and share information. E-Portfolios allow students to reflect on their learning, communication with instructors, document credentials, and provide potential employers with examples of their work (EDUCAUSE, 2005). eLearning: Any use of Web and Internet technologies to create learning experiences (Horton, 2003, pp. 13).

LMS: The components in which learners and tutors participate in online interactions of various kinds, including online learning (Becta, 2003) Portfolio: A collection of documents relating to a learner’s progress, development, and achievements (Beetham, 2009). Social Software: Social software is a community driven technology which facilitates interaction and collaboration and depends largely on social convention (McKelvie et al., 2007). Web 2.0: The social use of the Web which allow[s] people to collaborate, to get actively involved in creating content, to generate knowledge and to share information online (Grosseck, 2009, pp. 478). Wiki: A website (or other hypertext document collection) that allows users to add content, as on an Internet forum, but also allows anyone to edit the content (G. Avram, 2006)

73

Section 2

Implementing and Evaluating

75

Chapter 4

Evaluations of Online Learning Activities Based on LMS Logs Paul Lam The Chinese University of Hong Kong, Hong Kong Judy Lo The Chinese University of Hong Kong, Hong Kong Jack Lee The Chinese University of Hong Kong, Hong Kong Carmel McNaught The Chinese University of Hong Kong Hong Kong

ABSTRACT Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized Web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University.

INTRODUCTION Learning management system (LMS) is a broad term that is used for a wide range of systems that organize and provide access to online learning services for students, teachers and administrators. These services usually include access control,

provision of learning content, communication tools, and organizations of user groups (Paulsen, 2002). Jovanovic et al. (2007) defined an LMS as “a software environment that enables interactive web-based teaching and supports administration of distance courses, allowing instructors to distribute information to students, producing course mate-

DOI: 10.4018/978-1-60960-884-2.ch004

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Evaluations of Online Learning Activities Based on LMS Logs

rial, preparing assignments and tests, engaging in discussions, and managing courses and distance classes” (p. 46). In 2005, 95% of all higher-education institutions in the UK were using an LMS (Browne, Jenkins, & Walker, 2006). At The Chinese University of Hong Kong (CUHK), two LMS (WebCT and Moodle) are centrally supported. Indeed, the majority of the University’s eLearning activities are supported by these central services; apart from one faculty (Engineering), there are relatively few non-centrally-hosted course websites. Effective record-keeping, and extraction and interpretation of eLearning logs, can reveal valuable information on how these LMS are used to facilitate teaching and learning. As noted by Sen, Dacin and Pattichis (2006), the use of logs for tracking user activities is quite common in commercial settings where customer habits and trends are traced and monitored. Reading user logs also applies in educational settings – for example, the study by Black, Dawson and Priem (2008) on how to obtain measures of ‘community’ in online courses. In universities with centralized web-based teaching and learning systems, the logs can be monitored through inbuilt mechanisms to track and record a certain amount of information about online activities. Colace and De Santo (2003) commented that monitoring an LMS can enable detailed and useful information on the LMS’s utilization and efficacy. This information can include trend data if the logs have been collected regularly over time. Such information can provide the basis for various decisions related to the implementation and promotion of eLearning. However, these inbuilt web-log tracking systems do not normally provide institution-level data. The weblogs are reported in an interface designed for individual teachers to get a summary of activities recorded in individual courses (Mazza & Milani, 2005) rather than for analyses of online learning activities at higher levels (e.g. department, faculty or institution). Retrieval of

76

data in WebCT (version 3.8) is even more challenging. As it does not employ a database structure, these records or logs of activities can be extracted using the provided display logs functions which are very limited in functionality. The locations where the record information is stored are not clear because of the lack of a database structure, and so time and effort are needed for: 1) testing through trial and error for the allocation of the desired information; 2) checking whether the data are accurate; and 3) developing software to extract the information for all courses in the University. After retrieval of information, additional work is needed to standardize and automate the integration, interpretation and reporting processes of the log data so that we have a common ground to compare and contrast eLearning uses over time. Romero, Ventura and Garcia (2008) discussed using data-mining techniques to explore the raw log data of servers in order to understand various aspects of learner activities. However, such strategies, though allowing great flexibility in the topics of study, are technically complex. As noted by Black, Dawson and Priem (2008) “server logs are plagued with a low signal-to-noise ratio: simply preparing the data for modeling can consume 80% to 95% of a project’s resources” (p. 67). A system that is more powerful than the LMS inbuilt activity log systems, and can regularly retrieve and interpret the logs into a number of fixed representations for year-by-year comparison and contrast, seems to be what we need. Zhang et al. (2007) reported a similar system called Moodog which monitor students’ activities live on the Moodle LMS. Our system looks at the issue more from an institutional point of view. We may not need to monitor student activity every moment but retrieve and analyze the logs once every semester. Also, because of our particular context, our system reads and integrates as much as possible of the logs from both of the LMS we host – WebCT and Moodle. Earlier development and the framework of the system were reported in Lam et al. (2006).

Evaluations of Online Learning Activities Based on LMS Logs

The work at that time was more focused on the data-retrieval stage. The present paper extends our discussion to the automation of the data interpretation and reporting processes. As noted in Lam, Keing, McNaught, and Cheng (2006), system logs data can provide information on: 1) popularity; 2) the nature of the functions/ strategies in use; and 3) engagement of teachers and students. 1. The notion of popularity is a very simple yes/no specification for each course in the University whether any eLearning activities are recorded in our logs or not. 2. The nature of the eLearning activities recorded for each web-enabled course refers to uses of forums, assignment-submission service, course-content delivery, online quizzes or surveys, grade-book facility, etc. 3. Engagement reflects how involved teachers and/or students are in these activities. This is the level among the three that convey the finest amount of detail about a site. After the recognition that there is a course website (popularity), more information can reveal the actual features and activities having occurred on the site (nature). After learning about the nature of the website, further information can gauge how active teachers and students are on site (engagement). The data report the actual activities recorded and, to a certain extent, they fit the requirements of the naturalistic research paradigm (Alexander & Hedberg, 1994; Grasha, 1990) that collection of evaluation data is usefully done in non-intrusive ways (Lam, McNaught & Cheng, 2008) and should, where possible, be situated in authentic educational settings (Froehlieh, 1994). We do not claim that weblog data represent a comprehensive solution to all evaluation needs. More comprehensive evaluation studies would consider evidence, both quantitative and qualitative in nature, from a variety of sources. The

present paper serves only to illustrate the advantages as well as limitations to using weblogs as a data source. Apart from relatively easy access to the data, the extraction of system logs is also completely non-intrusive to both teachers and students. Furthermore, repeated measures can be taken for a long period of time so that comparison of usage over time is possible. The largely automatic extraction of the logs (once the software for logs extraction is devised), and the standardized methods in the follow-up analysis, reporting, and interpretation of the data, enable an institution to build a mechanism which is suitable for administration on a regular basis (e.g. annually). A comparable system of logs extraction and analysis can then be used in subsequent monitoring. One limitation of this approach, however, is that it monitors only uses of the web that utilize the central system. Also, it has a bias on quantity rather than quality as logs are numbers rather than a full picture of the educational quality of the content on course websites. Also, not all online activities and teachers’/ students’ engagement in these activities can be effectively recorded by the logs. For example, the availability of course outlines on course websites is an online activity that is of interest to universities. However, ‘online course outline’ is not an activity separately recorded by the logs of WebCT or Moodle. It is impossible to identify course outlines unless the researchers go into individual websites and read all the documents. The picture portrayed by the logs is only a partial representation of the total online learning activities and the engagement in these activities. Even if the logs reveal certain information on popularity of, nature of and engagement with activities in websites, care has still to be taken to understand the exact meanings of these logs based on the characteristics of the platforms and how logs are kept in them. Very often, minor adjustments have to be made or there are decisions to make

77

Evaluations of Online Learning Activities Based on LMS Logs

concerning the cut-off points beyond which the records are deemed to fall into another category. In this paper, we: 1. Explain how the system works. 2. Use empirical evidence recorded from 2007 to 2010 to show how the data could be employed to achieve various levels of analysis; and 3. Discuss how the better understanding of the online activities have enabled decisions in a number of eLearning support initiatives in our University. We consider that the data have assisted in better understanding and refining our eLearning strategies and supports at both an institutional level and faculty/ department level.

HOW THE SYSTEM WORKS The following measures have been taken to refine and standardize the data so that we identified the right types of logs for the types of activities we targeted. Clear definitions are also necessary, particularly in our case where we have merged weblog readings from two different LMS. The standardization makes sure the readings from the two platforms record the same underlying activities and record them in a compatible manner. Not all websites created on the servers should be considered ‘active’ websites; this means that there are some sites that are developed but not made accessible to students. For example, some of the websites might have been created by the teachers or the teaching assistants for testing purposes (e.g. perhaps to be used in the next term) and there were actually no real student activities on the site. In our weblog study, therefore, we took care to isolate only the so-called ‘active’ websites – websites that had at least one student access during the course period would be included in the data. The similar ‘active’ concept was also applied in the study of the other four eLearning strategies

78

in our study. A website that had none of these active features may mean a site that was used by the teacher for only announcing course information such as examination dates or news of events. •

• •

•

Whether or not a particular course used ‘forum’ for ‘active discussion’, for example, was judged not only from the existence of a forum on the site, but rather on whether there was at least one student access to the forum as well. ‘Active online quizzes’ meant quizzes that had at least one student attempt. ‘Active assignment submission’ meant that at least one student submitted work to the platform. ‘Active content’ meant the website contained at least one document (could be PowerPoint, Word or PDF documents, or any other multimedia files) for download, and there was at least one student download recorded in the logs.

The use of eLearning strategies is more meaningful in courses that have considerable class sizes. We have introduced a mechanism to filter cases based on class size. In our web logs study, we only studied classes with a class size equal to or larger than 10. Lastly, it is worth noting that the unit of analysis in our study was normally a course. In most cases, it was relatively easy to decide whether a certain course had used eLearning strategies or not. However, some decisions had to be made in many other cases. Sometimes one course was run in a number of sessions (at different times and even by different teachers). Our standpoint was to consider a course eLearning-enabled IF at least one of the sessions had an active website on any one of the LMS. Figure 1 shows the report interface of the weblog system. The interface is comprised of several parts such as searching filters (Boxes 1 and 2), result window (Boxes 3a and 3b) and graph

Evaluations of Online Learning Activities Based on LMS Logs

Figure 1. Inquiry layout of weblog system

window (Box 4). Once a set of filters is submitted into the search engine, results and a graph will be shown in the result and graph windows. Also, in order to provide the flexibility to enable multiple levels of analysis, different parameters can be set for the filters. For example, we can ask for a report on a certain year/ term or a report that contrasts activities in multiple years or terms. Reports can be on institution, faculty, department, or course levels. We can call for records of undergraduate courses, postgraduate courses, or both. The unit of analysis is normally a course but there is an option to drill down into the use of LMS by the individual sessions of a course (taught by the same or different teachers). Also, the system can report actual numbers (e.g. how many active course websites and how many courses we have in the University) or report percentages (e.g. the percentage of courses that have an active website). These parameters are available for the administrators in the Box 1 area in Figure 1.

In Box 2, administrators can specify the exact year or term to report on, as well as the exact faculty, course type and even the course code the report should be limited to if desired. 2009 on the system means the 2009–2010 academic year which is from September 2009 to May 2010 in our context. In addition, there is a choice about class size, i.e. the minimum student number in a class before it can be included (the default is 10). Boxes 3a and 3b in Figure 1 show the weblog results of the whole University in the year 2009. Both the ‘popularity’ (Box 3a) and the ‘nature of activities’ (Box 3b) types of information are shown. Table 1 further explains the sub-categories in these two areas. Box 4 contains graphical representations of the various numerical data reported in Box 3. They are usually sufficient for normal reporting purposes. However, the data and graphs can be exported as an Excel file for further processing and analysis if needed.

79

Evaluations of Online Learning Activities Based on LMS Logs

Table 1. Definitions of the sub-categories in the report Notion

Popularity

Nature of activities

Sub-categories

Definition/ Description

Course Total

Total number of courses

None Total

Courses without using any LMS

WebCT Total

Used WebCT

Moodle Total

Used Moodle

Multi Total

Used both WebCT and Moodle

Active website (AW)

At least one student access to LMS and view website page during the course period

Active content (AC)

At least one document on the active website for access and at least one student download recorded in the logs

Active discussion (AD)

At least one student posts a topic or a thread

Active assignment (AA)

At least one student submitted work to assignment drop-box

Active quiz (AQ)

At least one student attempted quiz on active website

Figure 2 illustrates how the weblog system represents teachers’ and/or students’ ‘engagement’ in various eLearning activities (called level 3 analysis in the system: Box 5). In this example, the data show the amount of activity recorded in using the various functions over all our active WebCT and Moodle websites in the year 2009.

Box 6 shows the tabs that lead to analyses of various types of activities. In the result-grid region now showing (Box 7), we can tell that among all the courses in the year, 128 of them had a website in which student access over the year was between 1 to 100, 731 of them had 101 to 1000 visits, 858 had 1001 to 10,000 visits and 167 had over 10,000 visits in the year. 1586 courses did not have a

Figure 2. Demonstration of result window reporting engagement data

80

Evaluations of Online Learning Activities Based on LMS Logs

website in any of our two platforms or their websites did not record any student visits at all. Similarly, data and graphs concerning engagement in the various activities can be exported as an Excel file for further processing and analysis.

VARIOUS LEVELS OF ANALYSIS The flexibility of the system allows us to conduct various levels of analysis easily. Six main types are distinguished in our framework as shown in Figure 3. The framework is an extended version of the one reported in Lam et al. (2006) as a result of enhanced analytical power through recent developments. In brief, the system provides a general institution-wide overview of eLearning activities (Types A, B and C analysis in Figure 3); it also

has the capability to differentiate faculty- or department-level eLearning practices (Types D, E and F analysis in Figure 3). Type A refers to how popular LMS are across the whole University. Type B refers to the nature of the features used in the LMS. Type C refers to how students and/ or teachers in general are engaged in the various activities. Type D refers to how popular LMS are used in a certain faculty/ department or even a certain course. Type E refers to the analysis of how various eLearning functions are used in the different faculties/ departments or courses. Lastly, Type F analysis distinguishes the engagement in activities by students and/or teachers in various faculties/ departments. Below, we use empirical evidence recorded from the years 2007 to 2009 to illustrate how the data could be employed to achieve these various levels of analyses. We looked at all undergraduate

Figure 3. Model of the monitoring mechanism through system logs

81

Evaluations of Online Learning Activities Based on LMS Logs

and postgraduate courses that had class sizes of 10 or above at our University. These illustrations are not meant to be a comprehensive discussion of the eLearning activities in our University. Only a few key features are discussed. We also only report up to the faculty level, rather than drilling into what happened in individual departments or even courses. The actual names of the faculties are also hidden as the interest of the present paper is not on detailed disciplinary differences. The data represent a few snapshots of some of the analyses the system allows us to do that, as will be discussed later, reveal useful information about the actual use and usage of the LMS in our University.

Popularity of LMS Across the Whole University Figure 4 shows the overall websites built on the WebCT and the Moodle platforms across three years.

The overall impression is that an increasing number of courses began to have a web presence in our LMS. The courses that had a website in one or both of our LMS grew steadily from around 42% (1432 active websites) in 2007, to 46% (1555 active websites) in 2008 and then to over 54% (1891 active websites) in 2009. More websites were built on the WebCT platform because Moodle was not introduced at CUHK until 2007. Nevertheless, the increase in Moodle use over the years is impressive.

Comparison of Two Functions What are the common functions used on these sites? Data show that the main use of these websites was for content delivery. The more interactive functions such as quizzes and discussion were used less. We illustrate this large contrast by highlighting the use of ‘active content’ and ‘active quiz’ in three years in Figure 5. Over 90% of the active websites contained some sorts of content (various types of files such

Figure 4. Usage of an LMS in CUHK courses in the academic years 2007–2009

82

Evaluations of Online Learning Activities Based on LMS Logs

Figure 5. LMS functionality used in the academic years 2007–2009

as doc, PowerPoint or PDF) which have been accessed by students in the course of study. In contrast, only a small percentage of courses had employed online quizzes as an active strategy. The percentage of quiz-containing websites actually decreased quite significantly over the years. In fact, low use was also observed for other interactive functions such as ‘active discussion’ and ‘active assignment’. On average, active-online discussions were found only in about 21.5% (2007), 20.9% (2008) and 14.8% (2009) of the total websites respectively. Also, 27.6% of the websites used the active-assignment feature in 2007. It gradually decreased to 24.0% in 2008 and then to 14.6% in 2009.

Engagement in Four Areas Figure 6 shows students’ visits to the websites and their engagement in some of the website functions in 2009. The data first of all confirmed that LMS are actively used by students. The ‘active website’ analysis showed that more than 15,000 students (over 90% of the total student population) visited one or more of the websites. Many of the students paid quite frequent visits (more than 30% of the students accessed the LMS more than 20 times

during the year). A few even visited the websites over 1000 times. However, the data also showed that students did not visit the websites for the interactive functions. Only 1995 students (recalling that over 15,000 students ever visited the LMS that year) had written anything on any of the online forums (many of these 1995 students wrote only one or two postings). Similarly, relatively few students used the assignment submission function and the quizzes. Many of the students who used the quizzes, however, had completed multiple online quizzes. 4742 students used quizzes and about one-third of them attempted 11–20 quizzes on the LMS. One-tenth even made 20 times or more attempts.

Use of LMS in Four Faculties Different faculties used LMS quite differently. ELearning experience was not the same across disciplines. Figure 7 shows the use of the two LMS in four of our eight faculties. First of all, we observed that an increasing number of courses had a course website and this was true in all of the faculties. This finding is in line with the general picture portrayed on Figure 4. However, the interest in using LMS

83

Evaluations of Online Learning Activities Based on LMS Logs

Figure 6. Frequency of usage of function within the LMS (WebCT) in 2009

varied quite a lot from one faculty to another (e.g. over 90% of courses in Faculty B versus around 50% to 60% in the other three faculties). It is also interesting to note that students were likely to be told to use two LMS because some of the teachers used WebCT while some used Moodle even if they were in the same faculty. This was particularly true in Faculty A where the preferences of WebCT and Moodle by teachers were roughly half and half. Using two different LMS and getting used to the navigation and controls of two systems might have imposed considerable confusion and unnecessary workload on students.

Use of Three Functions in Four Faculties Similar to the University-wide phenomenon as portrayed on Figure 5, using LMS for content delivery was also a common practice across the

84

four faculties inspected (Figure 8). Most of the websites (around 90% or more) in these faculties had active content that had been accessed by students in the respective years. This content-delivery focus was similar among the faculties. What makes faculties different from each other is their use of the other functions of LMS. In Faculty B, we found the use of online discussion a popular strategy (nearly half of the courses used it) in 2007 but the enthusiasm of using this function then dropped significantly (dropped to only about 5% in 2009). The interest of teachers in this faculty in the use of online quizzes also dropped to zero in the period. On the contrary, the use of online discussion in Faculty A and Faculty C courses has been relatively stable. Comparatively, teachers in Faculty D never seemed to have any great interest in online communication.

Evaluations of Online Learning Activities Based on LMS Logs

Figure 7. Use of LMS in different faculties (A–D)

Faculty C impressed us by their interest in using the quiz function. Percentage of courses having quizzes in this faculty has been well above the University mean all the years. Based on personal communications the authors have had with teachers in these faculties, there are many factors involved in explaining these observed differences. Faculty B, for example, was newly established in 2007 and then had significant expansion in teacher and student numbers in the three-year period. The faculty boosted the use of eLearning strategies in the early years but the focus was quickly shifted to other matters as more and more new teaching staff members were recruited and teaching loads became heavy. Faculty C is a professional discipline which requires students to remember and understand significant numbers of terms and information. Online quizzes seem to be a useful tool to facilitate factual learning so that classroom time can be used for activities that require higher-order thinking. Faculty A focuses on social and humanities subjects where discussion may be a more suitable online activity than quizzes. Topics in Faculty D centre on established laws and formulae and there appears to a perception that online discussions are not needed. The use of the online discussion function in the websites of this faculty has been decreasing over the years.

Engagement in Two Functions in Two Faculties Figure 9 selectively looks at two online activities in two of the faculties. It shows that, as expected, students’ engagement in content-oriented uses was similar but their involvement in other online activities could differ from one faculty to another, basically reflecting the uses of these functions as indicated in Figure 9. More than half of the websites in both Faculties A and B had 16 or more content files that were accessed by students in 2009. Websites were content-heavy and accessing of the content by students was popular in both faculties. Their use of the quiz function was, however, quite different. Websites in Faculty C obviously had a higher concentration of quizzes in them: over 10% of the websites contained 6 or more quizzes that have been actively used by students in the year. In general, thus, there is a ‘cultural’ difference in eLearning use in different faculties; perhaps not so much in using the web for content but much more when using more interactive functions.

85

Evaluations of Online Learning Activities Based on LMS Logs

Figure 8. Use of LMS functions in four faculties

REFINING OUR eLEARNING STRATEGIES Monitoring the LMS, or weblogs in our case, can achieve the objective of evaluating students’ online learning activities. By analyzing these data,

valuable guidelines in terms of eLearning design can be developed. Effective record-keeping, and extraction and interpretation of eLearning logs can reveal valuable information on standards of design and development, and program delivery.

Figure 9. Websites and students’ engagement in activities in two faculties

86

Evaluations of Online Learning Activities Based on LMS Logs

Using LMS for More than Content Delivery There are both positive and negative trends in our data concerning adoption of eLearning at CUHK. The overall impression seems to be that more courses have begun to have a web presence in our LMS. However, when we looked at these course websites closely, we did not find many functions were being used actively. Most of the websites had content for students to download. There might also be course information and announcements, but the other discussion, assignment and quiz functions were not popular (35% or below). Moreover, many of these functions did not seem to be sustainable in the sense that we observed a decrease in use of the more interactive functions such as online discussion and online quizzes over the years. Many scholars have also commented on this phenomenon that teachers used LMS mainly for delivery of content; our findings thus are more generally applicable outside our own context. Folajimi (2009) remarked that “computers aren’t fulfilling their potential to effect significant changes in education, are under-utilized, and are not being implemented in very effective or creative ways” (p. 1617). Ginsberg and McCormick (1998) also remarked that computers have only been used mostly to complement traditional teaching but are not designed as tools for effective learning activities on their own. One major drawback of existing LMS is that they are content-centric. Many teachers simply move all their teaching materials to the system. The materials are presented uniformly to all learners regardless of their background, learning styles and preferences. Modern trends in education are for learner-centric design where learners are facilitated to actively engage in the learning process to construct their personal knowledge. Teachers play the role of ‘facilitator’ who guides the learning process instead of being the sole information provider (Ong & Hawryszkiewycz, 2003). The

findings thus suggest that institutional eLearning support should not merely focus on having a web presence in courses or using the Web for courseware delivery. Attention also needs to be on the diffusion and sustained use of interactive online learning activities. Sustainability of these strategies seems to impose particular challenges in our context as the data show not only underuse of the related functions but also a general decrease of use for some of them. The question still remains about whether teachers are reluctant to interact with students on the Web in general, or whether this is only a limitation imposed by LMS. It can be an LMS problem; at CUHK, we are beginning to see a trend that teachers are interested in Web 2.0 strategies and social-media software and services. Web 2.0 or social software tools such as blogs, wikis, podcasts and media-file-sharing systems such as YouTube are supplementing or even “supplanting” (Gray, Chang & Kennedy, 2010, p. 33) the basic Web 1.0 strategies such as emails and forums in education. Users world-wide produce text and other media files for sharing and these materials can be easily adopted as teaching and learning resources elsewhere. The communications that are facilitated in social software also have the potential to extend communications to a scope far beyond the boundary of any single university. For example, Dillon, Wong and Tearle (2007) described internationalization of teaching and learning defying physical boundaries in terms of teachers and students in various places of the world teaching and learning together. We also know of a number of instances at our University where teachers have begun communicating with students through blogs and Facebook. Pituch and Lee (2006) observed that, although factors such as perceived usefulness influence LMS use, the strongest influence on student use might be system characteristics. Do these socialmedia services possess certain characteristics that make them better choices than the LMS, espe-

87

Evaluations of Online Learning Activities Based on LMS Logs

cially when implementing interactive activities? Is the LMS losing ground to these popular Web 2.0 technologies? Integration between the ‘Web 1.0’ LMS and Web 2.0 technologies seems to be needed (Boulakfouf & Zampunieris, 2008). However, teachers may simply refuse online interactions for teaching and learning in general. Levin and Arafeh (2002) suggested that a gap exists between students and teachers in the manner they use technology for teaching and learning purposes. Students use the internet for daily academic tasks, e.g. the internet as a virtual textbook and reference library, or virtual tutor, etc. However, most teachers are slower to adopt the more complex and interactive features of the Web (Morgan, 2003). If this is the case, our support should focus on motivating teachers to rethink teaching rather than just to promote functionalities of the LMS only. In a traditional, largely face-to-face university such as CUHK, blended learning (e.g. Garrison & Vaughan, 2007) offers possibilities that allow teachers to retain their traditional roles while gradually increasing the suite of online tools and strategies they use. In our context eLearning and LMS are at present supplementary tools to assist in the process of teaching and learning. What we want to achieve is that eLearning becomes more integrated into course design and thus complementary, rather than merely supplementary, to classroom teaching. Our support should not be limited to teachers only. Students may use technology in their everyday lives, but they may not be able to use the technology wisely for learning purposes. Kirkwood and Price (2005) pointed out that students may have ability with different computer applications, but only a few can transform and apply their digital competence to learning processes. For instance, students being “familiar with the use of email does not imply expertise in rigorous online debate and discussion” (p. 271). Support for meaningful eLearning thus requires professional development and support for both teachers and students.

88

The fact that students at CUHK use a number of LMS throughout their studies has led the University to plan to adopt a single LMS in the future. At the time of writing the paper, an evaluation of different LMS is underway; the imminent demise of WebCT is a facilitating factor. LMS logs have been important information in this evaluation process. For example, we understand that the future LMS should be very strong in course management and content delivery as these features are used by most teachers. However, we will also examine LMS for user-friendly interactive and communicative functions, and for convenient integration with existing Web 2.0 tools so that student-oriented eLearning strategies might be better used in the future.

Disciplinary Differences: Differentiating Support Strategies The LMS logs showed differences in the use of eLearning strategies between the faculties. This piece of information is crucial to refinement of our approaches to supporting eLearning in individual faculties. For example, we will focus on exploring the decline of interest of using certain eLearning strategies in places where the strategies do not seem to be sustained (e.g. Faculty B in our study). The support in that particular context will focus on known challenges rather than simply explaining and promoting eLearning in a traditional fashion. Another key factor is the perceived intrinsic return on effort in terms of better teaching practice and students’ learning, as well as the expected extrinsic benefit during performance appraisal. Our support to teachers in this circumstance includes both assisting them in lowering the effort in using technology, as well as showing them the usefulness of the strategies in fulfilling one or more of their needs. However, Thomas et al. (2009) interviewed eight teachers at CUHK and noted how these teachers considered innovative e-learning projects to be lonely and time-consuming. Being self-motivated and intellectually committed was a key factor that

Evaluations of Online Learning Activities Based on LMS Logs

sustained these teachers and allowed them to stay engaged. Our University is currently reviewing its strategies for assessing teaching quality and innovation in teaching will gain more prominence. This extrinsic reward should support a move to more active uses of eLearning. In disciplines where teachers seem to have a growing interest in using certain eLearning strategies, we can strengthen our support for these strategies and then also disseminate these teaching ideas to other teachers in related fields who are newer to eLearning strategies. Another structural condition that supports diffusion is the support provided by the department or by the University – in terms of monetary development grants, technical services, and consultative teaching and learning support in terms of eLearning pedagogical design. This support, though somewhat limited, still has significant impact on workload. The contents of this chapter will be presented at the University committee that considers policy for eLearning at CUHK. We know that differences in practice are also at the department level. We have devised an ‘eLearning Liaison Person’ (eLLP) strategy to better inform us the needs of individual departments. ‘eLLPs’ are representatives of individual departments who act as liaisons to inform central services of departmental needs and also act as conduits for conveying information about central services and events. To conclude, the LMS logs are useful in assisting us to continually adjust our eLearning support strategies. Some of the influences include: •

•

•

informing us the need to further investigate teachers’ attitudes and habits of using interactive e-functions; leading us to focus on meaningful uses of the functions rather than merely on explaining the functions of LMS in our promotion and support; informing us the important considerations when planning our future LMS;

•

•

informing us students’ online learning experiences which started a process to revisit support and services needed, including the discussion of a single platform; and assisting us in formulating faculty/department- specific support; eLLPs provide another source of information on disciplinespecific eLearning needs and habits.

FUTURE RESEARCH DIRECTIONS As mentioned above, LMS logs provide convenient data about web uses but are limited to showing the quantity rather than quality of these uses. Evaluating the LMS weblogs cannot provide information on the mode of online participation, interaction patterns and group dynamics. The interaction between users is of most importance when evaluating the function and effectiveness of an LMS (Gunawardena, Carabajal, & Lowe, 2001). The data highlight areas where further exploration seem to be warranted and we need other evaluation strategies (e.g. surveys and focus groups) to confirm these trends and patterns and look for the reasons behind them. One important area that is certainly worth exploring further is the observed decline in the use of communicative functions in the LMS. Is there a genuine decline of e-communication for learning or there are other software and services (such as Web2.0 technologies) that are taking teachers’ and students’ attention away from similar functions in the LMS?

CONCLUSION Effective record-keeping, and extraction and interpretation of eLearning logs can reveal valuable information on eLearning use. In universities with centralized web-based teaching and learning systems, monitoring the logs can be accomplished because most eLearning platforms have inbuilt mechanisms to track and record a certain amount of information about online activities. We reported

89

Evaluations of Online Learning Activities Based on LMS Logs

our strategy in building a system to automatically retrieve and then interpret the logs of eLearning activities in our two centralized LMS. The method has become a convenient and repeatable method for the evaluation of the richness of eLearning resources and interactions over time. In so doing, however, we have to acknowledge that the weblog analyses have limitations and should be interpreted with evidence about eLearning uses collected by other means and from other sources. We explained how the system works, and we used empirical evidence recorded from the academic years 2007 to 2009 to show how the data could be employed to achieve various levels of analyses. Weblogs represent a comparatively easy, automatic, and non-intrusive method to provide relatively quick and accurate data. We found that the logs have informed us the pattern of use of various online activities as well as the change in these activities over time. The new understanding has facilitated decisions in a number of eLearning support initiatives in our University, including the evaluation of new LMS systems for the future, and an increase in sensitivity towards faculty and departmental needs.

REFERENCES Alexander, S., & Hedberg, J. (1994). Evaluating technology-based learning: Which model? In Beattie, K., McNaught, C., & Wills, S. (Eds.), Multimedia in higher education: Designing for change in teaching and learning (pp. 233–244). Amsterdam, Netherlands: Elsevier. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11, 65–70. doi:10.1016/j. iheduc.2008.03.002

90

Boulakfouf, S., & Zampunieris, D. (2008). Integrating Web 2.0 features into a learning management system. Conference on e-Learning 2008. Retrieved December 29, 2010, from http://publications.uni.lu/record/1695/files/DZ_Pages%20 from%20ecel08-cd1.pdf Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177–192. doi:10.1080/10494820600852795 Colace, F., & De Santo, M. (2003). Evaluating online learning platforms: A case study. Proceedings of the 36th Hawaii International Conference on System Sciences. Retrieved December 29, 2010, from http://www.hicss.hawaii.edu/HICSS36/ HICSSpapers/ETWBE03.pdf Dillon, P., Wang, R., & Tearle, P. (2007). Cultural disconnection in virtual education. Pedagogy, Culture & Society, 15, 153–174. doi:10.1080/14681360701403565 Folajimi, Y. (2009). Transforming distant pedagogical learning to Web based collaborative system: An Intelligent Tutoring Systems Architecture. In T. Bastiaens et al. (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009 (pp. 1618–1622). Chesapeake, VA: AACE. Froehlich, T. J. (1994). Relevance reconsidered Towards an agenda for the 21st century: Introduction to special topic issue on relevance research. Journal of the American Society for Information Science American Society for Information Science, 45(3), 124–134. doi:10.1002/(SICI)10974571(199404)45:3<124::AID-ASI2>3.0.CO;2-8 Garrison, D. R., & Vaughan, N. D. (2007). Blended learning in higher education: Framework, principles, and guidelines. San Francisco, CA: Jossey-Bass.

Evaluations of Online Learning Activities Based on LMS Logs

Ginsberg, R., & McCormick, V. (1998). Computer use in effective schools. Journal of Staff Development, 19(1), 22–25. Grasha, T. (1990). The naturalistic approach to learning styles. College Teaching, 38(3), 106–113. Gray, K., Chang, S., & Kennedy, G. (2010). Use of social Web technologies by international and domestic undergraduate students: Implications for learning and teaching in Australian universities. Technology, Pedagogy and Education, 19(1), 31–46. doi:10.1080/14759390903579208 Gunawardena, C., Carabajal, K., & Lowe, C. A. (2001). Critical analysis of models and methods used to evaluate online learning networks. Education Resources Information Center (ERIC). Retrieved December 29, 2010, from http://eric. ed.gov/PDFS/ED456159.pdf Jovanovic, J., Devedzic, V., Gasevic, D., Hatala, M., Eap, T., Richards, G., & Brooks, C. (2007). Using Semantic Web technologies to analyze learning content. Internet computing, 11(5), 45–53. Kirkwood, A., & Price, L. (2005). Learners and learning in the 21st century: What do we know about students’ attitudes and experiences of ICT that will help us design courses? Studies in Higher Education, 30(3), 257–274. doi:10.1080/03075070500095689 Lam, P., Keing, C., McNaught, C., & Cheng, K. F. (2006). Monitoring e-learning environments through analyzing Web logs of institution-wide e-learning platforms. In L. Markauskaite, P. Goodyear, & P. Reimann (Eds.), Who’s learning? Whose technology? (pp. 429–440). Proceedings of the 23rd Annual Australian Society for Computers in Learning in Tertiary Education 2006 conference, Sydney, Australia. Retrieved December 29, 2010, from http://www.ascilite.org.au/conferences/sydney06/proceeding/pdf_papers/p62.pdf

Lam, P., McNaught, C., & Cheng, K. F. (2008). Pragmatic meta-analytic studies: learning the lessons from naturalistic evaluations of multiple cases. ALT-J, 16(2), 61–79. doi:10.1080/09687760802315879 Levin, D., & Arafeh, S. (2002). The digital disconnect: The widening gap between Internet savvy students and their schools. Pew Internet & American Life Project. Retrieved December 29, 2010, from http://www.pewinternet.org/~/ media//Files/Reports/2002/PIP_Schools_Internet_Report.pdf.pdf Mazza, R., & Milani, C. (2005). Exploring usage analysis in learning systems: Gaining insights from visualizations. 12th International Conference on Artificial Intelligence in Education (AIED 2005). The Netherlands: Amsterdam. Retrieved December 29, 2010, from http://linux3.dti.supsi. ch/~mazza/Web_area/Pubblicazioni/AIED05/ aied-ws2005.pdf Morgan, G. (2003). Faculty use of course learning systems. Educause center for applied research reports. Retrieved December 29, 2010, from http://net.educause.edu/ir/library/pdf/ecar_so/ers/ ers0302/ekf0302.pdf Ong, S. S., & Hawryszkiewycz, I. (2003). Towards personalised and collaborative learning management systems. Third IEEE International Conference on Advanced Learning Technologies (ICALT’03). Retrieved December 29, 2010, from http://www.computer.org/portal/web/csdl/ doi/10.1109/ICALT.2003.1215113 Paulsen, M. F. (2002). Online education systems: Discussion and definition of terms. Retrieved December 29, 2010, from http://www.porto.ucp. pt/open/curso/modulos/doc/Definition%20of%20 Terms.pdf

91

Evaluations of Online Learning Activities Based on LMS Logs

Pituch, K. A., & Lee, Y. K. (2006). The influence of system characteristics on eLearning use. Computers & Education, 47(2), 222–244. doi:10.1016/j. compedu.2004.10.007 Romero, C., Ventura, S., & Garcia, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51, 368–384. doi:10.1016/j.compedu.2007.05.016 Sen, A., Dacin, P. A., & Pattichis, C. (2006). Current trends in Web data analysis. Communications of the ACM, 49(11), 85–91. doi:10.1145/1167838.1167842 Thomas, K., Lam, P., & Ho, A. (2009). Knowledge diffusion in eLearning: Learner attributes and capabilities in an organization. In G. Siemens & C. Fulford, (Eds.), ED-MEDIA 2009 (pp.493–497). Proceedings of the 21st Annual World Conference on Educational Multimedia, Hypermedia and Telecommunications, Honolulu, Hawaii. (pp. 22–26). Chesapeake, VA: Association for the Advancement of Computers in Education. Zhang, H., Almeroth, K., Knight, A., Bulger, M., & Mayer, R. (2007). Moodog: Tracking students’ online learning activities. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 4415–4422). Chesapeake, VA: AACE.

ADDITIONAL READING Alavi, M., & Leidner, D. E. (2001). Review: Knowledge management and knowledge management systems: conceptual foundations and research issues. Management Information Systems Quarterly, 25(1), 107–136. doi:10.2307/3250961

92

Becker, R., & Jokivirta, L. (2007). Online learning in universities: Selected data from the 2006 Observatory survey–November 2007. Observatory on borderless higher education (OBHE) online. Retrieved December 29, 2010, from http://www. obhe.ac.uk/documents/view_details?id=15 Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Collis, B., & van der Wende, M. (2002). Models of technology and change in higher education: An international comparative survey on the current and future use of ICT in higher education. Center for Higher Education Policy Studies (CHEPS). Retrieved December 29, 2010, from http://doc. utwente.nl/44610/1/ictrapport.pdf O’ Reilly, T. (2005). What is web 2.0? Retrieved December 29, 2010, from http://www.ttivanguard. com/ttivanguard_cfmfiles/pdf/dc05/dc05session4003.pdf Piccoli, G., Ahmad, R., & Ives, B. (2001). Webbased virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. Management Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989 Selim, H. M. (2007). Critical success factors for eLearning acceptance: Confirmatory factor models. Computers & Education, 49(2), 396–413. doi:10.1016/j.compedu.2005.09.004 Wang, W. (2007). Features of future learning management system. In T. Bastiaens & S. Carliner (Eds.), Proceedings of World Conference on ELearning in Corporate, Government, Healthcare, and Higher Education (pp. 1332–1335). Chesapeake, VA: AACE.

Evaluations of Online Learning Activities Based on LMS Logs

Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75–79. doi:10.1145/986213.986216

KEY TERMS AND DEFINITIONS Active Content: Content on the website that has been accessed (meant downloading in our study) by students in the specified period of time. Active Discussion: Online forums that have been accessed (meant posting in our study) by students in the specified period of time. Active Quizzes: Online quizzes that have been accessed (meant attempting the quizzes in our study) by students in the specified period of time.

Active Websites: Websites that have recorded student access (meant viewing the front page of website in our study) in the specified period of time. Engagement: LMS logs revealing the amount of activities (e.g. frequency of use) recorded for the various functions. LMS Logs Retrieval and Reading System: An automatic system to retrieve suitable logs from an LMS server, interpret the logs and then represent them in ways that facilitate our understanding of the types and level of activities happened on the LMS. Nature of Activities: LMS logs revealing the common functions being used on the course websites. Popularity: Using LMS logs to find out how many courses opened an active website on one or more of the LMS in our study.

93

94

Chapter 5

ANGEL Mining Tyler Swanger Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock Yahoo!, USA Anthony Scime The College at Brockport, State University of New York, USA Brendan P. Post The College at Brockport, State University of New York, USA

ABSTRACT This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation.

INTRODUCTION A Learning Management System (LMS) is a course independent framework that provides, delivers, and manages instructional content, identifies and assesses learning, and tracks and records progress towards those goals. It may also provide course

registration and administration, as well as skills gap analysis, tracking, and reporting (Sclater, 2008; Watson & Watson, 2007; Paulsen, 2003). A Learning Management System generally has the same interface and features for all courses at a given school. Typically they have discussion forums, calendars, quiz capability, group work and chat spaces, and gradebooks, and perhaps

DOI: 10.4018/978-1-60960-884-2.ch005

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

ANGEL Mining

some customization capability for the instructor or individual student (Feldstein & Masson, 2006). Learning Management Systems undergo evaluation for a number of reasons. Evaluations of LMS data may be conducted to assist instructors in understanding how to enhance student learning (García, Romero, Ventura, & Calders, 2007, Romero, Ventura, & García, 2008). Schools may evaluate existing LMS options to select a particular initial choice (Sclater 2008), or when a decision is being considered on whether to replace or not the current LMS; replacement may be due to the current system no longer being supported (Sturgess & Nouwens, 2004). An investigative analysis of the usage of A New Global Environment for Learning (ANGEL), learning management system was conducted at a comprehensive 4-year college. This implementation of ANGEL has been used by the college’s faculty, staff, and students for eight years. The ANGEL Learning Management System “enables efficient and effective development, delivery and management of courses, course content and learning outcomes. Engaging communication and collaboration capabilities, enhance instruction to deliver leading edge teaching and learning” (ANGEL Learning, 2008). This investigation was prompted by the merger of the ANGEL company with a competitor, Blackboard, Inc; and Blackboard’s decision to stop support of the college’s version of ANGEL in 2012 (Blackboard, 2010). These changes present the college with the opportunity and need to assess the usage of the system. These events have forced the college to examine the need for a LMS, and if so what features are necessary in the event a new product needs to be purchased. Given these circumstances, data mining techniques were applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Data mining techniques are applicable to situations where large amounts of data exist, and the data may contain

internal relationships and patterns that characterize the data set as a whole. In the case of this study, the usage of ANGEL by students and instructors is described by the ANGEL data. This study clearly demonstrated that data mining techniques can be applied to find unknown patterns, interesting patterns, confirm assumptions, and consider statistical results for making decisions on the future of ANGEL or another LMS at a college. The data mining methods of classification, association, and clustering were applied to analyze the data. Classification produces a decision tree which predicts which courses will use the LMS system in the future, based on course specific attributes such as course type, discipline, and the number of students enrolled. Association mining discovers the usage of one feature’s effect on the usage of all other sets of features. Grouping features with similar values is the process of clustering. From the results of these analyses, metrics are formed to indicate usage of features in the LMS.

BACKGROUND There is a similarity between LMS and Knowledge Management Systems (KMS); both provide a repository for knowledge which is valuable for the user. In a KMS the knowledge is kept and used by an organization’s employees. In a LMS the purpose is to disseminate knowledge from instructors to students and to share knowledge in a way to enhance student learning (Haldane, 1998). There are also similarities between Learning Management Systems and distance learning. Distance learning uses LMS like software to provide students with learning materials and activities while tracking student activity (Falvo & Johnson, 2007). Whether the system is an LMS, a KMS, or a distance learning system the organization needs to select and implement the system best suited to their needs. Studies have been done to compare different Learning Management Systems (LMS)

95

ANGEL Mining

prior to installation or to evaluate effectiveness (Beatty & Ulasewicz, 2006). There have been many approaches to system evaluation and selection. Traditional systems requirements analysis has been used to evaluate and select LMS. Functionality desired by faculty has been achieved through requirements analysis followed by system selection and modification (Sclater, 2008) and functionality has been achieved through the use of design patterns (Avgeriou et al., 2003). Studies have been done to determine frameworks for evaluating LMS. Kim and Lee (2008) validated a model for evaluating Learning Management Systems (LMS) used in e-learning fields using exploratory factor analysis. Factor I was instruction management, screen design, and technology and factor II covered interaction and evaluation. Roqueta (2008) used Moore’s (1993) transactional distance theory, the diffusion of innovations theory (Rogers, 2003), and Malikowski’s (2007) model for evaluation of learning systems to conclude that there is a difference between learning management systems and course management systems, and that LMS are preferable in most cases. A commercial tool for evaluation has been developed by 3Waynet Inc. The LMS Evaluation Tool is spreadsheet like and designed to assist in selecting a LMS. With this LMS Evaluation Tool users specify criteria for evaluation, which are considerations that are important to the institution adopting the LMS, these criteria are cost of ownership, maintainability and ease of maintenance, usability, ease of use, user documentation, user adoption/ vendor profile, openness, standards compliancy, integration capacity, learning object metadata integration, reliability and effectiveness, scalability, security, hardware and software considerations, and multilingual support. Each criterion is expanded with relevant questions to be answered. The candidate LMS undergoing evaluation are also rated on their features. The features considered

96

are administration, security, access, integration with other systems, course design, development and integration, course monitoring, assessment design, online collaboration and communications, and productivity tools. These features each have a number of sub-features that are given weights to measure their relative importance. The user rates each candidate LMS on each criterion and feature. The evaluation tool computes an overall score for each LMS in each criterion (Commonwealth of Learning, 2004). Additionally, the independent research company Brandon Hall Research, has completed a study of over 90 LMS with custom comparisons across more than 200 features. The report of this study is available on the World Wide Web in an interactive format allowing users to specify their requirements and complete side-by-side comparisons of selected LMS (Chapman, 2010). Studies have shown that most evaluations of LMS look toward the technology of the system. Specifically, evaluations are done on communication, education management, and file management features and how effectively these features are implemented in the technology. Hall (2003) provides a list of technically oriented considerations in evaluating learning management systems. Lewis, et al. (2005) evaluated nine LMS by considering and comparing the features and capabilities of the systems in terms of content development, group work and participation, the calendar, communication capability, study tools available to students, handling of audio and video, monitoring of student participation and progress, and usability in terms of navigation, interface design and administration. Evaluations often done on the technology itself are conducted by experts and not participants. Feldstein and Masson (2006) outline features that teaching professionals, as well as technical staff, should consider when selecting an LMS. Specifically they suggest looking for the ability to customize the LMS to meet the pedagogical methods of individual instructors, which may vary

ANGEL Mining

by discipline, instructor, and learning styles of individual students. Sturgess and Nouwens (2004) report on a different approach where participation in the evaluation includes an understanding of the organizational culture and sub-cultures. Specifically they identified academics, information technology staff, multimedia development specialists and managers/administrators, as participant groups in selecting a new learning management system. Evaluation of student use of LMS found that the LMS needs to be integrated with other educational activities and other aspects of everyday life. This is more important than the interface design. Students’ idea of what an LMS should provide differs from the impression of the technical staff and other stakeholders who select and implement the system. While students effectively use an LMS remotely this is only undertaken when there is a practical advantage to the student, not just for some educational benefit (Alsop & Tompsett, 2002). Student learning style and its impact on learning management systems is investigated by Graf and Kinshuk (2006). Bayesian Networks have been used to discover user preferences to allow adaptation by the LMS to those preferences (Kritikou, Demestichas, Adamopoulou, Demestichas, Theologou, & Paradia, 2008). Instructors can use data mining techniques such as classification, association and clustering on student LMS data to improve instruction. Clustering techniques can find groups of similar students so that the instructor can direct instruction to the group’s particular needs classification will identify the characteristics of the students in each group. Association mining may discover relationships between these characteristics and student attributes (Romero, Ventura, & García, 2008). There are a number of data mining techniques and associated algorithms. A common technique shared with statistics is classification analysis. A decision tree model is constructed which finds a path to a predetermined class attribute for each data record. A classification decision tree contains

branches that can be converted to rules unique to the data set. Machine learning research produced a number of classification models, the best known of which is the C4.5 algorithm (Quinlan, 1993), which uses information gain to define the model’s structure. A product of machine learning research, association mining is used to find patterns of data where sets of attribute-value pairs occur frequently in the data set. With association mining what class attribute should be in the results is not predetermined. Apriori (Agrawal, Imieliński, & Swami, 1993) is the predominate association mining algorithm. It is an algorithm that produces many rules, and special techniques are needed to reduce the rule set to those that are interesting. The final technique used in this chapter is clustering. Clustering represents each attribute as a dimension and shows where data records occur in this multidimensional problem space. The most popular clustering algorithm is k-means (MacQueen, 1967). Again, analysis of the clusters needs special techniques. In an attempt to increase the robustness and reliability of rules, often a combination of data mining methodologies are applied. Deshpande and Karypis (2002) and Padmanabhan and Tuzhilin (2000) improved classification rules by first using association mining. Li, Han, and Pei (2001) classified records using CMAR (Classification Based on Multiple Class-Association Rules), which identifies frequent patterns and associations between records and attributes. Jaroszewicz and Simovici (2004) employed user background knowledge and a Bayesian Network to determine the interestingness of sets of attributes prior to association mining. Fu and Wang (2005) reduced data dimensionality using a separability-correlation measure that ranks the importance of attributes to improve classification and the usefulness of rules. Scime and Murray (2007) and Murray, Riley, and Scime (2007) used expert knowledge to reduce data dimensionality while iteratively creating classification models. Rajasethupathy, Scime,

97

ANGEL Mining

Rajasethupathy, and Murray (2009) improved the usefulness of rules by identifying “persistent rules,” which are those rules identified by both classification and association mining.

DATA MINING ANGEL DATA Data mining is a process of inductively analyzing data to find interesting patterns and previously unknown relationships in the data. Data mining is used not only to predict the outcome of a future event but also to provide knowledge about the structure and interrelationships among the data. The data mining process identifies relationships that are expressed as classification rules, association rules, and clusters of records. Data mining algorithms are used to create models that describe existing data and relationships within the data. These methodologies create rules used to analyze new data and predict future outcomes. The data set used in this analysis contains information across two years. Data is recorded in the ANGEL data base for every class offered during Fall, Spring, and Summer for every feature. This data consisted of class specific data, e.g. semester, department, course number, and student count, and feature data, e.g. feature (files, messages, folders etc.) and frequency of feature usage. The data set was processed using classification, association, and clustering algorithms as implemented by the Waikato Environment for Knowledge Analysis (WEKA, Witten and Frank, 2005). The intent of this study is not to compare ANGEL to other LMS but rather to assess the usage of the ANGEL features by students and instructors. Basing this study on recent historical data provides a good indication of the expected ANGEL use in the near term future. The college’s course offering and student and faculty profiles are not expected to change in the next few years. Classification mining determines the influence of combinations of attributes on a specific goal. In the case of this ANGEL data, multiple classification

98

models determine the usage of each feature based on student count, department, level, and semester. Association mining finds patterns in the data that reoccur frequently. This is an effective method for finding features that co-occur. Association rule mining was preformed to find the relationships between the features of ANGEL. This could lead to acceptance or rejection of sets of features. Clustering finds natural clusters occurring in data. These multi-dimensional clusters divide the data into groups; sufficiently large groups indicate the most important features in the data. In the ANGEL data a large cluster will indicate the most commonly used features. The WEKA data mining tool is used to apply the data mining techniques, and Microsoft’s Excel was used to prepare the data.

Preprocessing Data often contains attributes and records that are not pertinent to the analysis being conducted. In the case of the ANGEL data set, it consisted of records from all classes offered by the college for the Fall and Spring semesters from Fall of 2007 to Spring of 2009. For each of these classes the feature usage was collected. This original data set contained 9430 records, one for each class section, and 20 attributes (features). From this data set records for Thesis Continuation Credit (TCC) classes were removed, because TCC classes are an extension of a previous thesis course but were not identified by department. The information gained from these sections would have been inconclusive since the original class is not identified. The General Education Program (GEP) class records with sections of 0 or 99 were also removed. These sections are used to place all incoming transfer students in the computer skills exam, passing the exam is a graduation requirement, but an actual class does not exist. Five attributes were removed from the data set. The attribute identifying the section number was not relevant to the scope of our analysis. Determining the difference between section 1

ANGEL Mining

and section 2 is trivial. The attributes indicating the use of wikis, games, blogs, and assessments features were removed due to their extremely low usage. For example, assessments, which posted the highest usage of the four, was only used by 64 classes or .68% of total classes. After removal of the records and attributes the data set was reduced to 9312 department-course level- semester records with 15 attributes (Table 1). Some attributes had their values adjusted. The course number was changed to a nominal Level number, for example 436 became L400, Semester was changed to a nominal value, for example 200709 became S200709 for the Fall semester 2007. Other attributes had values for the number of times the feature was used in the course. For all empty attribute values a zero (0) was inserted, because the absence of a value meant the feature was used zero times for that class. These adjustments created the base data set. The data mining algorithms required slight variations from this base data set. For classification and association mining a nominal data set was needed. The numerical feature data was

discreteized to nominal data using six bins with equal depth binning. This method creates six bins for each attribute where the number of records in each bin is kept close to equal. Nominal data for student count used the college’s Common Data Set from 2008-2009. This document binned student count by the ranges 2-9 (very low), 10-19 (low), 20-29(medium low), 30-39 (medium), 40-49 (medium high), 50-99 (high), 100+ (very high). Classes of a single student are not specified in the Common Data Set, therefore they were assigned a value of ‘single’. Preprocessing resulted in a data set characterized by department-course level-semester records with binned counts of feature usage. Beyond this data preparation, preprocessing done on the data set is specific to the data mining method.

Classification Using Decision Trees The goal of classification of the ANGEL data is to examine the influence of semester, department, level, and student count on each feature. This requires classifying the data with respect to each

Table 1. Attributes Name

Description

Department

The academic department of the course

Drop Box

Number of drop boxes used to electronically collect assignments

Files

Count of files available for download

Folders

Number of folders used to organize content

Form

Number of surveys to be completed by students

Forum

Number of online discussions

Grade Book

Number of grade books (usually one)

Links

Number of URLs to resources outside the LMS

Level

The level of the course (e.g. 100, 200, 300, etc.)

Messages

Number of emails sent from within the course

Pages

Number of content pages available for students to view

Quizzes

Number of LMS administered tests

Semester

The semester in which the section was taught

Student Count

Number of students in the section

TurnItIn

Number of assignments collected and evaluated for originality

99

ANGEL Mining

feature individually. From the data set a feature specific data set was created consisting of semester, department, level, student count and one feature. These data sets were processed individually to create feature specific decision trees. Decision tree analysis creates a tree structure, where each internal node denotes a test on an attribute, each branch represents an outcome of the test, and every leaf represents a class or class distribution (Bagui, 2006). Decision tree analysis allows for easy conversion to classification rules. A decision tree starts as a single node. If all records in that node are the same it becomes a leaf. If they are not all the same, a selection algorithm uses entropy, a measure of the inconsistency of the data, to decide how to divide the records. An attribute is chosen that best divides the records into further purer nodes. The attribute chosen is labeled as the decision attribute. Branches are then created for the known values of the decision attribute, and the records are divided accordingly into the child nodes. This process is performed Figure 1. Generic decision tree

100

recursively on all child nodes (Bagui, 2006). For example, if given a data set consisting of both boys and girls and their favorite sport, using entropy the algorithm selects gender as the first decision attribute producing two child nodes. The data set records are divided at this node. Each subset undergoes the same algorithmic process at the individual gender nodes to further divide the records. This process continues until only pure nodes occur, creating leaves. An example of a completed classification decision tree is presented in Figure 1. This generic decision tree has four nodes or points at which decisions are made. A record would be classified depending on the values of its attributes into one of the leaf nodes, where the class attribute has a specific value. In this example the possible class attribute values are w, x, y, and z. A record with the attribute values (Attr1 = a then Attr2 = 4) would be classified as the class attribute with the value x. This record is classified by following the Attr1- Attr2 edge of the tree and then the

ANGEL Mining

Attr2-ClassAttr=x edge, reaching a ClassAttr=x leaf. Other records may also reach a leaf with ClassAttr=x (e.g., Attr1 = c then Attr3 = g then Attr4 = s), but via a different path. After the decision tree is constructed, each branch of the decision tree is converted into a rule. The tree presented in Figure 1 can be converted into the following Rules: • • • • • •

Rule 1: IF Attr1 = a AND Attr2 ≤ 5 THEN ClassAttr = x Rule 2: IF Attr1 = a AND Attr2 > 5 THEN ClassAttr = y Rule 3: IF Attr1 = b THEN ClassAttr = z Rule 4: IF Attr1 = c AND Attr3 = g AND Attr4 = s THEN ClassAttr = x Rule 5: IF Attr1 = c AND Attr3 = g AND Attr4 = t THEN ClassAttr = w Rule 6: IF Attr3 = h THEN ClassAttr = y

The rules provide insight into how the class attribute’s value is, in fact, dependent on the other attributes. A complete decision tree provides for all possible combinations of the attributes and their allowable values reaching a single, allowable class attribute value. The decision tree algorithm C4.5 builds the decision tree from a training data set using information entropy (Quinlan, 1993). The goal of the algorithm is to classify all the training set records according to the goal or class attribute. The algorithm first determines the attributes available in the first node. The normalized information gain with respect to the class attribute is calculated for each attribute to determine how the records should be split into subsets. The attribute with the highest normalized information gain is selected as the decision attribute. The records are then broken into branches on the values of the decision attribute, creating branches of the tree ending in child nodes. This process is performed recursively (Quinlan, 1993). The class attribute and its values constitute the leaf nodes of the decision tree.

Information gain is the change in information entropy from the current state of the set of records to the proposed state of the set of records. Entropy is a measure of the randomness of the distribution of records in a subset of records with respect to the class attribute. The attribute with the greatest information gain is selected as the node for dividing the data in the decision tree at each node. The information gain with the overall highest gain with respect to the dependent attribute is the root of the tree. The confidence that a record is correctly classified provides the accuracy of splitting the record set. Confidence is calculated by dividing the number of records correctly classified by the total number of records that are classified at the node. Confidence is used to prune the decision tree, ensuring a tree with a reasonable number of leaf nodes and acceptable accuracy. The level of pruning is controlled by setting a minimum confidence level. If a node falls below the minimum confidence that branch is pruned; that is, the records are rolled up to the parent node until the minimum confidence is met This allows for only rules that meet the required accuracy to remain on the final tree. The data set was divided into training and test sets, where the training set contained two-thirds of the records, randomly selected. The remaining records are in the test set. The decision tree was constructed using the training set and validated with the test set. To determine the appropriate confidence level multiple trees were constructed using individual features as the class attribute. Evaluation of multiple decision trees is common in classification problems (Osei-Bryson, 2004). After several trials of varying confidence, 55% confidence was selected. This choice was determined after trials focusing on minimizing the percent incorrectly classified, while maintaining quality leaf nodes. However, the direct relationship of these features needed to be taken into account. As the number of leaf nodes decreases, the num-

101

ANGEL Mining

ber of incorrectly classified records decreases, as well as the quality of rules formed from the nodes decreasing. The goal of classification of ANGEL data is to examine the influence of semester, department, level, and student count on a feature. To accomplish this task, our fully nominal data set was broken into eleven data sets, which in turn were divided into training and test sets. Each data set consisted of semester, department, level, student count and one feature, each having a different feature. For example, semester, department, level, student count, and file. Dividing the data set into these smaller data sets allowed each feature to be a class attribute. Records with zero values for the class (feature) attribute were removed from the respective feature data set. The C4.5 algorithm was executed on each data set with a confidence level of 55%. The decision tree is converted into IF-THEN rules. Each rule represents one branch of the decision tree from the root node to a leaf node. The intermediate nodes provide additional information in the form of branching on the branches. There is one rule for each of the leaf nodes of the tree. The results of the classifications followed a pattern. For the top five features (messages, files, folders, grade book, and links), the attribute with the highest information gain was student count followed by department. From this several rules were formed that concluded that as class size increased the usage of the feature increased. Figure 2 provides a sample of the rules found. For the little used features (drop box, form, forum, page, quiz and turn it in) the attribute with the highest information gain was first department followed by level. From this classification a definite conclusion pertaining to all classes cannot be made. Several departments have lower level classes with heavier users, while other departments have higher level classes with the heavier users (See Figure 3 for rule examples). From classification it is found that features with a less traditional online use such as, admin-

102

istering a test or having students turn in a paper, have usage that is directed more by the department and level than the class size. This suggests users who are more willing to experiment with the new technology create the demand. Those users then have an effect on their colleague’s use of that feature. Whereas, with the more commonly used online features, such as e-mail messages or file sharing, usage is directed by need. This suggests that the more students in a class, the more likely a professor is to post files rather than print the files to hand out in class. These findings support Nichols’(2003) hypothesis that the movement to eLearning, including the use of LMS, is an evolutionary, not revolutionary process. As success is seen in one course where an LMS is used the LMS will be adopted in other, similar courses.

Association Rule Mining The goal of association mining the ANGEL data was to evaluate each feature’s affect on the usage of the other features. This required modification to the original data set. Non-feature attributes were removed so that patterns of feature usage would show the use of one feature with another in the association rules. Association rule mining, unlike classification rule mining, does not need a class attribute in order to find meaningful results. Association mining allows for rules to be generated for any combination of attributes within the data set. Association Rules are not intended to be used together (Witten and Frank, 2005). Each rule generated in association mining implies a different regularity within the data set, and each could result in a very different conclusion. With the ANGEL data, relationships are found between the different features used for a class on campus. To construct association rules itemsets found in the data set are used. An itemset is a collection of attribute-value pairs (items) that occurs in the data set. Each itemset can be converted into a

ANGEL Mining

Figure 2. Sample classification rules of top features

number of rules, where each item in an itemset implies and is implied by every other item and combination of items in the itemset. This results in a very large number of rules. Since association rule mining results in a very high number of rules, restrictions need to be applied to specify whether a rule is significant and valid. The metrics of support and confidence are used to determine the significance of a rule. The support of an itemset is the count or percent of records that contain all the items in the itemset. A support threshold is set which must be met or exceeded for a itemset to be considered a frequent itemset. In the example above the support count threshold was set to two.

Rules that come from the same itemset all have the same support value. A rule contains a premise (IF-part) and consequent (THEN-part) and states that when the premise is true the consequent will be true. The confidence that this rule is correct is also calculated. The confidence is a conditional probability that a record containing the premise will also contain the consequent. It is calculated by dividing the support for the rule by the support just for the itemset that is the same as the premise. Only those rules meeting the user-specified confidence threshold are kept. Each rule is an observation of the data’s behavior. The algorithm chosen for this investigation is the Apriori algorithm. The Apriori algorithm uses itemsets to generate rules, and support based

103

ANGEL Mining

Figure 3. Sample classification rules for forums

pruning to help control the growth of rules. It also uses the Apriori Principle. This principle states that if an itemset is frequent, then all of its subsets must also be frequent (Agrawal, Imieliński, & Swami, 1993). The Apriori algorithm first considers the single itemsets, those with one only item and counts how frequently the item appears in the data. The itemsets that do not meet the minimum support threshold are discarded from the possible one itemsets. The Apriori Principle ensures that all supersets of the one itemsets that are infrequent, are also infrequent. A list of two itemsets is generated from the list of one itemsets. The frequency of the two itemsets is determined and those not meeting the minimum support threshold are removed. Three itemsets are created from the two itemsets and the process continues until no more frequent itemsets can be found. Once all of the frequent itemsets are generated, the rules are generated. If a rule does not meet the minimum confidence

104

level, it gets discarded as well as any rule that is a subset of that rule. The goal of association mining the ANGEL data was to evaluate a feature’s affect on the usage of one or more other features. To create the data set necessary to achieve this goal several attributes (semester, department, level and student count) were removed from the original data set. These attributes were removed to generate only rules pertaining to the use of one feature’s influence on another. For example, the use of folders indicates the use of files. The minimum value for support was set to 0.7, the value for confidence was set to 0.9, and the number of rules was set at 3000. Support and confidence were set at such high values to ensure presentation of rules with certainty. The number of rules was set so as to be exhaustive and find all possible rules within the bounds of support and confidence. The algorithm generated 2,638 rules.

ANGEL Mining

The useful rules can be defined by applying a template (Klemettinen et al. 1994). Only those rules where the predicate consisted of a single feature were kept. The rules then had the appearance of, if attribute X is used, then these other attributes are used. These rules show how using one feature of the ANGEL system influences usage of other features. However, only rules where the feature count was zero were found. Nevertheless, this still is an interesting, and meaningful result. The rules now have the form, if you do not use this feature, then you will not use these other features. In order to reduce the number of rules further, if there existed a rule X where the consequence was a subset of the consequence of rule Y, then X was removed (Rajasethupathy, Scime, Rajasethupathy, & Murray, 2009). After the filtering of rules, 37 rules remained. (Appendix A). Inspecting the rules from association rule mining leads to a few general conclusions. Given the level of confidence and support in this study, the use, or non-use, of email messages or files uploaded to ANGEL have no affect on the other features. There is no correlation between those two features and any of the others. One might assume that if a particular class on ANGEL uploads a large number of files that this high frequency of usage could be seen with other features, however this investigation did not find that to be the case. Another conclusion is that if a class does not use the drop box feature, then they will not use any feature, or any combination of features. When rules were being purged, there existed only one rule with drop box in the predicate. This rule can be used as a metric for ANGEL usage. (Note: it is not logically correct to say that if they do use the drop box feature then they will use other features.) At the theoretical level, García, Romero, Ventura, and Calders (2007) determined that in association mining critical factors in determining usage are the number of messages, emails, documents, and Web pages on the course site. However, the findings of this study indicate that

the number of messages and emails do not influence the feature usage. In association mining student Moodle activity in a course, Romero, Ventura, and García (2008) found that students that do not send messages (or email) do not read them either, which then leads to course failure. While there is no relationship between using the message components of ANGEL clearly use of a LMS features, while not ensuring a passing grade, does mitigate the likelihood of failure.

Cluster Rule Mining The goal in clustering the ANGEL data set was to provide an indication of the most used features in larger classes and therefore by the most students. Additionally, indications of infrequent use of some features, while identifying where these features are used, is found in smaller clusters. In conventional terms, clustering is when you group similar objects together. In data mining, this definition still holds. Clustering in data mining is taking similar records and grouping them together into clusters based on a measurable distance between them. Once these clusters are created, similarities can be determined within the data set that may not have been previously apparent. In clustering each attribute is a dimension in the problem space. Each record is placed as a point in the problem space based on the values of the record’s attributes. The process of clustering is based on comparing the records, calculating the distance between them, and then grouping like records together. The collection of clusters (known as a clustering) has clusters consisting of records that are very similar to records inside the cluster, but dissimilar to those within other clusters. The goal of clustering is to indentify common feature groupings found in ANGEL, as well as, the count of students associated with them. The algorithm used for clustering for this investigation is k-means. Since clustering is graphical the data must be fully numeric. Therefore, the

105

ANGEL Mining

pre-discretized data set was used. This data set contained numerical data for feature usages and student count. The nominal attributes (semester, department, and level) were removed. The k-means algorithm requires an input for the number of clusters (k). The algorithm randomly selects k points to be the initial cluster centroids. The records are formed into these k clusters based on to which centroid a given record is closest, closeness being determined by the distance measure. The centroid of each cluster is recalculated using the records in the cluster. The distance from each record to each centroid is recalculated and records may change clusters. This process continues until the difference between the current centroid and the previous centroid is zero or near zero for each cluster. The value of k determines the number of clusters, varying k results in different clusterings or grouping of the records. To determine the optimal clustering the sum of squared errors (SSE) are compared to the number of clusters. As the number of clusters increases the SSE should decrease, when the SSE begins to remain nearly constant the smallest number of clusters in typically chosen. This process was performed on the ANGEL data set. Fifteen clusterings were computed, increasing k by two from two to thirty. For each analysis of k the SSE was recorded and graphed (Figure 4). From the graph, 23 clusters were chosen for cluster rule mining. This cluster size was chosen as it is the point where the graph levels out. The area of the graph where leveling occurs is considered to be optimal for cluster size because once the graph begins to level off the measure of SSE from one k to the following k+2 is less meaningful. Therefore, the analysis is producing sub optimal clusters, or with less meaningful information than their predecessor. For example, in our data set using twenty-six clusters would not produce better results. It would be more likely to divide good clusters.

106

The goal in clustering the data set was to find natural feature groupings in the data set and the class size associated with that grouping. In the 23 cluster clustering, five clusters were formed that contained 72% of the records, and revealed files and messages as the only features used by those clusters. Furthermore, clustering revealed seven clusters totaling 3.7% of records, that used nine to ten of eleven features. Lastly, the largest cluster holding 24% of records contained an average student count of 1.78 and average of 0 to 0.45 for feature use. That cluster formed the rule that if a class has two or less students they will not use ANGEL. The results received by clustering show that the most commonly used features in ANGEL are files and messages. Most classes do not use the more advanced features in ANGEL, yet there is a small subset of instructors that heavily use advanced features. Finally, classes with two or less students do not use any feature in ANGEL; this may imply that it is not worth the instructor’s time to create content for just a few students. Romero, Ventura, and García’s (2008) study of types of Moodle students found three clusters defining types of students: very active, active, and non-active. Very active students are characterized as having sent more than one message, read about three messages, passed a large number of quizzes, and spent time on Moodle doing assigned work. Non-active students did no assignments, read few messages, completed very few quizzes, and did not spend time on Moodle. Active student’s behavior was between the other two groups. While messages are one of the most commonly used features in ANGEL, it is up to the individual student to define themselves as very active, active, or non-active.

FUTURE RESEARCH DIRECTIONS Today, the use of the computer has become common for statistical analysis of data. Software packages are easy to use, inexpensive, and fast. But

ANGEL Mining

Figure 4. Number of clusters vs. SSE

today’s vast stores of data with immense data sets make comprehensive analysis all but impossible using conventional techniques. A solution is data mining. Classification mining finds the rules that can predict future behavior. Association mining discovers patterns in the data. Clustering provides insight into the data. Currently data mining requires some specialized skill – the ability to understand the processes, the data, and expertise in the domain. In the case of Learning Management Systems, how the system operates and instruction techniques may be necessary to understand the results. This combination of skills is difficult to find in a single person. In the future data mining tools will become easier to use, as statistical and spreadsheet tools are today. Then domain experts will be able to apply data mining to their data without the need for a data mining expert. The rules found and conclusions drawn can then be used with more confidence when making decisions within the domain. Data mining is commonly conducted against transactional data, but data have gone beyond simple numeric and character flat file data. Future LMS data will come in many forms: image, video, audio, streaming video, and combinations

of data types. Data mining research is being conducted to find interesting patterns in data sets of all these data types. Beyond the LMS data, other data sources may supplement the LMS data. Data from corporate and government data warehouses, transactional databases, and the World Wide Web (Scime, 2008) may be added to enhance the LMS. The data mining of an LMS of the future will be multidimensional, accessing all these data sources and data types to find the optimal use of a school’s LMS. In addition to the new forms of data that will need to be mined in future iterations of the LMS, the very nature of the LMS is likely to see significant changes in approaching years. As colleges and universities become more attune to the needs of both regional (Middle States) and program accreditation (NCATE, AACSB, ABET) the usage of the LMS for purely learning or course management activities is likely to transform to put greater emphasis on assessment and accreditation activities. Where the use of specific system features has not been traditionally mandated, it is likely that there could be a shift to the mandated use of features such as rubrics and assessment tools.

107

ANGEL Mining

With this potential shift comes the opportunity for data mining to expand beyond evaluating just the usage of the LMS but to also provide deep insights into the performance of students against institutional and program objectives. Correlating the use of LMS features with student performance could provide the opportunity to enhance the use of the LMS to provide stronger educational outcomes. This in turn, could provide guidance in improving college and university programs to produce stronger and more competitive classes of graduates. Finding more interesting and supportive rules for a domain using multiple methods places constraints on the mining process. This is a form of constraint-based data mining. Constraint-based data mining uses constraints to guide the process. Constraints that can be used can specify the data mining algorithm. Constraints can be placed on the type of knowledge that is to be found or the data to be mined. Dimension-level constraints research is needed to determine what level of a summary, or the reverse, detail is needed in the data before the algorithms are applied (Hsu, 2002). Research in data mining will continue to find new methods to determine interestingness. Research is needed to determine what values of a particular attribute are considered to be especially interesting in the data and in the resulting rule set (Hsu, 2002). Currently there are 21 different statistically based objective measures for determining interestingness (Tan, Steinbach & Kumar, 2006). A leading area of research is to find new, increasingly effective measures. With regard to subjective measures of interestingness, research in domains that is both quantitative and qualitative can lead to new methods for determining interestingness. Further data mining research will find new methods to support existing knowledge and perhaps find new knowledge in domains where it has not yet been applied (Scime, Murray & Hunter, 2010).

108

CONCLUSION In this investigation, statistical methods were used to determine whether or not features were being used in the ANGEL system. Data mining was able to generate results, and from these results conclusions were made from the three different data mining techniques; classification, association, and clustering. A few general conclusions can be made, the majority of classes at this college do not use the more advanced features of ANGEL. The only parts of the system being used by a large number of classes are the messaging and file sharing features. Another conclusion is that there exists a relationship between need and use. If there are a large number of students, then using ANGEL to send messages or share files alleviates the instructor from a certain amount of additional effort. For example, instead of printing off enough copies for the class, the file is shared on ANGEL. Whereas, classes with a low student count do not use this feature as frequently. Association suggests that there is not a relationship between the use of one feature to the use of another feature. However, association does conclude the non-use of features implies the non-use of other features. Furthermore, during pre-processing, it was found that there are features that are not being used at all (wikis, blogs, assessments, and games). Nevertheless, it has been found that there are a small number of instructors who are heavy ANGEL users. These users often utilize many of the features available. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation or to identify feature areas needing additional training in their use.

ANGEL Mining

REFERENCES Agrawal, R., Imieliński, T., & Swami, A. (1993). Mining association rules between sets of items in large databases, Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington, DC. (pp.207-216). Alsop, G., & Tompsett, C. (2002). Grounded theory as an approach to studying students’ uses of learning management systems. ALT-J Research in Learning Technology, 10(2), 63–76. doi:10.1080/0968776020100207 Avgeriou, P., Papasalouros, A., Retalis, S., & Skordalakis, M. (2003). Towards a pattern language for learning management systems. Journal of Educational Technology & Society, 6(2), 11–24. Bagui, S. (2006). An approach to mining crime patterns. International Journal of Data Warehousing and Mining, 2(1), 50–80. doi:10.4018/ jdwm.2006010103 Beatty, B., & Ulasewicz, C. (2006). Faculty perspectives on moving from blackboard to the moodle learning management system. TechTrends, 50(4), 36–45. doi:10.1007/s11528-006-0036-y Blackboard (2010). Blackboardlearn+ ANGEL edition. Retrieved on April 19, 2010, from http:// www.blackboard.com/Teaching-Learning/LearnResources/ANGEL-Edition.aspx Chapman, B. (2010). LMS KnowledgeBase 2010: In-depth profiles of 90+ learning management systems, with custom comparison across 200+ features. Brandon Hall Research, Retrieved on November 9, 2010, from http://www.brandon-hall. com/publications/lmskb/ lmskb.shtml Commonwealth of Learning (2004) LMS Evaluation Tool User Guide, 1.2, 3waynet Inc. and the Commonwealth of Learning.

Deshpande, M., & Karypis, G. (2002). Using conjunction of attribute values for classification. Proceedings of the 11th International Conference on Information and Knowledge Management, McLean, VA (pp. 356-364). Falvo, D. A., & Johnson, B. F. (2007). The use of learning management systems in the United States. TechTrends, 15(2), 40–45. Feldstein, M., & Masson, P. (2006). Unbolting the chairs: Making learning management systems more flexible. E-learn magazine. Retrieved from http://www.elearnmag.org/subpage.cfm?section= tutorials&article=22-1 on April 19, 2010. Fu, X., & Wang, L. (2005). Data dimensionality reduction with application to improving classification performance and explaining concepts of data sets. International Journal of Business Intelligence and Data mining, 1(1), 65-87. García, E., Romero, C., Ventura, S., & Calders, T. (2007). Drawbacks and solutions of applying Association Rule mining in learning management systems. Proceedings of the International Workshop on Applying Data mining in e-Learning, Crete, Greece (pp. 13-22). Graf, S., & Kinshuk (2006). Considering learning styles in learning management systems: Investigating the behavior of students in an online course. Proceedings of the 1st IEEE International Workshop on Semantic Media Adaptation and Personalization, Athens, Greece (pp. 25-30). Haldane, A. (1998). The convergence of e-learning and knowledge management: Fusion or confusion? People development in a knowledge economy, 2. Learning Futures Ltd., Retrieved on April 26, 2010 from http://www.learningfutures.co.uk/LFdocs/ MKelleher_06-01-03_15-29-27.pdf. Hall, J. (2003). Assessing learning management systems. Chief learning officer. Retrieved on April 19, 2010, from http://www.clomedia.com/ features/2003/January/91/index.php.

109

ANGEL Mining

Hsu, J. (2002). Data mining trends and developments: The key data mining technologies and applications for the 21st century, Proceedings of the19th Annual Conference for Information Systems Education (ISECON 2002), San Antonio, TX (Art 224b). Jaroszewicz, S., & Simovici, D. A. (2004). Interestingness of frequent itemsets using Bayesian networks as background knowledge. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (pp. 178-186). Seattle, WA. Kim, S. W., & Lee, M. G. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24(4), 284–294. doi:10.1111/j.13652729.2007.00260.x Klemettinen, M., Mannila, H., Ronkainen, P., Toivonen, H., & Verkamo, A. I. (1994). Finding interesting rules from large sets of discovered association rules. Proceedings of the 3rd International Conference on Information and Knowledge Management (CIKM’94) Gaithersburg, Maryland, USA (pp. 401-408). Kritikou, Y., Demestichas, P., Adamopoulou, E., Demestichas, K., Theologou, M., & Paradia, M. (2008). User profile modeling in the context of Web-based learning management systems. Journal of Network and Computer Applications, 31(4), 603–627. doi:10.1016/j.jnca.2007.11.006 ANGEL Learning (2008). The ANGEL learning management suite. Lewis, B. A., MacEntee, V. M., DeLaCruz, S., Englander, C., Jeffrey, T., & Takach, E. …Jason W. (2005). Learning management systems comparison. Proceedings of the 2005 Informing Science and IT Education Joint Conference, Flagstaff, Arizona.

110

Li, W., Han, J., & Pei, J. (2001). CMAR: Accurate and efficient classification based on multiple class-Association rules. Proceedings of the 2001 IEEE International Conference on Data mining (pp. 369-376). MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability (pp. 281-297). Malikowski, S. R., Thompson, M. E., & Theis, J. G. (2007). A model for research into course management systems: Bridging technology and learning theory. Journal of Educational Computing Research, 36(2), 149–173. doi:10.2190/10021T50-27G2-H3V7 Moore, M. G. (1993). Theory of transactional distance. In Keegan, D. (Ed.), Theoretical principles of distance education (pp. 33–38). New York, NY: Routledge. Murray, G. R., Riley, C., & Scime, A. (2007, May). A new age solution for an age-old problem: Mining data for likely voters. Paper presented at the 62nd Annual Conference of the American Association of Public Opinion Research, Anaheim, CA. Nichols, M. (2003). A theory for elearning. Journal of Educational Technology & Society, 6(2), 1–10. Osei-Bryson, K.-M. (2004). Evaluation of decision trees: A multicriteria approach. Computers & Operations Research, 31(11), 1933–1945. doi:10.1016/S0305-0548(03)00156-4 Padmanabhan, B., & Tuzhilin, A. (2000). Small is beautiful: Discovering the minimal set of unexpected patterns. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data mining, Boston, MA (pp. 54-63).

ANGEL Mining

Paulsen, M. F. (2003). Online education and learning management systems. Bekkestua, Norway: NKI.

Sturgess, P., & Nouwens, F. (2004). “Evaluation of online learning management systems. Turkish Online Journal of Distance Education, 5(3).

Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco, CA: Morgan Kaufmann.

Tan, P.-N., Steinbach, M., & Kumar, V. (2006). Introduction to data mining. Boston, MA: Addison Wesley.

Rajasethupathy, K., Scime, A., Rajasethupathy, K. S., & Murray, G. R. (2009). Finding “persistent Rules”: Combining association and classification results’, Expert Systems With Applications, 36(3P2), 6019-6024. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j. compedu.2007.05.016 Roqueta, M. (2008). Learning management systems: A focus on the learner. Distance Learning, 5(4), 59–66. Scime, A. (2008). Web page extension of data warehouses. In Wang, J. (Ed.), Encyclopedia of Data Warehousing and Mining (2nd ed., pp. 1211–1215). Hershey, PA: Idea Group Reference. Scime, A. & Murray, G. R. (2007). Vote prediction by iterative domain knowledge and attribute elimination. International Journal of Business Intelligence and Data mining, 2(2), 160-176. Scime, A., Murray, G. R., & Hunter, L. Y. (2010). Testing terrorism theory with data mining. International Journal of Data Analysis Techniques and Strategies, 2(2), 122–139. doi:10.1504/ IJDATS.2010.032453 Sclater, N. (2008). Large-scale open source e-learning systems at open university UK. Research Bulletin EDUCAUSE Center for Applied Research, 2008(12), 1-13.

Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques (2nd ed.). San Francisco, CA: Morgan Kaufman.

ADDITIONAL READING Anand, S. S., Bell, D. A., & Hughes, J. G. (1995). The role of domain knowledge in data mining. Proceedings of the 4th International Conference on Information and Knowledge Management, Baltimore, MD (pp. 37-43). Andoh-Baidoo, F. K., & Kweku-Muata, O.-B. (2007). Exploring the characteristics of internet security breaches that impact the market value of breached firms. Expert Systems with Applications, 32(3), 703–725. doi:10.1016/j.eswa.2006.01.020 Ankerst, M., Martin, E., & Kriegel, H.-P. (2000). Towards an effective cooperation of the user and the computer for classification. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (pp. 179-188). Boston, MA. Bremer, D., & Bryant, R. (2005). A comparison of two learning management systems: Moodle vs blackboard. Proceedings of the 18th Annual Conference of the National Advisory Committee on Computing Qualifications (pp. 135-139). Tauranga, NZ.

111

ANGEL Mining

Brusilovsky, P. (2003). A distributed architecture for adaptive and intelligent learning managament systems. Proceedings of the AIED 2003 Workshop Towards Intelligent Learning Management Systems (pp. 5-13). Sydney, Australia.

Li, J., Tang, J., Li, Y., & Luo, Q. (2009). RiMOM: A dynamic multistrategy ontology alignment framework. IEEE Transactions on Knowledge and Data Engineering, 21(8), 1218–1232. doi:10.1109/ TKDE.2008.202

Carliner, S. (2005). Course management systems versus learning management systems. Learning Circuits, Retrieved from http://www.astd.org/ LC/2005/1105 _carliner.htm on April 19, 2010.

Machado, M., & Tao, E. (2007). Blackboard vs. Moodle: Comparing user experience of learning management systems. Proceeding of the 37th ASEE/IEEE Frontiers in Education Conference, (pp. S4J-7 - S4J-12). Milwaukee, WI.

Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Giarratano, J. C., & Riley, G. D. (2004). Expert systems: Principles and programming (4th ed.). New York: Course Technology. Han, J., & Kamber, M. (2001). Data mining: Concepts and techniques. Boston, MA: Morgan Kaufmann. Hofmann, M., & Tierney, B. (2003). The involvement of human resources in large scale data mining projects. Proceedings of the 1st International Symposium on Information and Communication Technologies (pp.103-109). Dublin, Ireland. Kass, G. (1980). An exploratory technique for investigating large quantities of categorical data. Applied Statistics, 29, 119–127. doi:10.2307/2986296 Kumar, D. A., & Ravi, V. (2008). Predicting credit card customer churn in banks using data mining. International Journal of Data Analysis Techniques and Strategies, 1(1), 4–28. doi:10.1504/ IJDATS.2008.020020 Lane, T., & Brodley, C. E. (2003). An empirical study of two approaches to sequence learning for anomaly detection. Machine Learning, 51(1), 73–107. doi:10.1023/A:1021830128811

112

Magidson, J. (1988). Improved statistical techniques for response modeling. Journal of Direct Marketing, 2(4), 6–18. doi:10.1002/ dir.4000020404 Magidson, J. (1994). The CHAID approach to segmentation modeling: Chi-squared automatic interaction detection. In Bagozzi, R. P. (Ed.), Advanced Methods of Marketing Research. Cambridge, MA: Basil Blackwell. Memon, N., & Qureshi, A. R. (2005). Investigative data mining and its application in counterterrorism. Proceedings of the 5th WSEAS International Conference on Applied Informatics and Communications (pp. 397-403). Malta. Paulsen, M. F. (2003). Experiences with learning management systems in 113 european institutions. Journal of Educational Technology & Society, 6(4), 134–148. Phelps, C., & Michea, Y. (2003). Learning management systems’ evaluation focuses on technology not learning. AMIA Annual Symposium Proceedings, (pp. 969). Ramu, K., & Ravi, V. (2008). Privacy preservation in data mining using hybrid perturbation methods: An application to bankruptcy prediction in banks. International Journal of Data Analysis Techniques and Strategies, 1(4), 313–331. doi:10.1504/IJDATS.2009.027509

ANGEL Mining

Robinson, S. M. (2008). Moodle vs. WebCT: How the design of features affect the efficiency of e-learning by influencing interaction and collaboration. Retrieved from http://web.njit.edu/~sr89/ MoodlevsWebct.doc on April 19, 2010. Roiger, R., & Geatz, M. W. (2003). Data mining: A tutorial-based primer. New York, NY: Addison Wesley. Sclater, N. (2008). Web 2.0, personal learning environments, and the future of LMS. Research Bulletin EDUCAUSE Center for Applied Research, 2008(13), 1-13. Su, X., Khoshgoftaar, T. M., & Greiner, R. (2009). Making an accurate classifier ensemble by voting on classifications from imputed learning sets. International Journal of Information and Decision Sciences, 1(3), 301–322. doi:10.1504/ IJIDS.2009.027657 Tozicka, J., Rovatsos, M., & Pechoucek, M. (2007). A framework for agent-based distributed machine learning and data mining. Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems (art 96). Honolulu, Hawaii. Turban, E., McLean, E., & Wetherbe, J. (2004). Information Technology for Management (3rd ed.). New York, NY: Wiley. Vatsavai, R. R., & Bhaduri, B. (2007). A hybrid classification scheme for mining multisource geospatial data, Proceedings of the 7th IEEE International Conference on Data mining Workshops (ICDMW 2007) (pp. 673-678). Omaha, NE. Wagstaff, K., Cardie, C., Rogers, S., & Schrödl, S. (2001). Constrained k-means clustering with background knowledge. Proceedings of the 18th International Conference on Machine Learning (pp.577-584). Williamstown, MA.

Wang, G., Zhang, C., & Huang, L. (2008). ‘A study of classification algorithm for data mining based on hybrid intelligent systems. Proceedings of the 9th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing (pp. 371-375). Phuket Thailand. Wang, Y., Zhang, Y., Xia, J., & Wang, Z. (2008). Segmenting the mature travel market by motivation. International Journal of Data Analysis Techniques and Strategies, 1(2), 193–209. doi:10.1504/ IJDATS.2008.021118 Webb, G. I., & Zheng, Z. (2004). Multistrategy ensemble learning: Reducing error by combining ensemble learning techniques. IEEE Transactions on Knowledge and Data Engineering, 16(8), 980–991. doi:10.1109/TKDE.2004.29

KEY TERMS AND DEFINITIONS Association Mining: A data mining method used to find patterns of data that show conditions where sets of attribute-value pairs occur frequently in the data set. Association Rule: A rule found by association mining. Classification Mining: A data mining method used to find models of data for categorizing instances; typically used for predicting future events from historical data. Classification Rule: A rule found by classification mining. Clustering: A data mining method used to group similar records together based on a measurable distance between the records. Clustering Rule: A rule found by the analysis of clusters. Data Mining: A collection of processes that inductively analyze data to assess known relationships as well as to find interesting patterns and unknown relationships.

113

ANGEL Mining

APPENDIX A: ASSOCIATION MINING RULES

countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6545 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6502 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 6496 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 6493 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 6463 conf:(0.97) countFolder=(-inf-0.5] 6649 ==> countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6463 conf:(0.97) countDropBoxes=(-inf-0.5] 8366 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7573 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countTurnItIn=(-inf-0.5] 8306 conf:(0.92) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countPage=(-inf-0.5] 8184 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countTurnItIn=(inf-0.5] 8161 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] 8137 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 8130 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 8032 conf:(0.91) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countDropBoxes=(-inf-0.5] countQuiz=(inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6713 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countLink=(inf-0.5] countTurnItIn=(-inf-0.5] 6704 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countForm=(inf-0.5] countLink=(-inf-0.5] 6694 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countQuiz=(-inf-0.5] countForm=(inf-0.5] countLink=(-inf-0.5] countTurnItIn=(-inf-0.5] 6684 conf:(0.9)

114

ANGEL Mining

countLink=(-inf-0.5] 7955 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7312 conf:(0.92) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] 7206 conf:(0.91) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7198 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7197 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countPage=(-inf-0.5] 7170 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7164 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.93) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7744 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7714 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] 7682 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countTurnItIn=(-inf-0.5] 7668 conf:(0.9) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7963 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 8306 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 8278 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 8202 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countDropBoxes=(-inf-0.5] countForm=(inf-0.5] 8161 conf:(0.9)

115

116

Chapter 6

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems Kamla Ali Al-Busaidi Sultan Qaboos University, Oman Hafedh Al-Shihi Sultan Qaboos University, Oman

ABSTRACT Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS. DOI: 10.4018/978-1-60960-884-2.ch006

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

INTRODUCTION Information and communication technologies (ICT) and the Internet have become major enablers of growth in business. The geographical outreach of the Internet and the wide global adoption of Web 2.0 technologies provide educational institutions with unprecedented opportunities to enhance their offerings. These technologies have transformed students’ perception of information and what they think about Web content and how to use it (Burnett & Marshall, 2003). Schools and universities are forced to investigate new means to revamp the educational process utilizing these technologies. Learning Management Systems (LMS) and e-learning have become the hype lately among stakeholders in education and training. The elearning market was worth more than US $18 billion worldwide in 2004 (Saady, 2005). Several leading universities around the world have adopted LMS for teachers and students to enhance the educational process (Hawkins & Rudy 2007; Browne, Jenkins & Walker, 2006; National Center for Educational Statistics, 2003). About 95 percent of participating institutions in the UK have adopted LMS (Browne et al., 2006); likewise, more than 90 percent of all participating universities and colleges in the US are adopting LMS (Hawkins & Rudy, 2007). Users’ acceptance and actual use of information systems is critical to its success. Likewise for a learning management system, its success to a great extent depends on users’ acceptance and use. Evaluating individual users’ acceptance and use of the e-learning systems is a “basic marketing element” (Kelly & Bauer, 2004). Although a learning management system survives through instructor and student use, the adoption of LMS is initiated by instructors’ acceptance and use. Even when LMS are well in place, instructors may not fully utilize all the features. For example, a survey of more than 800 instructors at over 35 institutions using Blackboard learning management system found that very few teachers use LMS tools for assessing

students or promoting community (Woods, Baker & Hopper., 2004). In addition, research indicates that fear of technology and lack of time may limit instructors’ adoption of LMS (Yueh & Hsu, 2008). Instructors should embrace and prepare for LMS use before preparing students for online learning (Chan, 2008). Even when designing LMS applications, teachers’ needs and capabilities should thoroughly be investigated (Yueh & Hsu, 2008). Therefore, teachers’ perspective within the context of LMS is crucial and should be studied carefully to ensure comprehensive uptake of LMS. Thus, the objective of this study was to investigate the critical factors influencing instructors’ acceptance and use of LMS, which may be influenced by technical and non-technical issues such as the instructors’ personal characteristics and the organization’s characteristics. It is important to analyze non-technical factors that promote the adoption and diffusion of LMS initiatives (Albirini, 2006; ElTartoussi, 2009). Consequently, this study specifically aimed to examine the impact of instructor’s individual characteristics (computer anxiety, technology experience and self efficacy), LMS characteristics (system quality, information quality and service quality), and organizational characteristics (management support, training and incentives) on the instructors’ acceptance (perceived ease of use and perceived usefulness) and use of LMS as a supplementary tool. The study also assessed the impact of the instructors’ acceptance and use of LMS on their intention for continuous supplementary use of LMS and intention for pure use of LMS for distance education. Many organizations start their LMS adoption as a supplementary tool to traditional teaching, hoping that this supplementary adoption eventually will promote the pure use of LMS for distance education. The following sections discuss the background literature, research framework and methodology, analysis and results, and the conclusion.

117

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

BACKGROUND Learning Management Systems A learning management system is “software that automates the administration of training” (World Bank, 2010). It is the use of a Web-based communication, collaboration, learning, knowledge transfer, and training to add value to learners and businesses (Kelly & Bauer, 2004). Specifically, a learning management system is an Internet application that aims to support education and training activities (Cavus & Momani, 2009) and provides a platform to support e-learning activities (Yueh & Hsu, 2008). Course Management Systems (CMS) and Learning Content Management Systems (LCMS) are sometimes used to indicate LMS (Yueh & Hsu, 2008); other related terms are Computer-assisted Learning (CAL), Computer-based Learning (CBL), and Online Learning (Chan, 2008). It should be noted, however, that LMS applications are not unique to educational institutions; even public and private organizations use such systems for training purposes. Reiser and Dempsey (2002) stated that in large US corporations, 20 percent of all training is delivered via LMS.

LMS Applications and Benefits to Instructors Many LMS applications are available. The most popular LMS used at colleges and universities in the US is Blackboard followed by WebCT, which was acquired by Blackboard, Inc. in 2006 (Falvo & Johnson, 2007). WebCT was used by millions of students from more than 2,500 universities in more than 80 countries (Chan, 2008). Other LMS solutions are Moodle, ATutor, Learn.com, Joomla, and Krawler. Blackboard has course management features that support integration with student databases (Kneght & Reid 2009; Blackboard, 2009). WebCT supports electronic communications such as email, bulletin boards, and chat rooms (Chan,

118

2008). Moodle learning management system, on the other hand, is sometimes preferred over the previous popular LMS packages (see, for example, Beatty and Ulasewicz, 2006). It is scalable open-source software used mainly by North American and European universities that supports group forum participation and features interactive tools between students and instructors (Beatty & Ulasewicz, 2006). LMS applications provide several features to instructors, especially to those who see the real benefits of the Internet in the teaching process (Burniske & Monke, 2001). Yueh and Hsu (2008) described course management tools, group chat and discussion, assignment submission, and course assessment as the primary tools in LMS. In addition, LMS help teachers provide students with educational materials and track their participation and assessments (Falvo & Johnson, 2007). Yildirim, Temur, Kocaman and Goktas (2004) described more technically sophisticated LMS features such as maintaining office hours online, creating student groups, and assigning online projects to groups. Ceraulo (2005) mentioned ePortfolios as a key feature in some LMS applications, which enable instructors to maintain student submissions throughout the course (i.e., tests, assignments, projects). LMS solutions tend also to increase interest in learning and teaching among students and teachers, respectively (Mahdizadeh, Biemans & Mulder, 2008). Aczel, Peake and Hardy (2008), and Naidu (2006) stated that LMS enhance teaching process efficiency and result in cost-savings.

Learning Management Systems Acceptance and Use User Technology Acceptance and Use LMS have been adopted by academic institutions to support their distance education and/or supplement their traditional way of teaching (Rainer, Turban & Potter 2007). User acceptance and use

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

of information systems is critical to their success. The same is true for a learning management system: its success, to a great extent, depends on users’ acceptance and use. The assessment of technology success has been conducted by utilizing various acceptance and use dimensions. Several studies assessed the determinants of technology acceptance (Bailey & Pearson, 1983; Davis, 1989; DeLone & McLean, 2003; Doll & Torkzadeh, 1988; Venkatesh & Davis, 2000). For instance, Davis (1989) assessed technology acceptance by perceived usefulness and intention to use; Venkatesh and Davis (2000) assessed technology acceptance by perceived usefulness, intention to use, and usage behavior. Alternatively, DeLone and McLean (2003) assessed technology success by user satisfaction and usage. In the LMS context, researchers also assessed instructors’ acceptance and use in various ways. Liaw, Huang and Chen (2007) assessed LMS acceptance by learners’ and instructors’ behavioral intention to use e-learning, which is influenced by perceived usefulness, perceived self-efficacy, and perceived enjoyment. Ball and Levy (2008) assessed LMS acceptance by instructors’ intention to use. Teo (2009) assessed LMS acceptance by teachers’ perceived usefulness and perceived ease of use. None of these studies, however, investigated the direct impact of instructors’ characteristics, LMS’ characteristics, and/or an organization’s characteristics on actual system use. Perceived usefulness, perceived ease of use, users’ satisfaction, and intention to use are important measures for technology acceptance and may eventually correlate with actual use behavior. Some researchers have found direct effects between such external factors and technology use (Igbaria, Guimaraes & Davis, 1995). Nevertheless, these acceptance measures do not explain all the variance of actual usage behavior. In addition, measuring attitudes and their link to actual usage behavior is extremely difficult; therefore, many researchers may choose

to stay with actual use behavior (DeLone & McLean, 2003). Thus, it is important to evaluate the direct impact of these critical determinants on actual system use. System use is and will continue to be an important indication of IS success (DeLone & McLean, 2003). In addition, the actual benefits of any technology may be realized only from actual technology use. Thus, technology use is the main aim of organizations to promote LMS and realize some of its expected benefits. Perceived usefulness and ease of use may be important factors for continuous intention to use. Consequently, As indicated earlier this study aimed to examine the direct impact of instructors’ characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ actual usage behavior, instructors’ perceived ease of use of LMS, and instructors’ perceived usefulness of LMS. Furthermore, the study examined the impacts of these acceptance (perceived ease of use, perceived usefulness), and actual use measures on continuous intention for supplementary use and intention for pure use of the LMS for distance education. Perceived usefulness and perceived ease of use may be important to ensure continuous use of the system and future intention to adopt it for distance education.

Instructor Characteristics The acceptance and use of LMS may, to a great extent, be determined by the characteristics of its users. Several dimensions of users’ characteristics have been proposed and investigated as determinants of technology acceptance. Some of these are users’ experience of technology, users’ self-efficacy, and users’ computer anxiety. Users’ experience of technology has been highly utilized to determine users’ technology acceptance (Venkatesh & Davis, 2000; Thompson, Compeau, Deborah & Higgins, 2006). Individual technology experience is the individual’s exposure to the technology as well as the skills and abilities that

119

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

s/he gains through using a technology (Thompson et al., 2006); users’ self-efficacy pays a major role in users’ acceptance and use of technology. Self-efficacy is defined as “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (Bandura, 1977, p.391). Thus, computer self-efficacy means individual’s selfassessment of their ability to apply computer skills to accomplish their tasks (Compeau & Higgins, 1995). Furthermore, a user’s computer anxiety is considered an important factor for technology acceptance and use. Computer anxiety is defined as “the fear or apprehension felt by individuals when they used computers, or when they considered the possibility of computer utilization” (Simonson, Maurer, Montag-Torardi & Whitaker, 1987, p. 238). Fear of computers may negatively impact technology acceptance and use. An individual’s computer anxiety, self efficacy and experience with technology may share some correlations, but they are not exactly similar individual traits (Ball & Levy, 2008). Computer anxiety is not simply a negative, short-term attitude toward computers that can be reduced by increasing technology experience; it is the users’ individual fear associated with computer use with or without experience (Ball & Levy, 2008). Likewise, individuals’ self efficacy is an individual trait that some individuals might have with or without technology experience. Some individuals are capable to accept and use LMS without a prior experience with technology because they have high level of self efficacy. In the context of e-learning, few studies have investigated the impact of instructors’ dimensions on LMS acceptance. Ball and Levy (2008) investigated the impact of self-efficacy, computer anxiety, and technology experience on instructors’ intention to use emerging learning experience in a small private university in the US and found that self-efficacy was the only major determinant of instructors’ intention. Teo (2009) found that computer self-efficacy directly impacts pre-service

120

teachers’ perceived usefulness, perceived ease of use, and behavioral intention in Singapore. Liaw et al. (2007) found that perceived self-efficacy determines instructors’ behavioral intention to use e-learning in Taiwan. Albirini (2006) investigated the perception of school teachers of the use of ICT in education in Syria, and the results highlighted the importance of teachers’ vision of technology, their experiences with it, and the cultural conditions on their attitudes toward technology. Mahdizadeh and his colleagues (2008) found that teachers’ previous experience with e-learning environments and ease of use explain teachers’ perception of the usefulness of e-learning environments and their actual use of these environments.

LMS Characteristics The characteristics of LMS may have a great impact on the instructor’s acceptance and use of LMS. Characteristics of any information system, including LMS, may be related to system, information, and service support quality as classified by DeLone and McLean (2003). E-learning systems’ quality was found to be significant on the instructors’ perceived usefulness, perceived enjoyment, and perceived self-efficacy, which consequently affect their intention to use the system in the classroom (Liaw et al., 2007). System quality, which refers to the characteristics of a system, is a key in users’ acceptance and use of any technology, including LMS. Researchers, such as Bailey and Pearson (1983), DeLone and McLean (2003), and Seddon (1997) have highlighted the impact of system quality on technology acceptance and introduced several ways to measure system quality. Information quality, which refers to the perceived output produced by the system, is also an important factor in instructors’ acceptance and use of LMS. Information quality refers to the accuracy, relevance, timeliness, sufficiency, completeness, understandability, format, and accessibility of the information (Bailey

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

& Pearson, 1983; Seddon, 1997). In addition, service quality may be a factor on the instructors’ acceptance and use of LMS; it refers to the quality of support services provided to the system’s endusers. DeLone and McLean (2003) indicated that ignoring service quality endangers IS effectiveness measurements. Common measurements of service quality are tangibles, reliability, responsiveness, assurance, and empathy (Parasuraman, Zeithaml, & Berry, 1988; Kettinger & Lee, 1994). In the e-learning context, few studies have examined the general quality of technology or specific dimension. For instance, from instructors’ and learners’ perspective, Liaw et al. (2007) investigated the impact of e-learning systems’ general quality on perceived usefulness, perceived enjoyment, and perceived self-efficacy, which consequently affect their intention to use the system in the classroom, and found it significant. Albirini (2006) indicates that instructors’ vision of technology impacts their attitudes toward the use of ICT in education. Two significant studies on the impact of technology on users’ acceptance of LMS are Pituch and Lee’s (2006) and Roca, Chiu and Martinez’s (2006), but they are from the learners’ perspective. Roca et al. (2006) investigated learners’ perceived system quality from three dimensions (system quality, information quality, and service quality). They found that learners’ perceived system quality factors (system quality, information quality, and service quality) directly affect their e-learning satisfaction and intention to use and indirectly their perceived usefulness. Pituch and Lee (2006) examined the impact of system quality from three dimensions: the system’s functionality, interactivity, and response. As indicated, limited studies provide a detailed examination of the influence of the three dimensions (system quality, information quality, service quality) of LMS on instructors’ acceptance. This study integrates these three dimensions of LMS on the instructors’ acceptance.

Organization Characteristics An organization’s characteristics play a major role in the behaviors of its employees, including the acceptance and use of any technology such as LMS. Corporate culture plays a key role in the success of any project. Schein defines culture as “the way we do things around here” (1985, p. 12). Cultural values shape an organization’s norms and practices, which consequently influence employees’ behaviors such as LMS utilization. Some of an organization’s characteristics that might be relevant to the utilization of LMS are management support, incentives, and training. Organizational support, represented by senior managers’ support, is also important for instructors to accept and use LMS in their teaching. Management’s support of end-users significantly improves computer usage (Igbaria, 1990). In the e-learning context, senior management support and the alignment of e-learning with the department and university curriculum are important for its adoption (Sumner & Hostetler, 1999). Motivators are also an important factor for instructors’ acceptance to integrate the technology in teaching. Motivators or incentives for instructors can be enforced by having the use of the technology as a factor in a nomination for teaching award, promotion, and tenure (Sumner & Hostetler, 1999). Finally, training end-users is important, and can be in form of workshops, online tutorials, courses, and seminars. There is a lack of empirical studies that capture the influence of organization factors on the acceptance and use of LMS generally. One of these is a qualitative study by Sumner and Hostetler (1999), who categorize the organizational factors that may influence the use of technology in teaching in terms of motivators/demotivators, training, technology alignment, and organizational and technical support. In addition, Teo (2009) found that facilitating conditions, measured by technical support, training, and administrative support, indirectly affect teachers’ acceptance of technology in education.

121

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

INSTRUCTORS’ LMS ACCEPTANCE AND USAGE FRAMEWORK Framework Development This study aimed to examine the impact of instructor’s individual characteristics, LMS’ characteristics, and an organization’s characteristics on instructors’ acceptance ad usage of LMS as a supplementary tool and, consequently, on continuous use and pure use intention for distance learning. As indicated, few studies have examined this integrated investigation of instructors’ LMS acceptance and usage. This study assessed the individual characteristics based on instructors’ computer anxiety and technology experience, LMS characteristics based on system, information, and service quality; and organizational characteristics based on management support, incentives and training. The study assessed the impact of these factors on the instructors’ acceptance (perceived ease of use and perceived usefulness) Figure 1. Instructors LMS acceptance and use model

122

and usage of LMS and, consequently, continuous supplementary use and future pure use intention. Figure 1 illustrates this study model.

Instructor Individual Characteristics Hypotheses Computer Anxiety Hypotheses Computer anxiety is “the fear or apprehension felt by individuals when they used computers, or when they considered the possibility of computer utilization” (Simonson, et al., 1987, p. 238). Computer anxiety is an important factor for the acceptance of the technology (Ball & Levy, 2008; Piccoli, Ahmad & Ives, 2001; Raaij & Schepers, 2008; Sun, Tsai, Finger, Chen & Yeh, 2008). Fear of computers may negatively affect the acceptance of LMS and the user’s perceived satisfaction (Piccoli et al., 2001). Empirical evidence of the impact of computer anxiety was mixed. Ball and Levy (2008) did not detect a significant link between

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

computer anxiety and instructors’ intention to use the e-learning; however, Sun et al.(2008) found that computer anxiety significantly impacts the learners’ perceived satisfaction of e-learning, and Raaij and Schepers (2008) found the computer anxiety impacts the learner’s perceived ease of use of e-learning. Therefore we hypothesized that: Hypothesis 1a: Instructors’ computer anxiety is negatively associated with perceived ease of use of LMS. Hypothesis 1b: Instructors’ computer anxiety is negatively associated with perceived usefulness of LMS. Hypothesis 1c: Instructors’ computer anxiety is negatively associated with use of LMS.

Technology Experience Hypotheses Users’ experience with the technology (EUT) also plays a major role in the acceptance of technology (Venkatesh & Davis, 2000; Thompson et al., 2006). An individual’s EUT is his/her exposure to the technology as well as the skills and abilities that are gained through using a technology (Thompson et al., 2006). Therefore, EUT may impact instructors’ acceptance of LMS for their classes. Although empirical quantitative research, such as that of Ball and Levy (2008), found no significant impact of EUT on instructors’ intention to use LMS, researchers Sumner and Hostetler (1999) indicate that current level of computer skills and extent of use of computing skills in teaching are important for instructors’ acceptance of ICT in education. Likewise, Wan, Fang and Neufeld (2007) highlight the importance of technology experience on the learning processes and, consequently, learning outcomes. Mahdizadeh et al. (2008) reveal that instructors’ prior experience with e-learning may explain their perception of the usefulness of e-learning environments and their actual use. Therefore we hypothesized:

Hypothesis 2a: The instructor’s experience with the use of technology is positively associated with perceived ease of use LMS. Hypothesis 2b: The instructor’s experience with the use of technology is positively associated with perceived usefulness of LMS. Hypothesis 2c: The instructor’s experience with the use of technology is positively associated with use of LMS.

Self-Efficacy Hypotheses Self-efficacy is “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (Bandura, 1977, p.391). Thus, computer self-efficacy means individuals’ selfassessment of their ability to apply computer skills to accomplish their tasks (Compeau & Higgins, 1995). Several empirical studies found significant effects of computer self-efficacy on the perceived usefulness on information systems (Venkatesh & Davis, 2000; Chau, 2001). The more ability the instructor has to apply a computer application, the most likely s/he perceives it easy to use and useful, and will eventually use it. In the context of e-learning systems, Ball and Levy (2008) found a significant effect of self-efficacy on instructors’ intention to use. In addition, computer self-efficacy was found to be significant on learners’ perceived ease of use (Lee, 2006; Pituch & Lee, 2006; Roca et al., 2006) and learners’ perceived satisfaction (Sun et al., 2008). Therefore, we hypothesized: Hypothesis 3a: An instructor’s computer selfefficacy is positively associated with perceived ease of use of LMS. Hypothesis 3b: An instructor’s computer selfefficacy is positively associated with perceived usefulness of LMS. Hypothesis 3c: An instructor’s computer selfefficacy is positively associated with use of LMS.

123

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

LMS Characteristics Hypotheses System Quality Hypotheses System quality is essential for the acceptance of any technology, including LMS. Researchers, such as Bailey and Pearson (1983), DeLone and McLean (1992), and Seddon (1997) highlight the impact of system quality on technology acceptance and have introduced several ways to measure it. Instructors’ acceptance of LMS may be determined to a great extent by system quality. The more functionalities, interactivity, and response of LMS, the better is its acceptance and utilization (Pituch & Lee, 2006). Quantitative empirical studies found a significant impact of system characteristics on e-learning acceptance: reliability (Wan et al., 2007; Webster & Hackley, 1997); accessibility (Wan et al., 2007); and system functionality, interactivity, and response (Pituch & Lee, 2006). Albirini (2006) indicates that instructors’ vision of technology impacts their attitudes toward the use of ICT in education. Therefore, we hypothesized that: Hypothesis 4a: LMS system quality is positively associated with the instructor’s perceived ease of use LMS. Hypothesis 4b: LMS system quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 4c: LMS system quality is positively associated with the instructor’s use of LMS.

Information Quality Hypotheses Information quality is also important for instructors’ acceptance of LMS, and refers to the perceived output produced by the system. Information quality with great accuracy, relevance, timeliness, sufficiency, completeness, understandability, format, and accessibility are important for the acceptance of an information technology (Bailey & Pearson, 1983; Seddon, 1997). There is a lack

124

of research on the impact of information quality on instructors’ acceptance of LMS. Some research was conducted from the learners’ perspective. Roca et al. (2006) measured information quality of LMS by indicators related to relevance, timeliness, sufficiency, accuracy, clarity, and format, and proved that information quality was directly significant for learners’ satisfaction and indirectly for perceived usefulness. Likewise, Lee (2006) found content quality was significant for learners’ perceived usefulness. In the Middle East, Al-Busaidi (2009), in an exploratory study in Oman, indicated that information quality (sufficiency, accuracy, relevance, timeliness, and understandability) was highlighted as a determinant of learners’ LMS use. Consequently, we hypothesize that: Hypothesis 5a: LMS information quality is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 5b: LMS information quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 5c: LMS information quality is positively associated with the instructor’s use of LMS.

Service Quality Hypotheses Service quality refers to the quality of support services provided to the system’s end-users. Instructors’ acceptance of LMS may, to some extent, be related to the quality of the support services. Common measurements of service quality are tangibles, reliability, responsiveness, assurance, and empathy (Parasuraman et al., 1988; Kettinger & Lee, 1994). Few studies have investigated the impact of service quality on LMS acceptance. For instance, Roca et al. (2006) assessed service quality by indicators related to responsiveness, reliability, and empathy, and confirmed its direct significance on learners’ satisfaction and indirect significance of perceived usefulness in the e-learning context. Thus, we hypothesized that:

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Hypothesis 6a: LMS service quality is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 6b: LMS service quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 6c: LMS service quality is positively associated with the instructor’s use of LMS.

Hypothesis 7a: Management support is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 7b: Management support is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 7c: Management support is positively associated with the instructor’s use of LMS.

Organization Characteristics Hypotheses

Incentives Hypotheses

Management Support Hypotheses Management support is a key factor for the acceptance of any organizational initiative. Senior managers’ open approval and endorsement of LMS adoption promote instructors’ adoption and acceptance of LMS. Managers may support an LMS by encouraging instructors to adopt it and identify a clear vision of the objective of the LMS and how it is aligned with the university vision. Little research has investigated the impact of management support on instructors’ acceptance of LMS. However, in the e-learning context, senior managers should clearly identify the goal of LMS for the university curriculum (Sumner & Hostetler, 1999). This managers’ support assures instructors that using LMS is part of the organization’s culture and is useful and encourages them to adopt and use the system. Managers are recognized as a high authority (Ali, 1990); thus, instructors’ adoption and acceptance of LMS may be associated with the endorsement of their senior managers. Management support of end-users significantly improves computer usage (Igbaria, 1990). Facilitating conditions, including administrative support, indirectly affect teachers’ acceptance of technology in education (Teo, 2009). Consequently, we hypothesized that:

Motivators, in terms of incentives, are important factors for instructors’ acceptance to integrate LMS in their teaching. Incentives can be “nontrivial” monetary and non-monetary incentives. E-learning research lacks the assessment of incentives on LMS acceptance. Motivators or incentives for instructors can be enforced by using the technology as a factor in nomination for a teaching award, promotion, and tenure (Sumner & Hostetler, 1999). These incentives’ policies push instructors to adopt and utilize LMS for their teaching. Therefore, we hypothesized that: Hypothesis 8a: An incentive policy is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 8b: An incentive policy is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 8c: An incentive policy is positively associated with the instructor’s use of LMS.

Training Hypotheses Providing end-users with training is important, as training improves instructors’ adoption of LMS and enhances the perceived ease of use of LMS, illustrates its potential usefulness, and encourages its use in teaching. Limited research has investigated the impact of training on instructors’ acceptance of LMS, which can be in the form of workshops, online tutorials, courses, and seminars (Sumner &

125

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Hostetler, 1999). Facilitating conditions, including training, indirectly affect teachers’ acceptance of technology in education (Teo, 2009). Thus, we hypothesized: Hypothesis 9a: Training is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 9b: Training is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 9c: Training is positively associated with the instructor’s use of LMS.

Usage and Future Intention Hypotheses Perception and Usage Hypotheses A technology’s perceived ease of use and perceived usefulness are found to be a significant determinant of the intention to use the technology (Venkatesh & Davis, 2000). The higher the instructors’ perceived ease of use and perceived usefulness of LMS, the higher the actual use. Perceived ease of use is also a significant determinant of users’ perceived usefulness of a technology (Venkatesh & Davis, 2000). Pituch and Lee (2006) found learners’ perceived ease of use of LMS to significantly affect perceived usefulness. Therefore, we hypothesized: Hypothesis 10a: Instructors’ perceived ease of use of LMS is positively associated with their use of LMS. Hypothesis 10b: Instructors’ perceived usefulness of LMS is positively associated with their use of LMS. Hypothesis 10c: Instructors’ perceived ease of use of LMS is positively associated with their perceived usefulness of LMS.

126

Continuous Supplementary Use Intention Hypotheses The intention to use the technology is significantly determined by users’ perceived ease of use and perceived usefulness (Venkatesh & Davis, 2000). The higher the instructors’ perceived ease of use of LMS, perceived usefulness of LMS, and actual use, the more likely it is that they will continue to use it. Continuous intention to e-learning use is determined by perceived usefulness and satisfaction (Hayashi, Chen, Ryan, Wu, 2004). Thus, we hypothesized: Hypothesis 11a: The instructors’ perceived ease of use of LMS is positively associated with their continuous supplementary use intention. Hypothesis 11b: The instructors’ perceived usefulness of LMS is positively associated with their continuous supplementary use intention. Hypothesis 11c: The instructors’ supplementary use of LMS is positively associated with their continuous supplementary use intention.

Future Pure Use Intention Hypotheses Many organizations begin their LMS adoption as a supplementary tool to traditional teaching, hoping that this supplementary adoption will eventually promote the pure use of LMS for distance education. Perceived ease of use, perceived usefulness, and actual use may have an important impact on continuous intention for supplementary use and intention for pure use of the LMS for education. When instructors believe that LMS is easy, useful, and can be utilized for supplementary purposes, they are more likely to adopt it purely for distance education. Technology perceived ease of use and perceived usefulness are found to be significant determinants of the intention to use the technology (Venkatesh & Davis, 2000). Perceived ease of use, perceived usefulness, and supplementary

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

use are significant determinants of learners’ use of e-learning for distance education (Pituch & Lee, 2006). Thus we hypothesized: Hypothesis 12a: The instructors’ perceived ease of use of LMS is positively associated with their pure use intention. Hypothesis 12b: The instructors’ perceived usefulness of LMS is positively associated with their pure use intention. Hypothesis 12c: The instructors’ supplementary use of LMS is positively associated with their pure use intention.

METHODOLOGY Participants’ Profile This study included 82 instructors from Sultan Qaboos University (SQU), the first and only public university in Oman. Since its launch, SQU has gone through many technology developments: it adopted WebCT and later switched to the open-source Moodle application. Instructors can voluntarily adopt LMS to supplement their traditional classes. The instructors were from different colleges in the university and with different demographics. About 62 percent were male and 38 percent were female. About 5 percent were assistant lecturers, 27 percent were lecturers, 50 percent were assistant professors, 13 percent were associate professors, and 5 percent were full professors. The instructors’ age varied from 20s to above 50s: about 8 percent were in their 20s, 26 percent were in their 30s, 16 percent in their 40s, and 32 percent were 50 or over. Almost 44 percent had less than six years of work experience, 30 percent had less than 11 years, 16 percent had less than 16 years, 7 percent had less than 21 years, and 2 percent had more than 20 years. Most indicated that their computer skills were above average. Almost 71 percent have above average computer skills; 23

percent, about average; and only 6 percent were below average. The majority, about 59 percent, has used the LMS for classes for three years or more; 30 percent have used it for one to two years; and 11 percent have used it for less than one year. The majority of the instructors, about 55 percent, have experience with only Moodle LMS; 9 percent have experience with only WebCT LMS; 31 percent have experience with both WebCT and Moodle; and 5 percent have experience with Blackboard.

Research Questionnaire The questionnaire was distributed to SQU instructors. An invitation email was sent to instructors to complete the study questionnaire either online or on an attached MS Word document. A reminder was sent two weeks after the initial invitation. Most of the instructors filled the questionnaire online (about 95 percent of them).Only five percent of instructors completed the questionnaire as a hard copy. The questionnaire included the constructs to be measured for quantitative analysis, along with demographic questions (e.g., gender, age, degree, LMS usage experience, work experience, and job title). Construct measurements items were phrased according to a five–point Likert scale (1= strongly disagree; 2=disagree; 3=Neutral; 4= agree and 5=strongly agree). To statistically evaluate the study framework, 28 indicators were used. Tables 1 and 2 show the total indicators used for each construct. The LMS characteristic constructs (system quality, information quality, and service quality) were adopted and modified from Roca et al. (2006) and Pituch and Lee (2006); the individual characteristics constructs (computer anxiety, self-efficacy, and technology experience) were adopted from Ball and Levy (2008); while the organizational characteristics’ constructs (management support, incentives, and training) were self-developed, based on Sumner and Hostetler (1999). The LMS acceptance construct (perceived ease of use and perceived

127

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Table 1. Independent constructs measures and loadings Construct Measures

Loading

Computer Anxiety 1. I believe that working with computers is very difficult.

0.8913

2. Computers make me feel uncomfortable.

0.9560

3. I get a sinking feeling when I think of trying to use a computer.

0.8698

Technology Experience 1. I feel confident using the e-learning system

0.8465

2. I feel confident downloading/uploading necessary materials from the Internet.

0.8756

3. I feel confident using the chatting and discussion forums.

0.6275

Self Efficacy 1. I could use the e-learning system if I had never used a system like it before.

-0.7948

2. I could use the e-learning system if I had only the system manuals for reference.

0.0894

3. I could use the e-learning system if I had seen someone else using it before trying it myself.

0.8652

System Quality 1. The system offers flexibility in teaching as to time and place.

0.7694

2. The system offers multimedia (audio, video, and text) types of course content.

0.7962

3. The response time of the system is reasonable.

0.6025

4. The system enables interactive communication between instructor and students.

0.7737

Information Quality 1. The information provided by the system is relevant for my job.

0.8434

2. The information in the system is very good.

0.8919

3. The information from the e-learning system is up-to-date.

0.8407

4. The information provided by the system is complete.

0.8494

Service Quality 1. The system support services give me prompt service.

0.8425

2. The system support services have convenient operating hours.

0.8410

3. The system support services are reliable.

0.8761

4. The system support services are easy to communicate with.

0.8904

Management Support 1. Senior administrators strongly support the use of e-learning system.

0.9396

2. I get support by department chair or dean on my use of e-learning system.

0.8506

3. My mangers highlight the importance of e-learning system on my curriculum.

0.9188

Incentives 1. The use of e-learning is a factor in the nomination for teaching award.

0.9529

2. The use of e-learning system is a factor in determining promotion.

0.9558

3. The use of e-learning system is a factor in annual elevation of teaching.

0.9611

Training

128

1. I receive training workshops on how to use e-learning tools.

0.7277

2. I receive on-line manuals on how to use e-learning tools.

0.8759

3. I receive seminars on the use of e-learning tools.

0.8378

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

usefulness) were adopted and modified from Venkatesh and Davis (2000), and supplementary use, continuous supplementary use, and future pure use were adopted and modified according to Pituch and Lee (2006).

DATA ANALYSIS & RESULTS PLS Analysis Methodology Data was analyzed by PLS-Graph 3.0 software. PLS (partial least square) is a variance-based structural equation model (SEM) technique that allows path analysis of models with latent variables

(Chin, 1998; Chin, 2001). The PLS approach is a variance-based SEM that assists researchers in obtaining determinate values of latent variables for predictive purposes. The PLS does that by minimizing the variance of all dependent variables rather than using the model to explain the co-variation of all indicators (Chin, 1998; Chin and Newsted, 1999). Thus, the model paths are estimated based on the ability to minimize the residual variances of the dependent variables. The PLS algorithm uses an iterative process for the estimation of weights and latent variables scores. The process almost converges to a stable set of weight estimates. The evaluation of the model is based on (1) the assessment of the model mea-

Table 2. Dependant constructs measures and loadings Construct Measures

Loading

Perceived Ease of Use 1. Using e-learning tools is easy to me.

0.8896

2. E-learning tools are clear and understandable to me.

0.9280

3. I find it easy to get the e-learning system to do what I want it to do.

0.7843

Perceived Usefulness 1. Using e-learning system enables me to accomplish tasks more quickly.

0.8581

2. Using e-learning system improves my performance.

0.9251

3. Using e-learning system increases my productivity.

0.9255

4. Using e-learning system enhances the effectiveness on the job.

0.8958

5. Using e-learning system gives me greater control over my work.

0.8831

Supplementary Use 1. I use the e-learning system as many occasions as possible to supplement my teaching.

0.9165

2. I use the e-learning system on regular basis to supplement my teaching.

0.9047

3. I frequently use the e-learning system to supplement my teaching.

0.9051

4. I use the e-learning system to share/seek course information.

0.7357

5. I use the e-learning system to communicate with students

0.8259

Continuous supplementary Use Intention (CUI) 1. I will frequently use e-learning system to do a teaching task.

0.8871

2. I will use e-learning system on regular basis to supplement my classes in the future.

0.8523

3. I will always try to use the e-learning system to do a teaching task whenever it has a useful feature.

0.8915

Pure Use Intention (PUI) 1. I plan to teach purely online courses for distance learners.

0.9249

2. I will use e-learning system to teach purely online courses.

0.9627

3. I plan to teach purely online courses in as many occasions as possible.

0.9430

129

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

surements by assessing their validity, reliability, and discriminant validity, (2) the analysis of the paths of the structural model (Chin, 1998). Table 1 and Table 2 show the independent and dependant constructs’ measures and loading respectively.

Constructs Validity and Reliability The reliability and the validity are two criteria used by researchers to evaluate the applicability of their measurements to their investigated model. Reliability refers to the consistency of the measures (indicators) of a specific latent variable; whereas, validity refers to how well the concept is defined by the measures (Hair, Anderson, Tatham & Black, 1998; Crano & Brewer, 2002). With PLS, the reliability of the measurements was evaluated by internal consistency reliability, and the validity was measured by the average variance extracted (AVE), which refers to the amount of variance a latent variable captures from its indicators. AVE was developed by Fornell and Larcker (1981) to assess construct validity. The recommended level for internal consistency reliability is at least 0.70, and is at least 0.50 for AVE (Chin, 1998). Tables 1 and

2 show the model constructs’ measurements and loading. Table 3 shows that the study constructs’ reliability and AVE are above the recommended levels for all the constructs except self-efficacy. Therefore, the self-efficacy construct was dropped from the model evaluation. To achieve the discriminant validity of the constructs, Fornell and Larcker (1981) suggest that the square root of AVE of each construct should exceed the correlations shared between the constructs and other constructs in the model. The discriminant validity is used to ensure the differences among constructs (Barclay, Higgins & Thompson, 1995; Chin, 1998). Table 4 shows that the model constructs satisfy that rule, as the square root of the AVE (on the diagonal) is greater than the correlations with other constructs. Thus, all the model constructs have a satisfactory discriminant validity construct.

Model Evaluation and Paths Analysis With PLS, R-square values are used to evaluate the predictive relevance of a structural model for the dependent latent variables, and the path

Table 3. Constructs reliability and validity Construct

130

Total Items

Reliability

AVE

Computer Anxiety (CA)

3

0.932

0.822

Technology Experience (TE)

3

0.831

0.626

Self Efficacy (SE)

3

0.504

0.267

System Quality (SQ)

4

0.827

0.547

Information Quality (IQ)

4

0.917

0.734

Service Quality (SvQ)

4

0.921

0.744

Management Support (MS)

3

0.930

0.817

Incentives (IN)

3

0.970

0.915

Training (TR)

3

0.856

0.666

Perceived Ease of Use (PEU)

3

0.902

0.756

Perceived Usefulness (PU)

5

0.954

0.806

Supplementary Use (USE)

5

0.934

0.740

Continuous Supplementary Use Intention (CUI)

3

0.909

0.769

Pure Use Intention (PUI)

3

0.961

0.890

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

coefficients are used to assess the effects of the independent variables(Chin, 1998). The significance of the model paths was checked based on their t-values. Table 5 shows the R2 values of the endogenous dependent constructs. The model explains 58 percent of variance in instructors’ usage of LMS; 52.2 percent of variance in instructors’ perceived usefulness of LMS; and 42.4 percent of variance in instructors’ perceived ease of use of LMS. In addition, instructors’ usage of LMS, their perceived usefulness of LMS, and their perceived ease of use of LMS explains 55.3 percent of variance in their continuous supplementary use intention, but only 22.9 percent of their pure use intention of LMS. Table 5 also shows the paths’ coefficients

analysis between the exogenous independent constructs (instructors’ characteristics, LMS’s characteristics, and organization’s characteristics) and the endogenous dependent construct (LMS supplementary use, LMS perceived ease of use, LMS perceived usefulness), and, consequently, future intention (continuous supplementary use intention and pure use intention). The statistical significant of the paths’ coefficients was measured by T-values with at least 95 percent confidence level. The analysis showed that the instructor’s characteristics, the LMS’s characteristics and the organization’s characteristics to some extent have impact on the instructor’s acceptance and use of LMS; see Table 5. First, instructors’ computer

Table 4. Construct’ correlations and discriminant validity Construct

CA

TE

SQ

IQ

SvQ

MS

IN

TR

PEU

PU

USE

CUI

Computer Anxiety (CA)

0.907

Technology Experience (TE)

-0.124

0.791

System Quality (SQ)

-0.072

0.202

0.740

Information Quality (IQ)

-0.069

0.188

0.675

0.857

Service Quality (SvQ)

-0.024

0.076

0.407

0.689

0.863

Management Support(MS)

0.175

0.013

0.257

0.207

0.206

0.904

Incentives (IN)

0.224

-0.196

0.144

0.133

0.140

0.447

0.957

Training (TR)

0.024

-0.001

0.235

0.343

0.347

0.220

0.283

0.816

Perceived Ease of Use (PEU)

-0.362

0.313

0.418

0.436

0.377

0.182

0.042

0.314

0.869

Perceived Usefulness (PU)

-0.175

0.230

0.614

0.381

0.176

0.223

0.175

0.204

0.546

0.898

Supplementary Use (USE)

-0.178

0.168

0.469

0.370

0.146

0.379

0.223

0.436

0.531

0.614

0.960

Continuous supplementary Use Intention (CUI)

-0.328

0.347

0.507

0.353

0.194

0.195

0.167

0.358

0.583

0.647

0.645

0.877

Pure Use Intention (PUI)

0.005

0.091

0.147

0.058

-0.024

0.058

0.094

0.148

0.324

0.467

0.351

0.431

PUI

0.943

131

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Table 5. Model Evaluation & Paths Analysis Construct

CA

TE

SQ

IQ

SvQ

PEU (0.424)

-0.340 (h1a)

0.212 (h2a)

0.173 (h4a)

C

0.068 (h5a)

0.150 (h6a)

0.106 (h7a)

0.006 (h8a)

0.182 (h9a)

(0.522) (PU)

-0.033 (h1b)

0.029 (h2b)

0.508G (h4b)

0.056 (h5b)

0.008 (h6b)

0.014 (h7b)

0.104B (h8b)

0.002 (h9b)

0.300G (h10c)

USE (0.580)

-0.088A (h1c)

0.003 (h2c)

0.051 (h4c)

0.115 B (h5c)

0.012 (h6c)

0.203G (h7c)

0.001 (h8c)

0.238G (h9c)

0.139E (h10a)

0.324G (h10b)

CUI (0.553)

0.238F (h11a)

0.320 F (h11b)

0.322G (h11c)

PUI (0.229)

0.077 (h12a)

0.376G (h12b)

0.080 (h12c)

G

E

C

MS

IN

TR

PEU

PU

USE

D

A. Significant at 95% |B. Significant at97.5% |C. Significant at 99% |D. Significant at99.5% E. Significant at 99.75% | F. Significant at 99.9% | G: Significant at99.95%

anxiety negatively impacts their perceived ease of use (Beta -β = -0.340, p= 0.0005, hypothesis h1a) and supplementary use of LMS (β = -0.088, p= 0.05; h1c). Second, instructors’ experience with the technology only significantly impacts their perceived ease of use (β = 0.212, p = 0.0025; h2a). Third, the study was unable to assess the impact of instructors’ self efficacy on their use and acceptance of LMS because of the low reliability and validity of the construct (self-efficacy) measures; thus, hypotheses H3a, H3b, H3c were not tested. Fourth, system’s quality significantly impacts instructors’ perceived ease of use (β = 0.173, p = 0.01; h4a); and perceived usefulness (β = 0.508, p= 0.0005; h4b) of LMS. Fifth, information quality significantly impacts instructor’s actual use of LMS (β = 0.149, p= 0.025; h5c). Sixth, service quality significantly impacts only instructors’ perceived ease of use of LMS (β = 0.150, p = 0.01; h6a). Seventh, management support significantly impacts instructor’s actual use of LMS (β = 0.203, p=0.0005; h7c). Eight, incentives policy significantly impacts instructors’ perceived usefulness of LMS (β = 0.104, p= 0.025; h8b). Ninth, training significantly impacts the instructors’ perceived ease of use (β = 0.182, p = 0.005; h9a) and actual use of LMS (β = 0.238, p= 0.0005; h9c). Tenth, instructors’ perceived ease of use (β = 0.139, p= 0.0025; h10a) and perceived

132

usefulness (β of 0.324, p-value= 0.0005; h10b) of LMS positively associated with their actual use of LMS, and instructors’ perceived ease of use of LMS positively associated with their perceived usefulness of LMS (β = 0.300, p= 0.0005; ; h10c). Eleventh, instructors’ perceived ease of use of LMS (β = 0.238, p = 0.001; h11a), perceived usefulness (β = 0.320, p = 0.001; h11b), and current actual use (β = 0.322, p = 0.0005; h11c) are positively associated with their continuous supplementary use intention. Twelfth, instructors’ perceived usefulness of LMS is only positively associated with their pure use intention of LMS for distance education (β = 0.376, p = 0.0005; h12b). Thus, hypotheses h1b, h2b, h2c, h4c, h5a, h5b, h6b, h6c, h7a, h7b, h8a, h8c, h9b, h12a and h12c were not significant as indicated in Table 5.

DISCUSSION & CONCLUSION Discussion of Findings and Implications LMS include several tools that provide academic and training institutions an efficient and effective means to support distance education and supplement their traditional teaching. Moreover, LMS enable these institutions to capture their

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

educational materials and preserve them for future reuse. This study examined the impact of instructors’ characteristics (computer anxiety and technology experience); LMS’ characteristics (system quality, information quality, and service quality); and an organization’s characteristics (management support, incentives, and training) on the acceptance of e-learning (supplementary use, perceived usefulness, and perceived ease of use) and, consequently, their future intention of supplementary use and pure use of LMS for distance education in an academic institution. The results showed that instructors’ individual characteristics, LMS’ characteristics, and an organization’s characteristics have various impacts on instructors’ use of LMS, their perceived usefulness of LMS, and their perceived ease of use. First, regarding the instructors’ individual characteristics, the study found that instructors’ computer anxiety impacts their perceived ease of use of LMS and their actual use; whereas instructors’ technology experience impacts their perceived ease of use of LMS. Second, regarding the LMS’s characteristics, the study found system quality impacts instructors’ perceived ease of use and perceived usefulness of LMS, information quality impacts instructors’ actual use of LMS, whereas, service quality impacts instructors’ perceived ease of use of LMS. Third, regarding the organization’s characteristics, the study found that management support impacts instructors’ actual use of LMS, incentives policy impacts the instructors’ perceived usefulness of LMS, whereas training impacts instructors’ perceived ease of use of LMS. In other words, instructors’ perceived ease of use of LMS is determined by instructors’ characteristics (computer anxiety and technology experience) and an organization’s characteristics of training, and LMS’ system and service quality. Instructors’ perceived usefulness of LMS is impacted by system quality, its perceived ease of use, and incentives policy; thus, none of the instructors’ individual characteristics has a direct significant impact on their perceived usefulness

of LMS. Instructors’ use of LMS is impacted by a combination of several factors: their perception of LMS (perceived usefulness of LMS and perceived ease of use), organizational characteristics (training and management support), LMS characteristics (information quality), and instructors’ individual characteristic (computer anxiety). Other factors impact system use indirectly through perceived usefulness and perceived ease of use. Furthermore, the results indicated that instructors’ continuous intention of LMS supplementary use is determined by their current use, perceived usefulness, and perceived ease of use, while future intention of LMS pure use for distance education is determined only by their perceived usefulness of LMS. LMS is promising for developing countries, as they provide tools to efficiently build human resources. This study offered significant findings for researchers and practitioners. First, it comprehensively examined the critical factors influencing instructors’ use of LMS. The study demonstrates that the use of LMS is determined by users’ individual characteristics, LMS characteristics, and an organization’s characteristics. Thus, users, technology, and an organization’s characteristics have various and significant impacts on instructors’ acceptance of LMS. Few studies have examined instructors’ acceptance of LMS; the literature also lacks the investigation of organizational characteristics on LMS acceptance. Second, the study shows that investigating these direct impacts on LMS use is worthwhile. Third, the study shows that although instructors currently use LMS, the perceived ease of use and perceived usefulness of LMS impact their continuous intention to use LMS for supplementary purposes; only their perceived usefulness of LMS impacts their intention to use LMS purely for distance education. Few studies have examined the link between instructors’ use of LMS as a supplementary and a pure tool for distance education. The study also provided useful insights for practitioners (instructors and academic institutions). Organizations, especially in the Middle

133

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

East where computer and Internet literacy is not as high as in developed countries, should provide training to improve users’ technology experience and lessen their computer anxiety. In addition, organizations should adopt high-quality LMS (in terms of system quality, information quality, and service quality) to promote their adoption and use by instructors. In addition, management support and incentives are important to improve instructors’ use of LMS and their perceived usefulness, respectively. The findings suggest that instructors’ computer anxiety plays a significant negative role on their perceived ease of use and actual use of LMS, therefore, providing a good training and service quality are necessary to improve instructor’s perceived ease of use of LMS. Also, organizations should ensure that management support is in place to encourage instructor’s use of LMS. Furthermore, since instructors’ intention to pure e-learning depends on their perceived usefulness of LMS, ensuring high system quality and attractive incentives policies, as they are determinants of perceived usefulness, are vital for pursuing a pure e-learning for distance education.

Limitations and Future Research This study has few limitations. First, the sample was from one academic institution in Oman and one learning management system; more research can be conducted in several organizations using different LMS in different countries to improve the generalization of the findings. Second, the study assessed LMS usage from instructors’ perspective; further research may assess learners’ and organizations’ acceptance. Third, this study was unable to assess the impact of self-efficacy; new measurements might be developed to improve its reliability and validity across different countries. Moreover, future research could also examine in detail the benefits of LMS for instructors and the critical factors influencing organizations’ deployment of LMS.

134

ACKNOWLEDGMENT This study was granted by Sultan Qaboos University in Oman

REFERENCES Aczel, J. C., Peake, S. R., & Hardy, P. (2008). Designing capacity-building in e-learning expertise: Challenges and Strategies. Computers & Education, 50, 499–510. doi:10.1016/j. compedu.2007.07.005 Al-Busaidi, K. A. (2009). Evaluating the use of e-learning system as a supplementary tool at an academic institution. International Journal of Internet Education, 4, 153. Albirini, A. (2006). Teachers’ attitudes toward information and communication technologies: The case of Syrian EFL teachers. Computers & Education, 47, 373–398. doi:10.1016/j.compedu.2004.10.013 Ali, A. J. (1990). Management theory in a transitional society: The Arab’s experience. International Studies of Management and Organization, 20(3), 7–35. Bailey, J., & Pearson, S. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29(5), 530–545. doi:10.1287/mnsc.29.5.530 Ball, D., & Levy, Y. (2008). Emerging educational technology: Assessing the factors that influence instructors’ acceptance in information systems and other classrooms. Journal of Information Systems Education, 19(4), 431–443. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioural change. Psychological Review, 84(2), 191–215. doi:10.1037/0033295X.84.2.191

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Barclay, D., Higgins, C., & Thompson, R. (1995). The partial least squares (PLS) approach to casual modelling: Personal computer adoption and use as an illustration (with commentaries). Technology Studies, 2(2), 285–309.

Chin, W. (1998). The partial least square approach to structural equation modelling. In Marcoulides, G. A. (Ed.), Modern Methods for Business Research (pp. 295–336). London, UK: Lawrence Erlbaum Associates.

Beatty, B., & Ulasewicz, C. (2006). Online teaching and learning transition: Faculty perspectives on moving from blackboard to the Moodle learning management system. TechTrends, 50(4), 36–45. doi:10.1007/s11528-006-0036-y

Chin, W. (2001). PLS Graph user’s guide, version 3.0. Houston, TX: Bauer College of Business, University of Houston.

Blackboard. (2009). Engaging learners for engaging learning. Retrieved on September 7th, 2009, from http://www.blackboard.com/TeachingLearning/Learn-Platform.aspx. Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177–192. doi:10.1080/10494820600852795 Burnett, R., & Marshall, D. P. (2003). Web theory: An introduction. New York, NY: Routledge Press.

Chin, W., & Newsted, P. R. (1999). Structural equation modeling analysis with small samples using partial least squares. In Hoyle, R. H. (Ed.), Statistical strategies for small sample research (pp. 307–341). Thousand Oaks, CA: Sage Publications. Compeau, D., & Higgins, C. (1995). Computer self-efficacy: Development of a measure and initial test. Management Information Systems Quarterly, 19, 189–211. doi:10.2307/249688 Crano, W., & Brewer, M. (2002). Principles and methods of social research. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

Burniske, R. W., & Monke, L. (2001). Breaking down the digital walls. Albany, NY: State University of New York Press.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13, 319–339. doi:10.2307/249008

Cavus, N., & Momani, A. M. (2009). Computer aided evaluation of learning management systems. Procedia Social and Behavioral Sciences, 1, 426–430. doi:10.1016/j.sbspro.2009.01.076

DeLone, W., & McLean, E. (2003). The Delone and Mclean model of Information Systems success: A 10-year update. Journal of Management Information Systems, 19(4), 9–30.

Ceraulo, S. (2005). Benefits of upgrading to an LMS. Distance Education Report, 9, 6–7.

DeLone, W. H., & McLean, E. R. (1992). Information Systems success: The quest for the dependent variable. Information Systems Research, 3, 60–95. doi:10.1287/isre.3.1.60

Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, J. H. (2008). A virtual learning environment for part-time MASW students: An evaluation of the WebCT. Journal of Teaching in Social Work, 28, 87–100. doi:10.1080/08841230802179027 Chau, Y., Patrick, K., & Hu, J. (2001). Information Technology acceptance by individual professionals: A model comparison approach. Decision Sciences, 32(4), 699–718. doi:10.1111/j.1540-5915.2001.tb00978.x

Doll & Torkzadeh, G. (1988). The measurement of end user computing satisfaction, IS Quarterly, 12(2), 259-274. El Tartoussi, I. (2009). Networked readiness in the United Arab Emirates, The 2nd Annual Forum on E-learning Excellence in the Middle East 2009, Dubai. UAE.

135

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Falvo, D. A., & Johnson, B. F. (2007). The use of learning management systems in the United States. TechTrends, 51(2), 40–45. doi:10.1007/ s11528-007-0025-9

Kneght, M., & Reid, K. (2009). Modularizing Information Literacy training via the Blackboard eCommunity. Journal of Library Administration, 49, 1–9. doi:10.1080/01930820802310502

Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error. JMR, Journal of Marketing Research, 18, 39–50. doi:10.2307/3151312

Lee, Y. (2006). An empirical investigation into factors influencing the adoption of an e-learning system. Online Information Review, 30(5), 517– 541. doi:10.1108/14684520610706406

Hair, J., Anderson, R., Tatham, R., & Black, W. (1998). Multivariate data anlaysis. Upper Saddle River, NJ: Prentice Hall.

Liaw, S., Huang, H., & Chen, G. (2007). Surveying instructor and learner attitudes toward elearning. Computers & Education, 49, 1066–1080. doi:10.1016/j.compedu.2006.01.001

Hawkins, B. L., & Rudy, J. A. (2007). Educause Core Data Service. Fiscal Year 2006 Summary Report. CO, USA: Educause. Hayashi, A., Chen, C., Ryan, T., & Wu, J. (2004). The role of social presence and moderating role of computer self efficacy in predicting the continuance usage of e-learning systems. Journal of Information Systems Education, 15(2), 139–154. Igbaria, M. (1990). End-user computing effectiveness: A structural equation model. Omega, 18(6), 637. doi:10.1016/0305-0483(90)90055-E Igbaria, M., Guimaraes, T., & Davis, G. B. (1995). Testing the determinants of microcomputer usage via a structural equation model. Journal of Management Information Systems, 11(4), 87–114. Kelly, T., & Bauer, D. (2004). Managing intellectual capital-via e-learning-at Cisco. In Holsapple, C. (Ed.), Handbook on Knowledge Management 2: Knowledge directions (pp. 511–532). Berlin, Germany: Springer. Kettinger, W. J., & Lee, C. C. (1994). Perceived service quality and user satisfaction with the Information Services function. Decision Sciences, 25(5/6), 737–765. doi:10.1111/j.1540-5915.1994. tb01868.x

136

Mahdizadeh, H., Biemans, H., & Mulder, M. (2008). Determining factors of the use of e-learning environments by university teachers. Computers & Education, 51, 142–154. doi:10.1016/j. compedu.2007.04.004 Naidu, S. (2006). E-learning a guidebook of principles, procedures and practices, 2nd revised edition, Commonwealth Educational Media Center for Asia (CEMCA), New Delhi, India. National Centre for Educational Statistics. (2003). Distance education at degree-granting postsecondary institutions: 2000-2001 U. Washington, DC: S. Department of Education. Parasuraman, A., Zeithaml, V. A., & Berry, L. (1988). SERVQUAL: A multiple-item scale for measuring customer perceptions of service quality. Journal of Retailing, 64(1), 12–40. Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4), 134–148. Piccoli, G., Ahmad, R., & Ives, B. (2001). Webbased virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skill training. Management Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Pituch, K., & Lee, Y. (2006). The influence of system characteristics on e-learning use. Computers & Education, 47, 222–244. doi:10.1016/j. compedu.2004.10.007 Raaij, E., & Schepers, J. (2008). The acceptance and use of a virtual learning environment in China. Computers & Education, 50, 838–852. doi:10.1016/j.compedu.2006.09.001 Rainer, R. K., Turban, E., & Potter, R. E. (2007). Introduction to Information Systems: Supporting and transforming business. NJ: Wiley. Reiser, R. A., & Dempsey, J. V. (Eds.). (2002). Trends and issues in instructional design and technology. Upper Saddle River, NJ: Pearson Education. Roca, J., Chiu, C., & Martinex, F. (2006). Understanding e-learning continuous intention: An extension of the technology acceptance model. International Journal of Human-Computer Studies, 64, 683–696. doi:10.1016/j.ijhcs.2006.01.003 Saady, A. (2005). E-learning curve, ITP Technology, Retrieved from www.ITP.net. Schein, E. (1985). Organizational culture and leadership. San Francisco, CA: Jossey-Bass. Seddon, P. B. (1997). A respecification and extension of the DeLone and McLean model of IS success. Information Systems Research, 8(3), 240–253. doi:10.1287/isre.8.3.240 Simonson, M., Maurer, R., Montag-Torardi, M., & Whitaker, M. (1987). Development of a standardized test of computer literacy and a computer anxiety index. Journal of Educational Computing Research, 3(2), 231–247. doi:10.2190/7CHY5CM0-4D00-6JCG Sumner, M., & Hostetler, D. (1999). Factors influencing the adoption of technology in teaching. Journal of Computer Information Systems, 40(1), 81.

Sun, P., Tsai, R., Finger, G., Chen, Y., & Yeh, D. (2008). What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50, 1183–1202. doi:10.1016/j. compedu.2006.11.007 Teo, T. (2009). Modelling technology acceptance in education: A study of pre-service teachers. Computers & Education, 52, 302–312. doi:10.1016/j. compedu.2008.08.006 Thompson, R., & Compeau, R., Deborah, & Higgins, C. (2006). Intentions to use Information Technologies: An integrative model. Journal of Organizational and End User Computing, 18(3), 25–47. doi:10.4018/joeuc.2006070102 Venkatesh, V., & Davis, F. (2000). A theoretical extension of the technology acceptance model: Four longitudinal. Management Science, 46(2), 186–204. doi:10.1287/mnsc.46.2.186.11926 Wan, Z., Fang, Y., & Neufeld, H. (2007). The role of Information Technology in technologymediated learning: A review of the past for the future. Journal of Information Systems Education, 18(2), 183–192. Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40(6), 1282. doi:10.2307/257034 Woods, R., Baker, J., & Hopper, D. (2004). Hybrid structure: Faculty use and perception of Webbased courseware as a supplement to face-to-face instruction. The Internet and Higher Education, 7(4), 281–297. doi:10.1016/j.iheduc.2004.09.002 World Bank. (2010). Learning management system. World Bank Institute. Retrieved on April 7th, 2010, from http://web.worldbank.org

137

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Yildirim, S., Temur, N., Kocaman, A., & Goktas, Y. (2004). What makes a good LMS: An analytical approach to assessment of LMS. In Proceedings of the 5th International Conference on Information Technology Based Higher Education and Training, Istanbul, Turkey (pp. 125–130). Yueh, H.-P., & Hsu, S. (2008). Designing a learning management system to support instruction. Communications of the ACM, 51(4), 59–63. doi:10.1145/1330311.1330324

ADDITIONAL READING Aggarwal, A., & Bento, R. (2000). Web-based education. In Aggarwal, A. (Ed.), Web-based learning and teaching technologies: Opportunities and challenges (pp. 1–16). London, UK: Idea Group Publishing. Al-Busaidi, K., & Al-Shihi, H. (2010). Instructors’ acceptance of learning management systems: A theoretical framework. Communications of IBIMA [Open Source Journal], Vol. 2010, Article ID 862128, 10 pages. Al-Rawajfih, K., Fong, S., & Idros, S. (2010). Effects of principals’ support on teachers’ selfefficacy in integrating e-learning in the Jordanian Discovery Schools. Modern Applied Science, 4(9), 147–151. Aydin, C. C., & Tirkes, G. (2010). Open source learning management systems in e-learning and Moodle. In Proceedings of Education Engineering (EDUCON), 2010 (pp. 593–600). IEEE. Bailey, C. J., & Card, K. A. (2009). Effective pedagogical practices for online teaching: Perception of experienced instructors. The Internet and Higher Education, 12(3–4), 152–155. doi:10.1016/j. iheduc.2009.08.002

138

Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, J. H. (2008). A virtual learning environment for part-time MASW students: An evaluation of the WebCT. Journal of Teaching in Social Work, 28, 87–100. doi:10.1080/08841230802179027 Garrison, R., & Vaughan, H. (2008). Blended learning in higher education: Framework, principles and guidelines. San Francisco, CA: Jossey-Bass. Ge, X., Lubin, I. A., & Zhang, K. (2010). An investigation of faculty’s perceptions and experiences when transitioning to a new learning management system. Knowledge Management & E-Learning: An International Journal, 2(4), 43–62. Graf, S., Kinshuk, & Liu, T.-C. (2009). Supporting teachers in identifying students’ learning styles in learning management systems: An automatic student modelling approach. Journal of Educational Technology & Society, 12(4), 3–14. Hu, P. J., Chau, P. Y., Clark, T. H., & Ma, W. W. (2003). Examining technology acceptance by school teachers: A longitudinal Study. Information & Management, 41, 227–241. doi:10.1016/ S0378-7206(03)00050-8 Kim, S. W., & Leet, M. G. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24, 284–294. doi:10.1111/j.1365-2729.2007.00260.x Kluever, R. C., Lam, T. C. M., & Hoffman, E. R. (1994). The computer attitude scale: Assessing changes in teachers’ attitudes toward computers. Journal of Educational Computing Research, 11(3), 251–256. doi:10.2190/484T-CPGXEUHG-QW8P Kritikou, Y., Demestichas, P., Adamopoulou, E., Demestichas, K., Theologou, M., & Paradia, M. (2008). User profile modeling in the context of Web-based learning management systems. Journal of Network and Computer Applications, 31, 603–627. doi:10.1016/j.jnca.2007.11.006

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd ed.). New York, NY: Routledge/Falmer. doi:10.4324/9780203304846

Russell, M., Bebell, D., O’Dwyer, L., & O’Connor, K. (2003). Examining teacher technology use: Implications for preservice and inservice teacher preparation. Journal of Teacher Education, 54(5), 297–310. doi:10.1177/0022487103255985

Lonn, S., & Teasley, S. D. (2009). Saving time in innovating practice: Investigating perceptions and uses of learning management systems. Computers & Education, 53, 686–694. doi:10.1016/j. compedu.2009.04.008

Selim, H. M. (2007). Critical success factors for e-learning acceptance: Confirmatory factor models. Computers & Education, 49, 396–413. doi:10.1016/j.compedu.2005.09.004

Martin, K., Quigley, M. A., & Rogers, S. (2010). Implementing a learning management system globally: An innovative change management approach. IBM Systems Journal, 44(1), 125–145. doi:10.1147/sj.441.0125 McGill, T., Klobas, J., & Renzi, S. (2011). LMS use and instructor performance: The role of task-technology fit. International Journal on ELearning, 10(1), 43–62. Moore, M. G. (1989). Three types of interaction. American Journal of Distance Education, 3(2), 1–6. doi:10.1080/08923648909526659 Nagar, S. (2010). Study of learning management systems and its effects on distance education. International Journal of Educational Administration, 2(2), 323–327. Ngai, E., Poon, J., & Chan, Y. (2007). Empirical examination of the adoption of WebCT using TAM. Computers & Education, 48(2), 250–267. doi:10.1016/j.compedu.2004.11.007 Ocak, M. (2011). Why are faculty members not teaching blended courses? Insights from faculty members. Computers & Education, 56, 689–699. doi:10.1016/j.compedu.2010.10.011 Ramayah, T., Ahmad, N., & Lo, M. (2010). The role of quality factors in intention to continue using an e-learning system in Malaysia. Procedia - Social and Behavioral Sciences, 2(2) 5422-5426.

Teo, T. (2011). Modeling the determinants of pre-service teachers’ perceived usefulness of e-learning. Campus-Wide Information Systems, 28(2). doi:10.1108/10650741111117824 Toral, S. L., Barrero, F. J., Torres, M. R., Gallardo, S., & Lillo, A. J. L. (2005). Implementation of a Web-based educational tool for digital signal processing teaching using the technological acceptance model. IEEE Transactions on Education, 48(4), 632–641. doi:10.1109/TE.2005.853074 Tucker, A. (2010). Effective practices in e-learning: An online instructor and learner perspective. In J. Sanchez & K. Zhang (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2010 (p. 1251). Volery, T., & Lord, D. (2000). Critical success factors in online education. International Journal of Educational Management, 14(5), 216–223. doi:10.1108/09513540010344731 Waheed, M., & Jam, A. (2010). Teacher’s intention to accept online education: Extended TAM Model. Interdisciplinary Journal of Contemporary Research in Business., 2(5), 330–344. Wang, S.-C. (2010). University instructor perceptions of the benefits of technology use in e-learning. In Proceedings of Computer and Electrical Engineering, (ICCEE ’09). 2nd International Conference (pp. 580-585).

139

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

Wang, W., & Wang, C. (2009). An empirical study of instructor adoption of Web-based learning systems. Computers & Education, 53(3), 761–774. doi:10.1016/j.compedu.2009.02.021 Zhu, C., Valcke, M., & Schellens, T. (2009). A crosscultural study of online collaborative learning. Multicultural Education & Technology Journal, 3(1), 33–46. doi:10.1108/17504970910951138

KEY TERMS AND DEFINITIONS Learning Management System: A system that provides a platform to support e-learning activities such as communication, collaboration, learning and information/ knowledge transfer.

140

Supplementary LMS Use: The use of learning management system tools to support the traditional face-to-face meetings. Pure LMS Use: The use of learning management system for pure e-learning environment. LMS Success Factors: Factors that determines the success of LMS deployment LMS’s Characteristics: Factors that are related to the system quality, information quality and service support quality of LMS. Instructor’s Characteristics: Factors that are related to the instructor’s individual characteristics such as self efficacy, computer anxiety, technology experience etc. Organization’s Characteristics: Factors that are related to the organization such as management support, training programs and incentives policies.

Section 3

Trends and Challenges

142

Chapter 7

A Comparative Study on LMS Interoperability José Paulo Leal CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal

ABSTRACT A Learning Management System (LMS) plays an important role in any eLearning environment. Still, the LMS cannot afford to be isolated from other systems in an educational institution. Thus, the potential for interoperability is an important, although frequently overlooked, aspect of an LMS system. In this chapter we make a comparative study of the interoperability features of the most relevant LMS in use nowadays. We start by defining a comparison framework, with systems that are representative of the LMS universe, and interoperability facets that are representative of the type integration with other broad classes of eLearning systems. For each interoperability facet we categorize and identify the most representative remote systems, we present a comprehensive survey of existing standards and we illustrate with concrete integration scenarios. Finally, we draw some conclusions on the status of interoperability in LMS based on our study.

INTRODUCTION Interoperability is the ability of different computer systems, applications or services to communicate, share and exchange data, information and knowledge in a precise, effective and consistent way (Martínez & Navarra, 2007). In the eLearning DOI: 10.4018/978-1-60960-884-2.ch007

field this topic is extremely important since there is the need for all systems that typically compose an eLearning environment to communicate and share data consistently. The LMS plays a central role in any eLearning architecture. Choosing an LMS can be a challenging task for an organization. Several studies have been conducted to analyse and evaluate these types of systems from pedagogical and in-

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

A Comparative Study on LMS Interoperability

stitutional perspectives (Pantel, 1997; Britain & Liber 1998). However, we are not aware of any study to evaluate the interoperability of LMS with other systems typically found in an educational institution. A major issue in LMS interoperability is the eLearning standardization. The concept of course, student, educational resource, summary or grade must be formally described in order to be shared among all the systems in an educational institution. For instance, the difficulty to reuse of a course in schools with LMS from different vendors (or even from the same vendor) is an apt example of the problems found currently in the majority of the LMS. These interoperability issues affect the flexibility of the teaching-learning process and lead to a decrease of end user satisfaction and learning success. In this chapter we make a comparative study of the LMS support for interoperability. This study is part of an effort to select an LMS on which to base the development of eLearning systems integrating heterogeneous components. We chose two LMS vendors - Moodle and Blackboard - since combined they a have a significant share of the LMS market and they follow different approaches to LMS development, namely open source and commercial. We analyse the interoperability features in these LMS split in two facets reflecting the broad classes of systems of a typical LMS operational environment. These broad classes are Learning Content Management Systems and Academic Management Systems. This chapter starts by tracing the evolution of LMS. We proceed with the selection of the systems representative of the LMS universe and of a methodology for comparing them based in interoperability facets. The following two sections analyse separately the learning management content facet and the academic management facet. For each facet we categorize and identify the most representative system, the existing standards and the interoperability issues regarding the communication with the LMS. In the final section we draw conclusions on the results of this study.

LMS EVOLUTION The evolution of eLearning in the last decades has staggering, from the early monolithic systems developed for specific learning domains to new systems featuring reusable tools that can be effectively used virtually in any eLearning course. These types of systems evolved from Content Management Systems (CMS). The CMS was introduced in the mid-1990s mostly by the online publishing industry. This type of system can be defined as a data repository that also includes tools for authoring, aggregating and sequencing content. The main goal of these tools is to simplify the creation and administration of online content (Nichani, 2009). CMS are focused on content with the main purpose to store information and provide access to it. CMS content is organized in small self-contained pieces of information to improve reusability at the content component level. These content components when used in the learning domain are called “learning objects” (LO) and the systems that manage them are called Learning Content Management Systems (LCMS). Nowadays, an LMS plays a central role in any eLearning architecture and can be defined as software application for the administration, documentation, tracking, reporting of training programs, classroom and online events, and training content (Ellis, 2009). Typically it is used by two types of users’ groups: learners and teachers. The learners can use the LMS to plan their learning experience and to collaborate with their colleagues; the teachers can deliver educational content and track, analyze and report the learner evolution within an organization. There are open source systems, such as Moodle, Sakai, .LRN or Dokeos, and commercial systems such as WebCT/ Blackboard or Desire2Learn. They all feature general tools for delivering content and for recreating a learning context. From a course/discipline perspective they provide tools for handling assignments, managing chat rooms and forums, evaluating multiple-choice tests and quizzes, among others. From a learners’ manage143

A Comparative Study on LMS Interoperability

ment perspective they provide tools for keeping grade books, managing groups of students, and browsing logs. Ashford-Rowe and Malfroy (2009) organize these tools in four groups, namely: •

•

• •

Content - Unit/Course online, Lecture and Tutorial notes, Media (i.e. lectopia, podcast, videocasts), links to scholarly information (readings), links to content resources (i.e. websites), interactive resources (.swf .fla .flv and other file types); Communication - Chat, Announcements, Discussion Board, Email, Blogs and Forums; Collaboration - Wikis, Virtual Classroom and Voice-based communication; Assessment - Quizzes, Reflective learning journals, Portfolios, Grades, Surveys, Practice activities and past exams.

Recently the eLearning community started valuing more the interchange of course content and learners’ information, which led to the definition of standards for eLearning content sharing and interoperability. Standards can be viewed as “documented agreements containing technical specifications or other precise criteria to be used consistently as guidelines to ensure that materials and services are fit for their purpose” (Nichani, 2009). In the eLearning context, standards are generally developed for the purposes of ensuring interoperability and reusability in systems and of the content and meta-data they manage. In this context, several organizations (e.g. IMS GLC, IEEE, ISO/IEC, ADL) are developing specifications and standards (e.g. IMS CP, IMS CC, IMS DRI, LOM, SCORM) in the last years (Dagger & O’Connor & Lawless & Walsh & Wade, 2007). These specifications are closely related with the learning object concept as context independent, transportable and reusable pieces of instruction that are digitally managed and delivered (Rehak & Mason, 2003). There are other definitions for Learning Objects (LO). Rehak & Mason (2003) define a learning object as: “a digitized entity 144

which can be used, reused or referenced during technology supported learning”. As every kind of software, LMS continue to evolve to meet market demands. In relation to interoperability the main trend for the next LMS generation is service-oriented architectures (SOA) (Dagger & O’Connor & Lawless & Walsh & Wade, 2007). In these architectures LMS expose their functions as services and consume services from their operational environments, improving their interoperability with other eLearning systems. In fact, the last few years brought us several initiatives (Smythe, 2003; Wilson & Blinco & Rehak, 2004) to adapt SOA to eLearning. These initiatives, commonly named eLearning frameworks, have the same goal: to provide flexible learning environments for learners worldwide. Usually they provide a set of open interfaces to numerous reusable services organized in genres or layers that can be combined in service usage models (Queirós & Leal, 2010). Other trends result from new market demands such as Web 2.0, Talent Management, Mobile Learning, “Software as a Service” and Open Source Software. With the recent appearance of Web 2.0 tools and the popularity of social networking tools like Facebook and Twitter, there has been a great demand to use similar tools in the LMS to enhance the communication among teachers and students. Talent Management software systems are an extension of traditional human resource management systems. Some researches (Bersin & Howard & O’Leonard & Mallon, 2009) shows that in 2009 more than 70% of large companies have an LMS already and almost 1/3 of these companies are considering replacing or upgrading these systems with integrated talent management systems (Levensaler & Laurano, 2009). With more students working at distance, there has been also a strong demand to make eLearning applications accessible through mobile devices (e.g. Smartphones, PDA) know as Mobile Learning or m-learning. Using LMS Software as a Service (SaaS) schools can relieve the financial burden of maintaining their LMS by outsourcing the host-

A Comparative Study on LMS Interoperability

ing service. Commercial LMS (e.g. Blackboard, WebCT) have dominated the education market in previous years, but as costs increase, schools and companies are now looking for other options such as open-source solutions (e.g. Moodle, Sakai) that are financially more attractive.

COMPARISON FRAMEWORK The goal of this work is to analyse and compare LMS interoperability features. Given the number of LMS vendors it would be impracticable to study them all. Therefore we selected two LMS that we consider representative of the LMS universe. This selection is based on their prominence in the LMS market and the fact that they cover the open source and commercial development models. Interoperability is in general a complex concept that can be analysed in multiples perspectives and this is surely the case with LMS. To organize our study we identified two broad classes of systems that usually integrate the operational environment of the LMS. Thus, we considered two facets in LMS interoperability, regarding communication and data sharing with these classes.

Learning Management Systems A good number of LMS that were developed in the past fifteen years are still in use and under active development. For the purpose of our study we must concentrate on a few systems that are representative of the LMS universe in terms of their characteristics and market share. A simple categorization of this type of systems is according to their development model. There are fundamentally two: open source systems, such as Moodle, Sakai, .LRN or Dokeos; and commercial systems such as WebCT/Blackboard or Desire2Learn. Figure 1 presents a timeline of the development of several initiatives grouped by their development model.

The Figure 1 shows that the first major LMS adopted a commercial development model but since the beginning of this century there has been a shift towards open source systems. In fact, this shift was already recognized as a trend in LMS development (Davis & Carmean & Wagner, 2009). In spite of the growing popularity of open source, commercial systems are still relevant and they must be included in any representative sample of LMS. In these two categories we decided to select the most popular systems taking as reference the available data on global LMS usage (Davis & Carmean & Wagner, 2009). As part of this study we conducted a survey on eLearning systems usage on Portuguese higher education institutions. We received responses from 20 different institutions and the results for LMS usage are shown in Figure 2. The two most popular LMS in these institutions follow the global trend, which reinforces our choice of the reference systems for our study. We decided to focus our study on Moodle and Blackboard. We chose them since they represent the two main development models used by LMS vendors (open source and commercial); and combined they have a significant share on the LMS market (33.2% on the international market (Davis & Carmean & Wagner, 2009) in 2009 and 80% on our own recent survey). The following paragraphs provide an overview of the selected systems. Moodle (version 1.9.9 - 8th June 2010) is a free and open-source LMS written in PHP and created by Martin Dougiamas. Its name is an acronym for Modular Object-Oriented Dynamic Learning Environment. In early January of 2010, Moodle had a user-base of 46,624 registered sites with 32,464,992 users in 3,161,291 courses in 209 countries and in more than 75 languages (Cole & Foster, 2007). The most common functions of Moodle are the course information and documentation, documents repository, announcements, synchronous and a synchronous communication

145

A Comparative Study on LMS Interoperability

Figure 1. Timeline of development of major LMS

Figure 2. LMS usage in Portuguese higher education institutions

146

A Comparative Study on LMS Interoperability

(email, chat room, discussion forum) and assignments. Blackboard (version 9.1 - 1th April 2010) was developed by Blackboard Inc. in 1997 and is an online proprietary virtual learning environment system that is used by over 3700 educational institutions in more than 60 countries. In February 2006, the virtual learning environment called WebCT (Course Tools) was acquired by Blackboard Inc. (Blackboard, 2005) and, as part of the acquisition terms, the Blackboard brand was assumed until now.

Interoperability Facets The interoperability features of a system reflect the operational environment where it is expected to be deployed. The operational environment of an LMS includes different systems and services with which it may have to communicate and exchange data. As depicted in Figure 3 we identified two broad classes of systems that usually integrate the operational environment of an

LMS, each corresponding to a different facet in LMS interoperability. We identified also a layer of infrastructural systems and services that are domain independent but that play an important role in LMS interoperability. For the purpose of this study, the broad classes of systems that we identified as part of the operational environment of an LMS are the following: Learning Content Management Systems (LCMS) are used for the development, management and publishing of digital learning content (e.g. Learning Objects) that the LMS delivers. Examples of these systems are the Learning Object Repositories, e-Portfolio Systems, Authoring Tools, Specialized Evaluators and others. Academic Management Systems (AMS) are used for managing academic data information of an educational institution. Typical features of these systems are the management of courses, classes and students, the enrolment of students in courses, the submission of summaries and grades by teachers, among others.

Figure 3. LMS interoperability facets

147

A Comparative Study on LMS Interoperability

Apart from these facets the LMS is supported by infrastructure services providing basic functions that are not specific to eLearning, such as directory services for authentication and authorization or printing services. We consider also as part of this infrastructure the web or application server, the database engine and the operative system. In many cases this infrastructural layer is used for implementing ad hoc interoperability solutions. In the following sections the selected systems are analysed and compared regarding these two facets. We categorize and identify the remote systems in each facet, the existent standards and the interoperability issues regarding LMS communication with those systems. There is a huge asymmetry among these facets and this is reflected in the structure of the following sections. Of the two facets, the first has a larger number of systems and mature standards. The systems in second facet are mostly home-grown with few and immature standards to regulate both content (e.g. academic records, course forms, grades, summaries) and communication.

LEARNING CONTENT MANAGEMENT FACET The Learning Content Management facet focuses on the interoperation with systems that provide pedagogical content and services delivered by the LMS. We start by identifying their main types, followed by the existing standards for content and communication, and ending with an example of system integration in this facet.

System Types The content delivered by an LMS can be created, obtained, gathered or evaluated in several types of systems such as Learning Objects Repositories, E-Portfolio systems, Authoring Tools, Specialized

148

Evaluators or Quizzes. In the following sub-subsections some of these system types are detailed.

Learning Object Repositories A repository of learning objects can be defined as a “system that stores electronic objects and meta-data about those objects” (Holden, 2004). The need for this kind of repositories is growing as more educators are eager to use digital educational contents and more of it is available. Learning object repositories can be organized in free (CAREO, POOL, CLOE, EducaNext, ARIADNE) and payed (Sentient Learning, Harvest Road Hive, Learn Exact, Luminas) (JORUM team, 2006). One of the best known repositories of LOs is Merlot (Multimedia Educational Resource for Learning and Online Teaching) which provides pointers to online learning materials and includes a search engine. The Jorum Team made a comprehensive survey (2006) of the existing repositories and noticed that most of these systems do not store actual LOs. They just store meta-data describing LOs, including pointers to their locations on the Web, and sometimes these pointers are dangling. Most of the current repositories are specialized search engines of LOs and with little support for interact with specialized eLearning systems, such as evaluation engines and experimentation environments. These systems require both complete interoperability and specific Metadata. They need service oriented repositories of learning objects, fully compliant with the existing interoperability standards, and supporting new definitions of learning objects for specialized domains. An example of a specialized repository of LO is crimsonHex (Leal & Queirós, 2009), the repository developed as part of the EduJudge project to act as a programming problem repository service to the Evaluation Engine (EE) and the LMS.

A Comparative Study on LMS Interoperability

ePortfolio Systems

Authoring Tools

An electronic portfolio is a digital collection of student work (artefacts) usually managed in ePortfolio systems and displayed for specific audiences and purposes. The ePortfolios systems usually include (or link) a repository where students organize their artefacts typically for the purpose of assessment. The benefits of an ePortfolio system in an educational institution are shared by students and teachers. Students are able to reflect on their educational experiences and showcase their work in a repository. Teachers are able to evaluate the student progress and provide concrete evidence of the students’ learning. Helen C. Barrett (Barret, 2008) organizes the ePortfolio tools in two categories: individual and institutional. Both are presented in Table 1. In the individual category we can use authoring tools to author portfolios offline (requires web server space to publish online) or web services to create online and publish a presentation portfolio allowing interactivity (Web 2.0). In the institutional category we can use a software-server where an institution installs on their own server to provide space for hosting portfolios or hosted services that an institution adopts (no server required) that host portfolios. In the survey we conducted on Portuguese high education institution no one indicated to be using an ePortfolio system. This fact allows us to conclude that the dissemination of these tools in the educational institutions, at least in Portugal, is still low.

The growing popularity of learning objects lead to the development of specialized editors supporting eLearning metadata. These tools, either open source, freeware or commercial, export the content to SCORM packages and other formats such as IMS CP, IMS CC, HTML, PPT, PDF and Flash. The most important authoring tools can be grouped by their development model such as: • • •

Open Source: eXe, Xerte, ScenariChain Opale and LOMPad; Freeware: Hot Potatoes, MyUdutu, MOS Solo, Reload and Courselab; Commercial: Camtasia, Captivate, QuizCreator, PPT2Flash, PowerQuizPoint.

The majority of the authoring tools support multiple application profiles. RELOAD is arguably the most mature of these projects and is available both as a standalone Java application and as an Eclipse IDE plugin. It supports a broad range of metadata formats but cannot be extended to support specialized formats. The SHAME project (2006) - Standardized Hyper Adaptable metadata Editor – stands out from the rest since it is actually a metadata editing and presentation framework for RDF metadata with support for all kind of metadata based on a previous mapping for the RDF syntax. Some of these tools are specialized in a certain type of multimedia format (e.g. Captivate for video) or activities (e.g. Hot Potatoes for quizzes) and are the best place for the users to create the respective content.

Table 1. ePortfolio tools by categories Individual

Institutional

Authoring tools

Web Services

Software – Server

Hosted Services

Mozilla Composer Dreamweaver Microsoft Office Adobe Acrobat Movie Maker

Google Docs Zoho Writer WikiSpaces

Elgg Mahara OSPI Moofolio, MyStuff (embedded in Moodle)

Digication iWebfolio Epsilen GoogleApps for Education

149

A Comparative Study on LMS Interoperability

Specialized Evaluators Examples of eLearning systems that provide content can be drawn from different domains. At the heart of a system with automatic evaluation resides an Evaluation Engine (EE). This is an apt example of a specialized eLearning service, performing a specific task and reusable in different scenarios. An EE can supply its services not only to LMS but also to other specialized application services, such as quizzes and contest management systems. Desktop based applications also fit in this approach. This model of combining specialized services can be extended to competitive learning in other domains such as business training, for instance. In this domain teachers use business simulation games to improve the strategic thinking and decision making skills students in particular areas (e.g. finances, logistics, and production). Through these simulations students compete among them, as they would in a real world companies. A business simulation service fulfils a role similar to that of the EE in programming exercises and it also requires a repository containing specialized LO describing simulations. An example of a evaluation engine is the UVA Online Judge EE (Regueras & Verdú & Castro & Pérez & Verdú, 2008), the EE developed as part of the EduJudge project to act as an evaluator of programming problems submitted by students.

Standards In this subsection we introduce several standards related to learning objects. We structured these standards in four groups: packaging, metadata, organization and communication.

Packaging Packaging is crucial to store eLearning material and reuse it in different systems. The most widely used content packaging format is the IMS Content

150

Packaging (IMS CP, 2007). An IMS CP learning object assembles resources and meta-data into a distribution medium, typically an archive in zip format, with its content described in a manifest file in the root level. The manifest file - named imsmanifest.xml -adheres to the IMS CP schema and contains the following sections: • • •

•

Metadata: describes the package as a whole; Organizations:describes the organization of the content within a manifest; Resources:contains references to resources (files) needed for the manifest and metadata describing these resources; Sub-manifests: defines sub packages.

The manifest uses another standard - the IEEE Learning Object Metadata (IEEE LOM, 2002) to describe the learning resources included in the package (c.f. Sub-subsection 4.2.2). Recently, IMS Global Learning Consortium proposed the IMS Common Cartridge (IMS CC, 2010) that adds support for several standards (e.g. IEEE LOM, IMS CP, IMS QTI, IMS Authorization Web Service) and its main goal is to shape the future regarding the organization and distribution of digital learning content.

Metadata The content of LO packages is described by metadata. Its purpose is to support the interoperability and reusability of learning objects. As mentioned previously, the IMS CP manifest contains four sections and is precisely metadata that provides an overall description of the package. Metadata can be used to describe file features in the Resource section. In the manifest the Metadata element is used at two levels: package (overall description of the package) and resource (description of the resource and contained files). In both cases metadata information usually follows the IEEE LOM schema. The IEEE LOM is a data model used to

A Comparative Study on LMS Interoperability

describe a learning object. The model is organized in several categories that cover general data, such as title and description, technical data such as object sizes, types and durations, educational characteristics and intellectual property rights, among many others. These categories are very comprehensive and cover many facets of a LO. However, LOM was designed for general LO and does not to meet the requirements of specialized domains. For instance, there is no way to assert the role of specific resources. Fortunately, IMS CP was designed to be straightforward to extend through the creation of application profiles. The term Application Profile generally refers to “the adaptation, constraint, and/ or augmentation of a metadata scheme to suit the needs of a particular community”. A well know eLearning application profile is the Sharable Content Object Reference Model (SCORM, 2009) that extends IMS CP with more sophisticated sequencing and Contents-to-LMS communication. The IMS GLC is also responsible for another application profile, the Question & Test Interoperability (QTI) specification. QTI describes a data model for questions and test data and, since version 2.0, extends the LOM with its own metadata vocabulary. QTI was designed for questions with a set of pre-defined answers, such as multiple choice, multiple response, fill-in-the-blanks and short text questions. There are other metadata specifications, such as, the Dublin Core metadata, which provides a simpler and a more loosely-defined set of elements useful for sharing metadata across heterogeneous systems. At the present, the Dublin Education Working Group is extending the Dublin Core for the specific needs of the education community.

Organization Learning objects can be organized in items and an organization defines a path through those items. The IMS CP specification includes a manifest section called Organizations. This section can be

used to design pedagogical activities and articulate the sequencing of instructions. By default, it uses a tree-based organization of learning items pointing to the resources (assets) included in the package. However, other standards could be accommodated in this section, such as IMS Simple Sequencing (IMS SS) and IMS Learning Design (IMS LD). These specifications aims to provide to the teachers mechanisms for coordination of the educational instructions based on students’ profile making the instruction more dynamic and flexible. The IMS LD specification is a meta-language for describing pedagogical models and educational goals. Several IMS LD-aware tools are available as players (e.g. CopperCore, .LRN) and authoring/ export tools (e.g. Reload, LAMS). The IMS SS is a specification used to describe paths through a collection of learning activities. The specification declares the order in which learning activities are to be presented to a learner and the conditions under which a resource is delivered during an eLearning instruction. Despite all these specifications, the design of more complex adaptive behaviour is still hard to achieve.

Communication The standardization of the learning content it is not enough to ensure interoperability, which is a major user concern with the existing systems. The definition of common protocols and interfaces for the communication among systems is also an issue that the major eLearning interoperability initiatives (e.g. NSDL, POOL, OKI, EduSource, IMS) try to address. As an illustration we present the communication guidelines defined by IMS, arguably the most developed ones in this category. The IMS Learning Tools Interoperability (IMS LTI) provides a uniform standards-based extension point in LMS allowing remote tools and content to be integrated into LMS. The main goal of the LTI is to standardize the process for building links between learning tools and the LMS. The LTI has 3 key concepts (Gilbert, 2009): the Tool Provider,

151

A Comparative Study on LMS Interoperability

the Tool Consumer and the Tool Profile. The Tool Provider is a learning application that runs in a container separate from the LMS. Publishes one or more tools through the Tool Profiles. The Tool Profile is an XML descriptor that describes how a tool integrates with a tool consumer. It is composed by information about the tool metadata, vendor information, resource and event handlers and menu links. The Tool Consumer publishes a Tool Consumer Profile (XML descriptor of the Tool Consumer’s supported LTI functionality that is read by the Tool Provider during deployment), provides a Tool Consumer Runtime and exposes the LTI services. The IMS Digital Repositories Interoperability (IMS DRI) specification deals with the communication with a specific eLearning system: the repository. Within eLearning, repositories are used to store, manage and share LO. One of such efforts was the IMS Digital Repositories (IMS DRI). The IMS DRI specification was created by the IMS Global Learning Consortium (IMS GLC) and provides a functional architecture and reference model for repository interoperability. The IMS DRI provides recommendations for common repository functions, namely the submission, search and download of LOs. It recommends the use of web services to expose the repository functions based on the Simple Object Access Protocol (SOAP) protocol, defined by W3C. Due to their growing popularity other web service interface flavours, such as Representational State Transfer (REST) (Fielding, 2000), should be considered, since they are not excluded from the recommendation. This will improve interoperability with systems that adhere to a more informal style of development.

Integration In the majority of the cases an LMS integrates an organization infrastructure in conjunction with other systems. In the following sub-subsections we present the interoperability features of the refer-

152

ence LMS with the learning content management systems usually found in educational institutions. An integration example between a LMS and one of these systems is also presented.

State of Art The integration with eLearning content management systems can be implemented on the LMS data or business layer. In the former the integration uses the import / export features of both system and relies on the support of common formats. In the later the integration relies on the existence of compatible web services in both systems. Data integration is the simplest and most popular form of integration in content management. For instance, the RELOAD authoring tool can be used to create learning objects in SCORM format and Blackboard supports and imports SCORM packages. Table 2 lists some of the most important eLearning content standards and specifications defined in the last years by educational organizations. For each standard we present the LMS support status. The studied LMS support almost all the LO package standards with exception of the recent IMS CC that is only partially supported. In relation to the design and sequencing of learning activities standards are not yet supported by these LMS, probably due to their complexity. Data integration assumes an important role in the LMS interoperation with system types that do not require a tight integration, as is the case with authoring tools. For instance, the Hot Potatoes system enables the creation of quizzes - interactive multiple-choice, short-answer, jumbled-sentence, crossword, matching/ordering and gap-fill exercises - in HTML format. Moodle includes an activity that imports the quiz (HTML file) previously generated in the Hot Potatoes system. It should be noted that although Moodle supports the QTI format for quizzes described previously, Hot Potatoes cannot export in this format.

A Comparative Study on LMS Interoperability

Table 2. Reference LMS support of eLearning content standards Moodle

Blackboard

IMS CP

yes

yes

SCORM

yes

yes

IMS CC

partial

partial

IMS QTI

yes

yes

IMS LD

no

no

IMS SS

no

no

It is possible also to integrate an eLearning tool with an LMS on the business layer. For instance, the IMS Learning Tools Interoperability (IMS LTI) provides a uniform standards-based extension point in LMS allowing remote tools to be integrated into LMS. Although this specification is still not explored by the major LMS vendors, obtaining the certified support for IMS LTI is already a major milestone in their development plan. Another integration approach is through Application Programming Interface (API). The LMS include APIs to allow developers to extend their predefined features through the creation of plugins. Table 3 enumerates the approaches used by the selected LMS to address the interoperability issues regarding the integration with the system types referred in subsection 4.1. Moodle version 2.0 (due in September 2010) includes several APIs to enable the development of plugins by third parties to access repositories and portfolios (c.f. the following sub-sub-section). Blackboard uses the Building Blocks technology to cover the integration issues with other systems. A Building Block is simply a web application that runs on the Blackboard application server. This

technology allows third parties to develop modules using the Building Blocks API. For instance, the company Verbena Consulting LLC created a building block that provides a search user interface that allows searching in the MERLOT repository and returns matching results along with the metadata for each learning object.

Example of Integration In this subsection we illustrate the use of the communication APIs in Moodle, arguably the most popular LMS nowadays. Concretely we present the new file APIs of Moodle 2.0 and how it was used for implementing a plug-in for crimsonHex repositories (Leal & Queirós, 2009). The beta version of Moodle 2.0 includes support for different types of repositories. Two APIs are already available to enable the development of plug-ins by third parties systems, including: • •

Repository API for browsing and retrieving files from external repositories; Portfolio API for exporting Moodle content to external repositories.

Table 3. Integration APIs in reference LMS Moodle

Blackboard

Repositories

Repository API

Building Blocks API

E-Portfolios

Portfolio API

Building Blocks API

Evaluators

OPAQUE ws

no

153

A Comparative Study on LMS Interoperability

We chose the Repository API since it is the most stable of the two. It is organized in two parts: Administration, for administrators to configure their repositories, and; File picker, for teachers to interact with the available repositories. Figure 4 presents the file picker GUI of the crimsonHex plug-in. On the left panel are listed the available repositories as defined by the administrator. Two crimsonHex repository instances are marked with label 1. Label 2 marks the default listing of the selected repository. Pressing the “Preview” link marked with 3 presents a preview of the respective LO. Pressing the “Search” link pops-up a simple search form, marked as 4. For federated search in all available crimsonHex repositories is used the text box marked as 5. The development of this plug-in was straightforward.

Figure 4. crimsonHex plugin interface

154

In terms of programming effort we spent half a day to produce approximately 100 new lines of code. For Moodle each repository is just a hierarchy of nodes. This allows Moodle to construct a standard browse interface. The repository server must provide: • •

a URI to download each node (e.g. a LO); a list of nodes (e.g. LO and collections) under a given node (e.g. collection).

In addition to these requirements, a repository can optionally support authentication, provide additional metadata for each node (mime type, size, related files, etc.), describe a search facility or even provide copyright and usage rules. Each feature of the plug-in is implemented by a method in a PHP class. A typical method includes:

A Comparative Study on LMS Interoperability

a repository invocation (SOAP or REST), the parsing of its response (using a PHP function to parse the XML data), a selection of the pertinent data (using XPath) and an iteration over the new results (for instance, populating an array with the relevant data).

ACADEMIC MANAGEMENT FACET In this section we analyse the Academic Management facet. The main system type on this facet is the Academic Management System (AMS). An AMS aggregates all the information regarding administrative, financial, technical or scientific processes usual in educational institutions. Examples of these processes are the enrolment of students in courses, the management of grades or the payment of fees. This interoperability facet is not as mature as the one analysed on the previous section and there are still few standards available. This fact burdens the integration of academic management systems with LMS that must resort to ad hoc solutions based on the infrastructural layer.

Integration Unlike in content management, there is a sole type of system in this facet - the AMS - and apparently with very few vendors. We were not able to find in the literature any study on AMS usage. For this reason our analysis is based on the use of AMSs by Portuguese higher education institutions as reflected in the survey we conducted for this study. As mentioned before, the questionnaire inquired on vendors of different types of eLearning systems in use at each institution. We received responses from 20 different institutions and the results for AMS are presented in Figure 5. This data shows that no system is clearly preferred by Portuguese educational institutions. The choices are divided by the systems SIGA, SIGARRA, SOPHIA and Web on Campus. It

should be noted that most of these evolved from home grown systems and are in use in different schools from the same university or polytechnic institute. In some cases spin-offs were created to develop and commercialize these systems but the size of these companies cannot be compared with those developing other types of systems related to eLearning, such as LMS.

Standards An AMS manages different kinds of information. The concept of course, student, summary or grade should be described formally in order to be shared among all the systems included in a educational institution. As far as we know, there are few standards that formalize these content types and how they are communicated to allow the AMS to share data with other systems. Two know-standards are the IMS Learner Information Services (IMS LIS) and the IMS Learner Information (IMS LIP). The IMS Learner Information Services (IMS LIS) is the definition of how systems manage the exchange of information that describes people, groups, memberships, courses and outcomes within the context of learning. The IMS LIS, like its predecessor (IMS Enterprise specification), is focused on the connection between an LMS and an AMS. The IMS Learner Information (IMS LIP) specification addresses the interoperability of internet-based Learner Information systems with LMS. It describes mainly the characteristics of a learner. The learner information is a collection of information about a learner (individual or group learners) or a producer of learning content (creators, providers or vendors). The former is partial supported by Moodle and its implementation in Blackboard is in development. The latter has no support in either LMS.

155

A Comparative Study on LMS Interoperability

Figure 5. AMS usage

Integration There is an obvious gain in integrating AMS and LMS: avoiding the duplication of processes. For instance, course management is required in both systems and with a tight integration it can be performed in just one of them. Several processes can be performed in only one side and reflected in the other such as: course management, enrolment of students, grades management, summaries management, exams schedule, absences management. In general, educational institutions use ad hoc solutions to implement this type of integration. The most common strategies are: • • •

156

Portals: aggregating content from multiple sources with a common presentation layer; Database Replication: different applications but sharing content; Features Share: presentation independent but sharing some features (e.g. authentication).

Integration usually includes at least one web application, and these are typically designed based on the well known three-tier architectural pattern. There is a potential for integration in any the three classical tiers: presentation, logic and data. The portal strategy integrates at the presentation tier, providing and unified web interface to a number of independent subsystems, including eLearning systems. The major advantage of this strategy is the fact that it gives users a sense of unity, sometimes at the cost of compromising consistency. Feature sharing is integration at the logic tier and is becoming increasingly popular as more systems expose their functionality using web services. Moreover, there are a number of infrastructural services, using or not web services, which can be exploited by eLearning systems. User authentication based in directory services, such as LDAP, is an apt example of this type of integration. Finally, integration may occur at the data tier, and partial database replication is arguably most common example. For instance a LMS may im-

A Comparative Study on LMS Interoperability

port data on students, courses and student enrolment in courses from administrative systems to avoid the burden of entering this data manually. These integration models are usually combined. For instance, a portal that provides and unified presentation may also adhere to a single sign-on mechanism shared with other services.

CONCLUSION This chapter presents a comparative study on LMS interoperability. Given size of this category we focused on a couple of representative systems Moodle and Blackboard - since combined they represent a significant market share and cover both the commercial and open source development models. We proposed a framework for analysing LMS interoperability by distinguishing two different facets in the way theses systems communicate with their operational environment: learning content management and academic management. We characterized the types of systems that communicate with the LMS trough each facet. Standards are the corner stone of interoperability. Thus we made a comprehensive presentation of the existing standards. We completed the analysis with illustrations of system integration for each facet. The main conclusion of this study is that there is still a long road ahead in LMS interoperability. In general it is not straightforward to connect an LMS to another system. A lot of work has already been done in defining standards but many of them are supported neither by the LMS nor by the system that surround them. The content management facet is much more developed then the academic management facet. Content formats, especially those of learning objects, are already mature and widely supported by the analysed systems. The notable exception is the recent Content Cartridge of IMS that is not yet supported, as is not its companion specification - the Learning Tools Interoperability - that is still being implemented

in Moodle and Blackboard. This specification promises to be a major step towards content interoperability among eLearning systems. Meanwhile, to integrate LMS with content management systems we must resort to system specific APIs. An example of using Moodle 2.0 Repository API was presented to illustrate this type of integration. On the academic management facet there are no AMS system standing out from the crowd and most of those in use, at least in Portuguese higher education institutions, are home grown systems. Standards in this facet are few and immature and not widely supported by existing AMS systems. As a consequence, the integration of LMS and AMS relies on infrastructure services. We presented a set of integration strategies that are commonly used for implementing these ad hoc integrations. This study is part of an effort to select an LMS on which to base the development of eLearning systems integrating heterogeneous components. Unfortunately, from that viewpoint we cannot conclude on the superiority of any of the analysed systems.

REFERENCES Ashford-Rowe, K., & Malfroy, J. (2009). Elearning benchmark report: Learning management system (LMS) usage. Barret, H. C. (2008). Categories of ePortfolio tools, technical report. Retrieved on December 13, 2010, from http://electronicportfolios.com/ categories.html. Bersin, J., Howard, C., O’Leonard, K., & Mallon, D. (2009). Learning management systems. Bersin & Associates. Blackboard.com. (2005). Blackboard & WebCT Announce Agreement to Merge. Retrieved on December 13, 2010, from http://investor.blackboard.com/phoenix.zhtml?c=177018&p=irolnewsArticle&ID=767025.

157

A Comparative Study on LMS Interoperability

Britain, S., & Liber, O. (1998). A framework for pedagogical evaluation of virtual learning environments. Retrieved on December 13, 2010, from http://www.leeds.ac.uk/educol/documents/00001237.htm. Cole, J., & Foster, H. (2007). Using Moodle Teaching with the popular open source course management system. Sebastopol, CA: O’Reilly Community Press. Common Cartridge Specification, I. M. S. (2010). Version 1.0 Final Specification. IMS Global Learning Consortium Inc. Retrieved November 12, 2010, from http://www.imsglobal.org/cc/ index.html. Dagger, D., O’Connor, A., Lawless, S., Walsh, E., & Wade, V. (2007). Service oriented e-learning platforms: From monolithic systems to flexible services. IEEE Internet Computing Special Issue on Distance Learning. Davis, B., Carmean, C., & Wagner, E. D. (2009). The evolution of the LMS: From management to learning - Deep analysis of trends shaping the future of e-learning. Sage Road Solutions. LLC. Ellis, R. K. (2009). Field guide to learning management systems, ASTD learning circuits. Fielding, R. (2000). Architectural styles and the design of network-based software architectures. Phd dissertation. Gilbert, T. (2009). Leveraging Sakai and IMS LTI to standardize integrations. In 10th Sakai Conference Pearson Education. Holden, C. (2004). What we mean when we say “repositories”, user expectations of repository systems, Academic ADL Co-Lab. Retrieved on December 13, 2010, from http://www.hewlett.org/ NR/rdonlyres/158FC043-A56F-43C6-ABA7EB9A62656FCB/0/RepoSurvey2004-1.pdf.

158

IEEE LOM. (2002). IEEE Standard for Learning Object Metadata IEEE 1484.12.1-2002. Retrieved on November 12, 2010, from http://www.ieeeltsc. org/standards/1484-12-1-2002/. IMS CP. (2007). IMS Content Packaging v1.2 Final specification. Retrieved on November 12, 2010, from http://www.imsglobal.org/content/ packaging/. JORUM team. (2006). E-learning repository systems research watch. Retrieved on December 13, 2010, from http://www.jorum.ac.uk/docs/pdf/ Repository_Watch_final_05012006.pdf. Leal, J. P., & Queirós, R. (2009). CrimsonHex: A service oriented repository of specialised learning objects. In ICEIS 2009, 11th International Conference on Enterprise Information Systems. Levensaler, L., & Laurano, M. (2009). Talent management systems 2010. Bersin & Associates. Martínez, J. Á., & Navarra, P. L. (2007). Content interoperability on e-learning platforms: Standardization, digital libraries, and knowledge management, Revista da Universidad y Sociedad del. Conocimiento. Nichani, M. (2009). LCMS = LMS + CMS [RLOs] – How does this affect the learner? The instructional designer? Retrieved on December 13, 2010, from http://www.elearningpost.com / articles/archives/ lcms_LMS_cms_rlos. Pantel, C. (1997). A framework for comparing Web-based learning environments. Master’s thesis, School of Computing Science, Simon Fraser University, Canada. Queirós, R., & Leal, J. P. (2010). E-learning frameworks: A survey. In INTED2010 Proceedings (pp. 1345-1354).

A Comparative Study on LMS Interoperability

Regueras, L. M., Verdú, E., Castro, J. P., Pérez, M. A., & Verdú, M. J. (2008). Design of a distributed and asynchronous system for remote evaluation of students’submissions in competitive e-learning. In International Conference on Engineering Education (ICEE ‘08). Rehak, D., & Mason, R. (2003). Engaging with the learning object economy . In Littlejohn, A. (Ed.), Reusing online resources: A sustainable approach to e-learning (pp. 22–30). London, UK: Kogan Page. SCORM. (2004). 4th Ed. Specification. Retrieved on November 12, 2010, from http://www.adlnet. gov/Pages/Default.aspx. SHAME. (2006). SHAME - Standardized Hyper Adaptable Metadata Editor. Retrieved on December 13, 2010, from http://kmr.nada.kth.se/shame. Smythe, C. (2003). IMS abstract framework - A review. IMS Global Learning Consortium, Inc. Wilson, S., Blinco, K., & Rehak, D. (2004). An e-learning framework - Paper prepared on behalf of DEST (Australia), JISC-CETIS (UK), and Industry Canada.

ADDITIONAL READING Aguirre, S., Salvachúa, J., Fumero, A., & Tapiador, A. (2006). Joint degrees in e-learning systems: A Web services approach. Collaborative computing: Networking, applications and worksharing. Al-Smadi & Gutl. (2010). SOA-based architecture for a generic and flexible e-assessment system. In EDUCON’10. Alario, C., & Wilson, S. (2010). Comparison of the main alternatives to the integration of external tools in different platforms, ICERI2010 Proceedings (pp. 3466-3476).

Alario-Hoyos, C., Asensio-Pérez, J. I., BoteLorenzo, M. L., Gómez-Sánchez, E., Vega-Gorgojo, G., & Ruiz-Calleja, A. (2010). Integration of external tools in virtual learning environments: Main design issues and alternatives. Proceedings of the 10th International Conference on Advanced Learning Technologies, ICALT, Sousse, Tunisia (pages 384-388). Apostolopoulos, T. K., & Kefala, A. (2003). An e-learning service management architecture. In Proceedings of the 3rd IEEE International Conference on Advanced Learning Technologies, Athens, Greece (pp. 140-144). Aroyo, L., Dolog, P., Houben, G., Kravcik, M., Naeve, A., & Wild, F. (2006). Interoperability . In Personalized adaptive learning. Educational Tecnhnology & Society. Barret, H. (2010). Electronic portfolios in STEM - What is an electronic portfolio. Retrieved from http://www.scribd.com/doc/40206175/E-Portfolio-Definition Bohl, O., Scheuhase, J., Sengler, R., & Winand, U. (2002). The shareable content object reference model (SCORM)-a critical review, Proceedings of the International Conference on Computers in Education (pages 950-951). Bryden, A. (n. d.). Open and global standards for achieving an inclusive information society. Casella, G., Costagliola, G., Ferrucci, F., Polese, G., & Scanniello, G. (2007). A SCORM thin client architecture for e-learning systems based on Web services. [Hershey, PA: IDEA Group Publishing.]. International Journal of Distance Education Technologies, 5(1), 13–30. doi:10.4018/ jdet.2007010103 Donello, J. (2002). Theory & practice: Learning content management systems.

159

A Comparative Study on LMS Interoperability

Earl, T. (2005). Service-oriented architecture Concepts, technology and design. Upper Saddle River, NJ: Prentice Hall. Eckerson, W. (1995). Server architecture: Achieving scalability, performance and efficiency in client server applications. Open Information Systems, 19(1). Friesen, N. (2005). Interoperability and learning objects: An overview of e-learning standardization. Interdisciplinary Journal of Knowledge and Learning Objects. Girardi, R. (2004). Framework para coordenação e mediação de Web services modelados comolearning objects para ambientes de aprendizado na Web. Hall, B. (2003). Learning management systems and learning content management systems demystified. Harasim, L. (2006). History of e-learning: Shift happened, The International Handbook of Virtual Learning Environments. Springer. Harman, K., & Koohang, A. (2007). Learning objects: Standards, metadata, repositories, and LCMS. Informing Science Institute, Edição de Informing Science. Hatala, M., Richards, G., Eap, T., & Willms, J. (n. d.). The Interoperability of learning object repositories and services: Standards, implementations and lessons learned. Proceedings of the 13th international World Wide. Holden, C. (2004). What we mean when we say “repositories”, User expectations of repository systems. In: Academic ADL Co-Lab. Malita, L. (2009). E-portfolios in an educational and ocupational context. Procedia - Social and Behavioral Sciences, 1(1), 2312-2316. doi:10.1016/j. sbspro.2009.01.406.

160

Nichani, M. (2009). LCMS = LMS + CMS [RLOs] – How does this affect the learner? The instructional designer? Retrieved from http:// www.elearningpsot.com/articles/archives/lcms_ LMS_cms_rlos Rehak, D. R., & Mason, R. (2003). Keeping the learning in learning objects . In Littlejohn, A. (Ed.), Reusing online resources: A sustainable approach to e-learning (pp. 22–30). London, UK: Kogan Page. Tastle, J., White, A. & Shackleton, P. (2005), E-learning in higher education: The challenge, effort, and return of investment, International Journal on ELearning. Williams, J., & Goldberg, M. (2005). The evolution of e-learning. [Global.]. Universitas, 21.

KEY TERMS AND DEFINITIONS API: Application Programming Interface is a set of rules that a program must follow to access the services provided by other program that implements that API. eLearning: A new form of learning based on technology. A model of teaching and learning based on the online environment, leveraging the capabilities of the Internet for communication and content distribution ePortfolio: A web collection of electronic evidence packaged and managed by a user. These evidences are stored in an ePortfolio tool with a twofold set of features: students organize its achievements and teachers use them to assess the work and evolution of students. Interoperability: Ability of systems to interoperate in a uniform way. The interoperability among systems is usually achieved using standards and specification on content and communication. LO (Learning Objects): Units of instructional content that can be used, and most of all reused, on web based eLearning systems. Usually it’s a

A Comparative Study on LMS Interoperability

set of files including a XML manifest wrapped with resources and packaged in a ZIP file. LMS: System used mainly by teachers and students to manage the teaching/learning process. Nowadays these type of systems occupy a central

role in the eLearning realm coordinating and integrating a set of features by third-party systems. Metadata: Data about data. Metadata will allow systems to index, search and retrieve files based on semantic data.

161

162

Chapter 8

Mobile Learning Management Systems in Higher Education Demetrios G. Sampson University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas University of Piraeus & Centre for Research and Technology Hellas, Greece

ABSTRACT Learning Management Systems (LMS) are widely used in Higher Education offering important benefits to students, tutors, administrators and the educational organizations. On the other hand, the widespread ownership of mobile devices has lead to educational initiatives that investigate their potential as the means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This mainly aims to support the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions, towards to a truly constant and instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. In this book chapter, we address the issue of designing mLMS for Higher Education by studying and applying the W3C Mobile Web Best Practices 1.0 to a widely used existing LMS, namely, the Moodle.

INTRODUCTION During the last years, several studies have been reported that Technology-Supported Education can be effectively used in Higher Education (HE) for enhancing and enriching the traditional ways DOI: 10.4018/978-1-60960-884-2.ch008

of teaching and learning, offering meaningful learning experiences that bare the potential to address the shortcomings of traditional classroombased learning (Catherall, 2004; Tham & Werner, 2005; Bonk & Graham, 2006; Mayes et al, 2009). Technology-Supported Higher Education can offer a number of services to all relevant actors in HE (namely, students, tutors, administrators and the

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Mobile Learning Management Systems in Higher Education

educational organization) such as: (a) transparent access to courses’ materials and activities, as well as, participation to synchronous and asynchronous group-based course activities, (b) on-line submission, marking and feedback on students’ assignments, (c) effective students’ course activities tracking, (d) efficient student enrollment and monitoring, and (e) on-line delivery of courses offered by an educational organization, reaching students outside campus. These services can enhance the opportunity of learning to anyone, anywhere and at anytime without place and time restrictions and promote wider participation, removing the traditional barriers to Higher Education studies (Laurillard, 2005). Learning Management Systems (LMS) are among the most widely used applications in Technology-Supported Education, providing a convenient way to organize and deliver educational and training e-services in formal educational settings (Weller, 2007; Cole & Foster, 2007). LMS are now considered as mainstream applications for the organization, management and delivery of on-line courses in Higher Education, since they enable efficient planning, implementation, administration, tracking and reporting of educational and training activities (Kim & Bonk, 2006). LMS provide to students facilities for enrolling to on-line courses, accessing lecture notes and supportive course material, communicating with their classmates and their tutors through online discussions, participating to on-line assessments, as well as, monitoring their progress and grades. Moreover, LMS enable tutors to organize their courses’ syllabus and teaching material, gather, grade and provide feedback to students’ assignments, track students’ progress and participation to their courses and communicate with their students for answering their questions. Furthermore, LMS provide to the administrators efficient ways for assigning tutors to the courses and administrating enrolled students. Finally, LMS are offering to the Higher Educational Institutions an effective way for on-line delivery of their courses towards

reaching a larger and globally dispersed audience beyond traditional campuses. On the other hand, the widespread ownership of mobile devices and the growth of mobile communications industry has offer the potential for the provision of new services including, internet access without place and device constraints, interpersonal and group communication without place and time restrictions and sharing of digital content in any format (text, image, audio and video) (Naismith et al., 2005). These new services offered by mobile devices can be used for educational purposes aiming to enhance traditional classroom-based and/or desktop-based web-facilitated educational experiences. This has lead to educational initiatives that investigate the potential of mobile devices as the technological means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This is important towards supporting the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions towards to a truly constant and instant access from anywhere. (Kukulska-Hulme & Traxler, 2005). Within this context, providing access to LMS via mobile devices bares the potential to enhance Technology-Supported Higher Education and achieve additional benefits for students, tutors, administrators and the educational organization. More precisely, mobile LMS (mLMS) can provide to students new opportunities for accessing courses’ materials and for communicating with their classmates and their tutors beyond desktop restrictions. Tutors can use the mobile devices, so as to instantly and continuously monitor issues related with their courses (i.e. timely submission of students’ assignments, questions that have submitted from the students etc) while they are on the move. System administrators can carry out basic system support tasks even without access to their desktops. Finally, the Higher Educational

163

Mobile Learning Management Systems in Higher Education

Institutions are able to make their on-line courses truly accessible to their students. As a result, the issue of developing mobile LMS is important for the deployment of feasible mobile supported educational services in Higher Education. In this book chapter, we target addressing this issue by studying and applying the Mobile Web Best Practices 1.0 (proposed within the World Wide Web Consortium) (Rabin & McCathieNevile, 2006) to a widely used existing LMS, namely, the Moodle, so as to be accessible via mobile devices. The chapter is structured as follows: Following this introduction, the background section discusses Learning Management Systems and their benefits in Higher Education, the mobile devices and their anticipated advantages in the educational process, the international guidelines for developing web applications and/or content for mobile devices, as well as related works and studies in the field of mLMS development. The next section presents the main design considerations for the development of the proposed mobile Moodle system, demonstrates how the W3C Mobile Web Best Practices have been employed in the development of mobile Moodle by giving examples of our implementation and presents the validation process followed, so as to ensure conformance of the developed mobile Moodle with the current W3C Mobile Web Best Practices 1.0. Finally, we discuss future and emerging trends in the field of mobile LMS development, as well as our concluding remarks.

BACKGROUND Learning Management Systems Learning Management Systems (LMS), also referred to as Course Management Systems (CMS) or Virtual Learning Environments (VLE), are web-based applications (that is, they run on a server and are accessed by using a web browser) designed to support planning, implementation, or-

164

ganization, delivery and administration of on-line courses (Weller, 2007). Over the past years, LMS have attracted the attention of Higher Education community, since their available features can simulate the features of classroom-based teaching (Watson & Watson, 2007). Nowadays, LMS are considered as mainstream applications for most HE Institutions (Catherall, 2004; Tham & Werner, 2005; Weller, 2007) and they can offer important benefits to all relevant actors in Higher Education (namely, students, tutors, administrators and the educational organization), which are presented below (Bouhnik & Marcus, 2006; Daniels, 2009): •

•

Students are able to enroll to on-line courses and have access to lecture notes and courses’ supportive material at anytime and from anywhere. They are also able to study specific components of an on-line course in a more flexible way beyond the restrictions imposed by the traditional classroom. Additionally, students are able to continue interaction and communication with their classmates and tutors beyond campus physical presence, through on-line course activities. Finally, LMS provide students with opportunities for participating in on-line assessments and receiving instant feedback for their performance, as well as opportunities for monitoring their individual progress and building their own e-portfolios of academic achievements. Tutors are able through the LMS to organize and manage their courses’ syllabus, upload and share courses’ materials with their students, as well as, gather and grade more efficiently students’ assignments. Moreover, LMS offer effective ways for tutors to communicate and interact with their students and to provide feedback to them, as well as to monitor their students’ academic progress and access their grades, so as to provide additional guidance and

Mobile Learning Management Systems in Higher Education

Table 1. Key features of LMS to the key actors in HE Features

Students

Upload and share course materials

Tutors √

Access and download lecture notes and course supportive material

√

Participation to online synchronous and/or asynchronous activities

√

Create online assessments

Administrators

Educational Organization

√

√ √

Participation to on-line quizzes/surveys

√

Access to online grade book

√

√

Gather and review assignments

√

Create and manage course syllabus

√

√ √

Students and Academic Personnel management and administration

√

√

Courses management and administration

√

√

√

√

√

√

Students’ progress tracking

√

Summative students’ statistics

•

•

support to students that are facing difficulties with their courses. Administrators can manage more efficiently the administrative tasks related with the on-line course organization, such as monitoring students’ enrollment, managing students’ records, as well as tutors allocation to the available on-line courses. Educational organizations have concrete benefits from LMS by extending the audience of their courses and by increasing the flexibility of studying and teaching at these institutions removing the barriers of physical presence at campus and/or classrooms. Furthermore, LMS can offer efficient management of HE institutions Human Resources (academic and administrative) and support the implementation of quality management plans.

Table 1 summarizes key features of LMS to the key actors (namely, students, tutors, administrators), as well as to the educational organization (Weller, 2007; Cole & Foster, 2007).

√

There are many different LMS available, which could be divided in two main categories: commercial and open source. Commercial LMS include: Blackboard/WebCT (http://www.blackboard.com), Desire2Learn (http://www. desire2learn.com/) and Lotus (http://www.lotus. com). On the other hand, open source LMS include: Moodle (http://moodle.org), ATutor (http://www. atutor.ca/) OLAT (http://www.olat.org) and Sakai (http://www.sakaiproject.org/). For the purpose of our work, we selected Moodle (http://moodle.org) as the LMS to be customized in a manner that it can be effectively accessed by mobile devices following the guidelines of W3C Mobile Web Best Practices 1.0. Moodle is an open source environment that allows the development of new functionalities and/or customization of the existing ones and it is offered at no cost. It provides compatibility with current learning technology specifications such as SCORM (http://www.adlnet.gov/scorm/) and IMS CP (http://www.imsglobal.org/content/packaging/). Moodle has a large and active community of users (including educational organizations that

165

Mobile Learning Management Systems in Higher Education

use it as their LMS, tutors that deploy it for their individual courses, as well as, web developers who are continually modifying and expanding its source code). This offers a solid setting of large scale deployment and an environment for community-based further development.

Mobile Devices Mobile devices, also referred to as handheld devices or handheld computers, are pocket-sized computing devices with capabilities for internet connectivity (either via a wireless network or a mobile communication network), typically having a display screen with touch input and/or a mini keyboard (Naismith et al., 2005). Based on this definition and for the purpose of our work, examples of mobile devices are: the personal digital assistants (PDAs), the smart phones and the mobile phones. During the last years, mobile devices have been used for educational purposes and there are many studies reporting the benefits that they are offering to the educational process, which could be summarized below (Sharples, 2006; Pettit & Kukulska-Hulme, 2007; Corbeil & Valdes-Corbeil, 2007; Faux et al, 2006): •

•

•

166

Mobile devices can facilitate students to continue learning activities, initiated inside the traditional classroom, outside the classroom through their constant interaction and communication with their classmates and/or their tutors without place, time and device restrictions. Mobile devices can enable tutors to interact with their students while on the move, to organize, update and monitor their courses, their course activities and their teaching material from anywhere and at anytime. Furthermore, mobile devices can offer important facilities in including contextual and situated learning activities in traditional on-line course.

Designing Web Applications and/or Content for Mobile Devices Typically, mobile devices have a number of technical restrictions (such as limited memory, low screen resolution, insufficient processor power, low network bandwidth and limited input possibilities), which affect both the web content and the web applications that they can handle. Furthermore, there are not available yet globally agreed standard ways for the design considerations of mobile web applications. To this end, the World Wide Web Consortium (W3C) has proposed an initial set of guidelines for Mobile Web Best Practices 1.0 that can be taken into consideration when developing web content and/or web application for mobile devices (Rabin & McCathieNevile, 2006). The main objective of these guidelines is to improve the user experience when web content and applications are accessed from different mobile devices. The W3C Mobile Web Best Practices 1.0 guidelines are grouped under the following main sections (Rabin & McCathieNevile, 2006): •

•

•

•

Overall Behavior: This section includes general guidelines related to the delivery of web content and/or web application to be accessed via mobile devices. Navigation and Links: This section includes guidelines related with the definition of the structure and the navigation model of a web page or a web site so as to match the technical limitations of mobile devices (such as display size, input mechanisms, etc). Page Layout and Content: This section includes guidelines related with the user’s perception of the delivered content and provides recommendations for the language used in its text and the spatial relationship between its constituent components. Page Definition: This section includes guidelines related with technical aspects on how elements (such as tables, images

Mobile Learning Management Systems in Higher Education

•

and video) should be presented in mobile web content. User Input: This section includes guidelines related with user input taking into consideration mobile devices restrictions, such as lack of pointing devices and standard keyboards for text entry, as well as, mobile devices alternative input mechanisms, such as through touch screens.

Related Work: Designing Mobile Learning Management Systems The review of related works in the field of Mobile Learning Management Systems design reveals that there are two main approaches. The first approach is device-dependent and involves the design (in accordance to the particular targeted mobile device technical characteristics) and the development of stand alone applications that can be installed to a specific mobile device, so as to provide access to the course data of the original LMS. The main commercial LMS (used by a number of Higher Education Institutions worldwide) are following this approach and provide stand-alone applications of the original LMS, Typical examples of such mobile LMS are: Blackboard Mobile (http://www. blackboard.com/Mobile/), Litmos Mobile (http:// www.litmos.com/mobile-learning/) and Meridian Mobile LMS (http://www.meridianksi.com/products/mobile_lms/). Moreover, this approach has been also adopted from some development groups that are working with open source LMS (mainly Moodle). Typical examples are stand-alone applications of Moodle for specific mobile devices such as Apple’s iPhones or smartphones that run the Google Android operating system. There are also cases of Higher Education Institutions that they developed and use stand-alone applications of Moodle, such as the Polytechnic University of Catalonia (Forment & Guerrero, 2008) and University of Belgrade (Minović et al., 2008). Although the aforementioned design approach can achieve optimized performance for a specific

device taking advantage of its particular technical characteristics, its main limitation is that a different version of the stand-alone application should be developed for every different type of mobile device. This can be cost-effective only for certain cases where a mobile device has a significant market penetration. The second approach, which was adopted also in our work, includes the customization of the original LMS, so as to enable access via mobile devices without device-specific restrictions. An important design consideration of this approach is that the customization of the mobile LMS should conform to the W3C Mobile Web Best Practices, so as to ensure that mLMS can be accessed by any type of mobile device. An initial work based on this approach has been proposed by Houser & Thornton (2005), who customized limited Moodle features (such as polls, quizzes, wikis and forums), in order to provide access to these features through mobile phones. Another work following this design approach has been proposed by Cheung, Stewart and McGreal (2006), who investigated the customization of Moodle for use via mobile devices within the Athabasca University in Canada. The limitation of these efforts is that they do not claim conformity of the developed mLMS with W3C Mobile Web Best Practices and, moreover, that the customized Moodle features are limited. In our work, we propose the design and the implementation of a server-based mobile version of Moodle with selected features suitable for Higher Education that follows the W3C guidelines for Mobile Web Best Practices 1.0.

THE MOBILE MOODLE Design Considerations In order for a Learning Management System to be designed to support access via mobile devices, we should take into account the following design considerations (Keegan, 2005; Sampson et al, 2008):

167

Mobile Learning Management Systems in Higher Education

•

•

•

•

Students, tutors and administrators should be able to access the LMS via any type of mobile device. Students should be able to enroll in and attend courses via their mobile device, check for new course material, upload assignments, send questions to their tutors and communicate with their classmates and tutors through discussion forums. Tutors should be able to conduct basic course management tasks via their mobile device such as: monitor their students’ progress, identify and download newly uploaded students’ assignments, answer students’ questions and communicate with their students through discussion forums. Administrators should be able to carry out basic administrative tasks such as: add new users to the mLMS and allow enrollment of new students to available courses.

According to the aforementioned considerations we decided about the features that the mobile version of Moodle should support. Every course in Moodle consists of resources, activities, and blocks (Cole & Foster, 2007). Resources provide the appropriate tools for adding content such as web pages and links to web sites while activities provide the appropriate tools to add course activities such as forums, quizzes, and assignments. Blocks are some additional features for course support such as a list of all participants or a calendar. Table 2 presents the main features of the proposed Mobile Moodle The implementation of our Mobile Moodle consists of two sub-systems, the PC version and the Mobile version, which communicate through a common database in order for users to have access to the same course material; by identifying the device used to access it (PC or mobile device). As presented in Figure 1, when a user accesses Moodle through a laptop/desktop PC then the information is presented to him/her ordinarily. When a user accesses Moodle through a mobile

168

device then the same information is presented to him/her in an appropriate customized manner. The source files of Moodle that have been customized in order to create the mobile version of the system are the following: •

•

•

Files for defining content format: that is, the CSS (Cascading Style Sheets) files that contain all the appropriate declarations for content format (font size, font color etc.). Files for defining content presentation: that is, the basic source files (PHP or HTML files) that define content presentation (i.e. information displayed in an array form). Files for calling features functions: Mainly PHP files that contain calls of Moodle features functions. For example in this type of files the feature of calendar is called. If we want to remove the calendar in the mobile version of Moodle, then we should delete the corresponding part of code that calls the calendar feature. It is important to note that we can remove also the calendar feature from the administrative part of Moodle, but this would have impact on the PC version of the system because, as mentioned before, the two versions share the same database. Thus, any changes in one version would simultaneously change the other version, too.

System Implementation In order to customize Moodle to support access via mobile devices we applied a set of guidelines selected from Mobile Web Best Practices 1.0, which was discussed in the background section. Table 3 presents the selected guidelines, as well as their description on how they can be implemented. An implementation example of Guideline 1 is that a user can type the URL “ask4reasearch.info/ moodle” instead of having to type “http://www. ask4reasearch.info/moodle/pda” which is the full path for accessing the mobile Moodle version.

Mobile Learning Management Systems in Higher Education

Figure 1. The adopted mobile Moodle approach

Figure 2(a) represents an implementation example of Guidelines 2 and 4. The basic navigation of the website should be placed on the top of the page and preferably on a single line. Moodle by default uses an appropriate menu at the top of each page conformant with Guideline 2, so no further adaptation has been made in relation to this guideline. Guideline 4 that indicates the use of clear language is of particular importance for mobile delivery to assist users determine whether information is of interest to them. Thus, the information presented in the system is given in clear and simple language (i.e. a link to lecture material) while headings are used to group similar items in order for a user to distinguish information presented in a page (i.e. an area that presents the Course Activities) (implementing Guideline 4).

Figure 2(b) represents an implementation example of Guidelines 5 and 7. Scrolling is limited to only one direction (vertical) (implementing Guideline 5). The fonts and colors used (defined in CSS files of the theme adopted) provide sufficient contrast of text colors and background color so that information is visible and readable from users (implementing Guideline 7). Figure 3(a) represents an implementation example of Guidelines 6, 10 and 13. The user should not have to scroll significantly to find the primary content of the page. Therefore the most important information (i.e. a forum for posting questions related to the course), should precede information that is not so important (i.e. list of courses that a user is enrolled) (implementing Guideline 6). Informing the browser in advance

169

Mobile Learning Management Systems in Higher Education

Table 2. Key features of proposed server-based Mobile Moodle

Blocks

Activities

Resources

Features

Description

Insert a label

Insertion of textual description with the possibility of image input.

Compose a text page

Creation of a simple page of text.

Compose a web page

Formatting options for a web page including more that text

Link to a file or a website

Creation of hyperlinks to files uploaded to Moodle web site or to web sites outside Moodle.

Display a Directory

Organizing files already uploaded to the system in directories.

Choice

A simple poll (multiple choice question).

Forum

Threaded discussion board.

Glossary

Dictionary of terms.

Quiz

A web-based quiz with a variety of question types, such as multiple choice, true/false, short answer and matching.

Assignments

Advanced uploading of files: Learners can upload to the system a specific number of files (the maximum number of files that can be uploaded is defined by the tutor). Upload a single file: Same functionality with “Advanced uploading of files” only that a single file is allowed to be uploaded. Online text: Learners answer to a question placed by the tutor in simple text format. Offline activity: The tutor defines the assignment without the possibility for file uploading.

Administration

Contains all the administration tools of a course such as students’ registration, grades, course backup etc.

Courses

A list of all courses in which a student has enrolled.

Activities

A list of hyperlinks with all the activities added to the course (i.e. forum, assignment).

Search Forums

Search based on keywords in all the existing forums in the course.

Calendar

A calendar for displaying various course or site events.

Participants

Link to a list of all participants in the current course.

about the size of an image avoids the re-flow of a page when the browser is handling it. Thus, in Mobile Moodle the explicit definition of the images’ size is given (i.e. height=”16px” and width=”16px”) (implementing Guideline 10). Given the input limitations of mobile devices, the interface of the system must minimize user input as far as possible. Thus, in current implementation of mobile Moodle limited typing is required (i.e. typing a keyword to find and access a certain thread in a forum) while it is possible for users to select items using the device’s navigation keys (i.e. select Questions Forum) (implementing Guideline 13). Figure 3(b) represents an implementation example of Guidelines 3, 8 and 9. It is important to provide concise and descriptive link text to help users decide whether to follow a link

170

or not (i.e. a link leading to course material). Also it would be helpful to identify the implications of following a link by denoting the size of the targeted file (i.e. 250KB) and/or its format (i.e. PDF) (implementing Guideline 3). It is also helpful to use features of the markup language (i.e. HTML) to indicate logical document structure in a page. Moreover, tables do not work well on limited size screens and by putting for example, navigational links into them may result for the user to scroll both horizontally and vertically to see possible navigational choices. A simple and easy to use alternative to tabular representation, which at the same time can indicate clear document structure, is the use of ordered and/or unordered HTML lists. For example two headings each one followed

Mobile Learning Management Systems in Higher Education

Table 3. The W3C Mobile Web Best Practices guidelines implemented in Mobile Moodle

User Input

Page Definition

Page Layout and content

Navigation and Links

Guidelines

Description

1. URIs of Site Entry Points

Keep the URIs of site entry points short.

2. Navigation Bar

Provide only minimal navigation at the top of the page.

3. Link Target Identification

Clearly identify the target of each link and note the target file’s format unless you know the device supports it.

4. Page Content

Ensure that content is suitable for use in a mobile context, use clear and simple language and limit content to what the user has requested.

5. Scrolling

Limit scrolling to one direction, unless secondary scrolling cannot be avoided.

6. Navigation Bars etc. (Extraneous material)

Ensure that material that is central to the meaning of the page precedes material that is not.

7. Color

Ensure that information conveyed with color is also available without color and ensure that foreground and background color combinations provide sufficient contrast.

8. Structural Elements

Use features of the markup language to indicate logical document structure.

9. Tables

Do not use tables unless the device is able to support them. Do not use nested tables. Do not use tables for layout and where possible, use an alternative to tabular presentation.

10. Image Size

Specify the size of images in markup and resize images at the server.

11. Measures

Do not use pixel measures and do not use absolute units in markup language attribute values and style sheet property values.

12. Style Sheets

Use style sheets to control layout and presentation, unless the device is known not to support them, organize documents so that if necessary they may be read without style sheets and keep style sheets small.

13. Input

Keep the number of keystrokes to a minimum, avoid free text entry where possible and provide pre-selected default values where possible.

Figure 2. Implementation example of W3C mobile Web best practices guidelines (i)

171

Mobile Learning Management Systems in Higher Education

by a number of links represented as an un-ordered list (implementing Guidelines 8 and 9). Finally, an implementation example of Guidelines 11 and 12 is given below. Style information may be contained in an externally linked style sheet if the device supports use of style sheets. Current implementation demands that the device used to access mobile Moodle supports usage of style sheets (implementing Guideline 12). Moreover, avoiding pixel and absolute measures to define the layout allows the browser to adapt the content presented to fit in the display. An exception to this rule is the situation in which an image has been specifically sized for a particular display as mentioned in Guideline 10 (implementing Guideline 11). The example below defines the layout of the navigation bar used in mobile Moodle where the width of the bar is given in percentage format (i.e. width: 100%) and all the other style elements are relatively sized (i.e. height: 25em).

.navbar { margin-right: 1em; width: 100%; padding: 0em; height: 25em; border-width: 1em; border-style: solid; }

Validation Process In order to validate the conformance of the developed mobile Moodle with the W3C Mobile Web Best Practices 1.0, we used an automated validation tool provided by W3C, namely, the mobileOK Checker (http://validator.w3.org/mobile/). This tool performs various tests on a web page to determine its level of mobile-friendliness by conforming it to the W3C Mobile Web Best Practices 1.0. When a web page passes all tests then it is characterized as mobileOK page and it is considered suitable to be accessed by any mobile device. In our case, all different web pages of the

Figure 3. Implementation example of W3C Mobile Web Best Practices guidelines (ii)

172

Mobile Learning Management Systems in Higher Education

developed mobile Moodle were validated with the mobileOK Checker and for all them the tool returned with mobile conformance higher than 98%. For those pages that the tool did not return with full mobile conformance, we corrected the source files that correspond to these pages based on the recommendations provided by the tool. This process was repeated, so as to achieve full conformance to W3C Mobile Web Best Practices 1.0 for all web pages of mobile Moodle.

FUTURE RESEARCH DIRECTIONS In our work, we propose the design and the implementation of a server-based mobile version of Moodle with extended features suitable for Higher Education that follows the W3C guidelines for Mobile Web Best Practices 1.0. This is important for the deployment of feasible mobile-supported educational services in Higher Education. However, it is only an initial step for the future evolution of mLMS. An emerging trend in developing mobile LMS is to exploit current mobile devices enhanced features like position, acceleration sensors or cameras, towards detecting users’ intentions and gathering of contextual information, aiming to provide him/her with personalized situated context-aware educational services on-demand. An initial study towards the development of a mobile LMS, which gathers contextual information and reasoning on them, has been recently proposed by Lehsten et al (2010). It enables personalized mobile learning experience with location-sensitive lecture streaming, campus navigation, and ubiquitous features of the whole university computing infrastructure (Lehsten et al, 2010). Another trend in developing mobile LMS aims to overcome identified limitations of LMS, such as (Zhang et al, 2004; Liu, 2004):

•

•

LMS are used as a ‘one-size-fits-all’ service without taking into consideration individual student learning styles, knowledge level, goals and interests. In such situation, all learners receive same activities and material, regardless of their varying pre-existing knowledge and experience. LMS are promoted as ways to manage students, rather than to promote rich, interactive experiences.

Within this framework, introducing adaptive techniques to mobile LMS, so as to provide students with educational material and activities that fits their learning styles, knowledge level, goals and interests, is attracting increased research attention. An initial study towards the development of an Adaptive Mobile LMS has been proposed by Park et al (2008) and introduces the application of an artificial intelligence technique to a mobile educational device in order to provide a Learning Management System that is adaptive to students’ learning styles (Park et al, 2008).

CONCLUSION Technology-Supported Education exploits the potential of web technologies for enhancing and enriching the traditional ways of teaching and learning, offering meaningful learning experiences that bare the potential to address the shortcomings of traditional classroom-based learning in which tutors and students must physically participate in the learning process. LMS are widely used web-based applications in Technology-Supported Education, providing a convenient way to organize and deliver educational and training e-services in formal education settings such as Higher Education. On the other hand, the widespread ownership of mobile devices and the potential benefits that they can provide to the learning process has attracted the attention of both researchers and prac-

173

Mobile Learning Management Systems in Higher Education

titioners in Technology-Supported Education. This has lead to mobile devices-supported educational initiatives that investigate the potential to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute, supporting the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions towards to a truly instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. Within this context, in this book chapter, we propose the design and the implementation of a server-based mobile version of Moodle with selected features suitable for Higher Education that follows the W3C guidelines for Mobile Web Best Practices 1.0.

REFERENCES Bonk, C. J., & Graham, C. R. (2006). The handbook of blended learning. San Francisco, CA: Pfeiffer. Bouhnik, D., & Marcus, T. (2006). Interaction in distance-learning courses. Journal of the American Society for Information Science and Technology, 57(3), 299–305. doi:10.1002/asi.20277 Catherall, P. (2004). Delivering e-learning information services in higher education. Oxford, UK: Chandos Publishing. Cheung, B., Stewart, B., & McGreal, R. (2006) Going mobile with MOODLE: First steps. IADIS International Conference on Mobile Learning 2006, Dublin, Ireland. Cole, J., & Foster, H. (2007). Using Moodle: Teaching with the popular open source course management system, (2nd Edition). Sebastopol, CA: O’ Reilly Community Press

174

Corbeil, J. R., & Valdes-Corbeil, M. E. (2007). Are you ready for mobile learning? EDUCAUSE Quarterly, 30(2). Daniels, P. (2009). Course management systems and implications for practice. International Journal of Emerging Technologies & Society, 7(2), 97–108. Faux, F., McFarlane, A. E., Roche, N., & Facer, K. (2006). Handhelds: Learning with handheld technologies. Bristol, UK: Futurelab. Forment, M., & Guerrero, J. C. (2008, April 1113). Moodlbile: Extending Moodle to the mobile on/offline Scenario. Proceedings of the IADIS International Conference on Mobile Learning, Algarve, Portugal. Houser, C., & Thornton, P. (2005). Poodle: A course-management system for mobile phones. In Proc. of the 4th IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE ’05). IEEE Computer Society. Keegan, D. (2005). Mobile learning: The next generation of learning. Distance Education International. Kim, K., & Bonk, C. J. (2006). The future of online teaching and learning in higher education: The survey says. EDUCAUSE Quarterly, 29(4). Kukulska-Hulme, & A., Traxler, J. (2005). Mobile learning – A handbook for educators and trainers. UK: Taylor & Francis. Laurillard, D. (2005). E-learning in higher education. In Ashwin, P. (Ed.), Changing higher education: The development of learning and teaching. London, UK: RoutledgeFalmer.

Mobile Learning Management Systems in Higher Education

Lehsten, P., Zender, R., Lucke, U., & Tavangarian, D. (2010, March 29-April 7). A service-oriented approach towards context-aware mobile learning management systems. 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops) (pp 268-273).

Sampson, D., Papapaulou, P., Zervas, P., & Papanikou, C. (2008). Supporting higher education teaching through wireless and mobile devices: The case study of mobile Moodle. In Proc. of the AACE E-Learn 2008-World Conference on ELearning in Corporate, Government, Healthcare, and Higher Education.

Liu, Y. (2004). Faculty development and CMS. In Proceedings of Society for Information Technology in Teacher Education International Conference, 2004, 2409–2412.

Sharples, M. (Ed.). (2006). Big issues in mobile learning. Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative. Nottingham, UK: University of Nottingham.

Mayes, J. T., Morrison, D., Bullen, P., Mellar, H., & Oliver, M. (Eds.). (2009). Transforming higher education through technology-enhanced learning. York, UK: Higher Education Academy. Minović, M., Štavljanin, V., Milovanović, M., & Starčević, D. (2008). Usability issues of elearning systems: Case-study for Moodle learning management system. In Meersman, R., Tari, Z., & Herrero, P. (Eds.), OTM 2008 Workshops, Lecture Notes in Computer Science 5333 (pp. 561–570). Heidelberg, Germany: Springer.

Tham, C. M., & Werner, J. M. (2005). Designing and evaluating e-learning in higher education: A review and recommendations. Journal of Leadership & Organizational Studies, 11(2), 15–26. doi:10.1177/107179190501100203 Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems. What are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y

Naismith, L., Lonsdale, P., Vavoula, G., & Sharples, M. (2005). Literature review in mobile technologies and learning. NESTA Futurelab Series.

Weller, M. (2007). Virtual learning environments: Using, choosing and developing your VLE. UK: Routledge.

Park, H., Baek, Y. K., & Gibson, D. (2008). Design of an adaptive mobile learning management system. User interface design and evaluation for mobile technology, I, 286-301.

Zhang, D., Zhao, J., Zhou, L., & Numamaker, J. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75–78. doi:10.1145/986213.986216

Pettit, J., & Kukulska-Hulme, A. (2007). Going with the grain: Mobile devices in practice. Australasia Journal of Educational Technology, 23(1), 17–33. Rabin, J., & McCathieNevile, C. (Eds.). (2006). Mobile Web best practices 1.0. (W3C Proposed Recommendation). Retrieved on August 25, 2010, from http://www.w3.org/TR/mobile-bp/

ADDITIONAL READING Amin, A. H. M., Mahmud, A. K., Abidin, A. I. Z., Rahman, M. A., Iskandar, B. S., & Ridzuan, P. D. (2006). M-learning management tool development in campus-wide environment. Issues in Informing Science and Information Technology, 3, 423–434.

175

Mobile Learning Management Systems in Higher Education

Andronico, A., Carbonaro, A., Colazzo, L., & Molinari, A. (2004). Personalisation services for learning management systems in mobile settings. International Journal of Continuing Engineering Education and Lifelong Learning, 14(4-5), 353–369. doi:10.1504/IJCEELL.2004.005726

Dalsgaard, C. (2006). Social software: E-learning beyond learning management systems. European Journal of Open Distance and E-Learning. Retrieved August 31, 2010, from http://www.eurodl. org/materials/contrib/2006/Christian_Dalsgaard. htm

Armatas, C., Holt, D., & Rice, M. (2005, December 4-7). Balancing the possibilities for mobile technologies in higher education. In Balance, fidelity, mobility: Maintaining the momentum? Proceedings ASCILITE 2005, Brisbane.

Davide, T., & Roberto, B. (2007). A platform to support anytime, anywhere, just-for-me mlearning. Paper presented at the Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007), Nigata, Japan.

Boticario, J.G., & Santos, O.C. (2007). An open IMS-based user modelling approach for developing adaptive learning management systems. Journal of Interactive Media in Education.

Driscoll, M., & Carliner, S. (2005). Advanced Web-based training strategies: Unlocking instructionally sound online learning. San Francisco, CA: Pfeiffer.

Chao, I. T. (2008). Moving to Moodle: Reflections two years later, EQ, EDUCAUSE Quarterly, 31(3), 46–52.

Evans, C. (2008). The effectiveness of m-learning in the form of podcast revision lectures in higher education. Computers & Education, 50(2), 491– 498. doi:10.1016/j.compedu.2007.09.016

Chen, C.-M., & Hsu, S.-H. (2008). Personalized intelligent mobile learning system for supporting effective english learning. Journal of Educational Technology & Society, 11(3), 153–180. Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Corlett, D., Sharples, M., Bull, S., & Chan, T. (2005). Evaluation of a mobile learning organizer for university students. Journal of Computer Assisted Learning, 21(3), 162–170. doi:10.1111/ j.1365-2729.2005.00124.x Cow, R., Santos, I. M., LeBaron, J., McFadden, T. A., & Osborne, F. C. (2010). Switching gears: Moving from e-learning to m-learning. MERLOT Journal of Online Learning and Teaching, 6(1), 268–278.

176

Goh, T., & Kinshuk. (2006). Getting ready for mobile learning – Adaptation perspective. Journal of Educational Multimedia and Hypermedia, 15(2), 175–198. Herrington, A., & Herrington, J. (Eds.). (2006). Authentic learning environments in higher education. Hershey, PA: Information Science Publishing. Järvelä, S., Näykki, P., Laru, J., & Loukkanen, T. (2007). Structuring and regulating collaborative learning in higher education with wireless networks and mobile tools. Journal of Educational Technology & Society, 10(4), 71–79. Kadyte, V. (2004). Learning can happen anywhere: A mobile system for language learning. In Attewell, J., & Savill-Smith, C. (Eds.), Learning with mobile devices (pp. 73–78). London, UK: Learning and Skills Development Agency. Kim, S. H., Mims, C., & Holmes, K. P. (2006). An introduction to current trends and benefits of mobile wireless technology use in higher education. AACE Journal, 14(1), 77–100.

Mobile Learning Management Systems in Higher Education

Kinshuk, Kommers, P., & Sampson, D. (2004). Adaptivity in Web and mobile learning services [Editorial]. International Journal of Continuing Engineering Education and Lifelong Learning, 14(4/5), 313–317. Liaw, S. (2008). Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & Education, 51, 864–873. doi:10.1016/j.compedu.2007.09.005 Parsons, D., Ryu, H., & Cranshaw, M. (2007). A design requirements framework for mobile learning environments. Journal of Computers, 2(4), 1–8. doi:10.4304/jcp.2.4.1-8 Rekkedal, R., & Dye, A. (2007). Mobile distance learning with PDAs: Development and testing of pedagogical and system solutions supporting mobile distance learners. International Review of Research in Open and Distance Learning, 8(2), 1–26. Sampson, D., Gotze, K., & Zervas, P. (2007). Delivering IMS learning design activities via mobile devices. In Proc. of the 7th IEEE International Conference on Advanced Learning Technologies (pp. 765-769), IEEE Computer Society. Shen, R., Wang, M., Gao, W., Novak, D., & Lin, T. (2009). Mobile learning in a large blended computer science classroom: System function, pedagogies, and their impact on learning. IEEE Transactions on Education, 52(4), 538–546. doi:10.1109/TE.2008.930794 Vovides, Y., Sanchez-Alonso, S., Mitropoulou, V., & Nickmans, G. (2007). The use of elearning course management systems to support learning strategies and to improve self regulated learning. Educational Research Review, 2(1), 64–74. doi:10.1016/j.edurev.2007.02.004

Wang, H. (2008). Implement and deploy mobile learning in open universities - The many promises and challenges ahead. In C. Bonk et al. (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2008 (pp. 1340-1343). Chesapeake, VA: AACE. Zurita, G., & Nussbaum, M. (2004). A constructivist mobile learning environment supported by a wireless handheld network. Journal of Computer Assisted Learning, 20(4), 235–243. doi:10.1111/ j.1365-2729.2004.00089.x

KEY TERMS AND DEFINITIONS Learning Management System (LMS): Also referred to as Course Management Systems (CMS) or Virtual Learning Environments (VLE), are web-based applications (that is, they run on a server and they are accessed by using a web browser) designed to support planning, implementation, organization, delivery and administration of web-based courses Mobile Device: Also referred to as handheld device or handheld computer, it is a pocket-sized computing device with capabilities for internet connectivity (either via a wireless network or a mobile communication network), typically having a display screen with touch input and/or a miniature keyboard. Examples of mobile devices include: tablet PCs, personal digital assistants (PDAs), smart phones and mobile phones. Mobile Devices Supported Education: The support of traditional ways of teaching and learning through the use of mobile devices Mobile Learning Management System (mLMS): A Learning Management system that can be accessed from students, tutors and administrators via mobile devices.

177

178

Chapter 9

Enhancing Electronic Examinations through Advanced MultipleChoice Questionnaires Dimos Triantis Technological Educational Institution of Athens, Greece Errikos Ventouras Technological Educational Institution of Athens, Greece

ABSTRACT The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems (LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, provided that the new method proves its equivalence to traditional methods that might be considered as standard, such as constructed response (CR) tests. Research efforts to that direction are presented.

INTRODUCTION Learning Management Systems (LMS) ideally support the whole cycle of activities related to the interactions between instructors and students, as well as their interactions with administrative

staff and learning material (Chu, Young, Ngai, Cun, Pearl, & Macario, 2010; Ellis, 2009a). This “all encompassing” characteristic provides an extensive set of features that are available to the users of a LMS. Educational institutions and corporations can use features such as certification of

DOI: 10.4018/978-1-60960-884-2.ch009

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

participants, compliance tracking, reporting and statistical processing of educational activities, management approval, authoring of material and content management, as well as assessment ans testing. In the present chapter focus will be placed on the assessment and testing features of a LMS and, more specifically, on variants of electronic examination methods. An electronic examination is an assessment item delivered by the LMS, while the development, management and publishing of the items that the electronic examination consists of are usually tasks of a Learning Content Management System (LCMS), a software application that is either incorporated into a LMS or closely interacts with it (Feldstein, 2004). The use of electronic examinations methods, in the context of LMS, as computer-based learning and evaluation items, created in the framework of a LCMS and managed by a LMS, should be considered for enriching and strengthening the existing assessment schemes, while at the same time constructing innovative assessment techniques (Ganguli et al., 2006; Triantis, Anastasiadis, Tsiakas, & Stergiopoulos, 2007). As stated above, LMS are not directly involved in producing the contents provided to the users of the systems. Nevertheless, the whole “superstructure” that a LMS provides to an educational organization that seeks to benefit from an automated system for managing educational processes, is based on a set of foundations, among which are the assessment and testing tools incorporated into the LMS. The American Society for Training and Development (ASTD) conducted two surveys, in 2009 (189 participants) and 2010 (342 participants), on users of LMS. When asked to rank the most valuable features of LMS, in both years, among 15 features, assessment and testing ranked 1st with 59.3% of participants ranking it most valuable feature in 2009 and 54.5% in 2010 (Ellis, 2009b; Ellis, 2010). In 2009 the feature that ranked 2nd was content management, with 48.4%, and in 2010 reporting, with 51.5%. Electronic examination modules are an important part of

assessment and testing tools used by LMS. In the light of the above, research concerning innovative assessment techniques, incorporated into electronic examination modules, might be an efficient way to further the adoption of LMS. Electronic examinations, as part of computeraided testing systems, enhance student assessment procedures in a variety of ways. For example, the time allocated to scoring the answers of the students and the administrative burden related to the registration of the examination grades might be drastically reduced or even eliminated. Selfexamination material can be provided, which can be used on a distance-based learning framework by the students (Tsiakas, Stergiopoulos, Nafpaktitis, Triantis, & Stavrakas, 2007). The automated extraction of statistical indicators of the students’ performance can also be straightforwardly incorporated to the software implementing and managing the electronic examination (Mattheos et al., 2008; Van der Linden & Glas, 2000). One of the basic features of a LMS is that it enables to keep track of the performance of students across the various types of training activities that the students are engaged in. The existence and management of quantitative indicators, in the framework of a LMS, might enable the detection by the teacher of topics whose assimilation is not satisfactory by the students. In turn, the knowledge that specific material has been deficiently learned by the examinees might focus remedial action in subsequent teaching periods of the examined courses (Buchanan, 2002; Bull, Conole, Davis, White, & Sclater, 2002). Such actions might be integrated in a LMS, by modifying the study material provided to the students, in both printed and electronic form, using a LCMS, and by consecutively keeping track of the modifications to the students performance indicators induced by the changes that have been introduced. Despite of those benefits, the extent of the usefulness of electronic examinations is still under investigation, since a number of matters have to be taken into consideration for each specific application, such as the familiarization of

179

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

the students with information technology (IT) and the burden imposed on the teacher for producing both the examination forms (questionnaires) and the related electronic study material. Additionally, the possible overcomplexity of the educational software that both examinees and teachers should use must be taken into account and reduced as much as possible, so as not to hamper the familiarization of users with the electronic systems (Buchanan, 2002; DeBord, Aruguete, & Muhlig, 2004; Mattheos et al., 2008; Thomas, 2003). To overcome these problems, the LCMS must be easy to use for the content developer and built in a way to produce user-friendly interfaces for the various electronic examination tools. Multiple-Choice Questions (MCQs) tests are inherently suited to and extensively used in electronic examination systems, due to the facility to extract the final score of the examinee in a fully automatic mode (Mattheos et al., 2008; Van der Linden & Glas, 2000). Nevertheless, drawbacks of the MCQs examination method, mainly related to the influence of guessing in choosing an answer, hamper the extension of their use (Scharf & Baldwin, 2007). In order not to be constrained by the problems related to the most commonly used MCQs grading schemes, examinations based on the more traditional method of constructed response questions (CRQs), where, for example, questions are answered by developing a set of subjects in the form of short essays, are still largely used and often preferred to MCQs. The preference to CRQs, which are not well suited for electronic examinations, since the grading process cannot be usually automated, relies on the fact that in CRQs the examiner has the possibility to check more fully the knowledge of the student and especially the way that he/she developed the subject under question. In order to alleviate those problems of MCQs tests, a specific kind of MCQs examination method has been proposed, which uses sets of pairs of MCQs, referred as “paired” MCQs method (Ventouras, Triantis, Tsiakas, & Stergiopoulos, 2010). MCQs in each pair concern the same topic, but this similarity should not be 180

evident for a student who does not possess adequate knowledge on the topic addressed in the questions of the pair. The questions’ answers are graded in pairs, providing a bonus (penalty) if both (only one) answers of the pair are correctly answered. The objective of the research presented in this chapter is to advance the investigation comparing the paired MCQs grading scheme to the traditional constructed response (CR) test, taken as a standard, as well as their statistical relationship to the most simple grading scheme of MCQs, i.e., the positive scoring rule, according to which students gain points for correct answers and have no losses for wrong answers and omissions. The variation of the parameters controlling the bonus/penalty mechanism of the scoring rule is investigated. In this way, indications could be provided that the examination method based on paired MCQs might constitute a reliable tool for the evaluation of the performance of the students, enhancing electronic examination use.

BACKGROUND Electronic examinations can significantly augment the speed of testing large number of students, if the scoring of the test can be done automatically through an electronic examination platform, without need of judgment application by a human scorer. Tests being comprised of statements that have to be classified as true or false and MCQs tests are such examination methods (Bush, 2006; Freeman & Lewis, 1998; Scharf & Baldwin, 2007). MCQs usually belong to the “single-best answer” scheme, that requires the examinee to indicate the most appropriate single response, from a set of responses/answers that are proposed to the examinee, per question (McCoubrie & McKnight, 2008). A much less frequently used variation consists in using the “elimination procedure” proposed by Coombs et al. (1956), in which examinees are asked to eliminate as many incorrect answers as they can.

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

The benefits of MCQs tests have been extensively analyzed (Bush, 2006; McCoubrie & McKnight, 2008; Race, 2005; Scharf & Baldwin, 2007). Apart from the obvious time-reduction benefits for the examiner, in testing large number of students, and the fact that the examinee’s score is not dependent on his/her writing speed skill, a broad range of student knowledge acquisition for the field being examined might be tested in a short period of time, provided that a sufficiently large question bank is correctly used in constructing the MCQs test. The question bank should be constructed in a way that all the teaching units/ chapters/modules comprised in the course that will be tested are covered by a number of MCQs, that MCQs of varying difficulty are created per teaching unit and that the number of questions in the data bank is large enough so that repetition of questions in consecutive tests will be avoided as much as possible. Furthermore, ensuring correctness of the use of the data bank implies that for a specific examination, questions will be selected from the data bank in a way to cover all teaching units, in a way proportionate to the relative significance of each unit in the course, with questions of varying difficulty existing in the test for each unit. In electronic examinations using MCQs, there is the possibility to provide as feedback the correct answer to the student, immediately after he/she answered each MCQ item. The provision of immediate on-screen feedback (Scharf & Baldwin, 2007; Race, 2005) might enable examinees to reflect on the procedure they used to reach a wrong answer and therefore avoid consecutive missteps, provided there is adequate time to reflect on the questions. In the more common case where the examinee is required to submit his/her answers to all the questions of the test, without intermediate feedback, the automated nature of the marking makes possible the provision of prompt feedback to the examinee, after the termination of the examination, concerning both the overall test grade and information for correct and incorrect answers that were given by the examinee per

multiple-choice (MC) item. It has been shown that, in addition to the obvious ability of MCQs to test simple memory skills, such as power of information retention and retrieval, MCQs tests can investigate complex abilities and understanding (Scharf & Baldwin, 2007; Manning & Dix, 2008; Curzon, 1997). In the framework of research concerning parameters used as markers of the reliability and validity of examination methods (Anastakis, Cohen, & Reznick, 1991; Bennett, Rock, & Wang, 1991; Bridgeman, 1991; Evans, Ingersoll, & Smith, 1966; McLean, Dauphinee, & Rothman, 1988; Wainer, Wang, & Thissen, 1994; Wass, Wakeford, Neighbour, & Van der Vleuten, 2003), there exist substantial indications that MC scores provide higher reliability and are as valid as scores extracted from examinations based on the CRQs method (Wainer & Thissen, 1993; Lukhele, Thissen, & Wainer, 1994). It could have been expected that those indications would help in promoting the use of MCQs tests in most educational settings where CRQs are still used, especially taking into account the drawbacks of the CRQs examination method concerning the fact that subjects that might be examined cannot cover a significant amount of the material taught during the courses and, more importantly in an electronic examination setting, the inherent inability of introducing automated grading in essay-like responses to questions. Nevertheless, CRQs tests are still widely used, and this might not only be due to their main advantage, namely that they provide to the examiner the possibility to check the path taken by the student to develop the subject under question and reach the requested answers, but also due to the variety of drawbacks that plague MCQs. One category of drawbacks of MCQs tests concerns the requirements imposed on the person who will construct the question bank for the MCQs test. The MCQs examination method relies heavily on the quality of the question bank. As mentioned above, this concerns the need of a large question bank, so as to cover all the teaching

181

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

units/chapters/modules of the course/material to be examined, in a way proportionate to the extent of each unit and with the existence of questions of varying difficulty per unit. The time burden to produce such an extended bank might often result in poorly written or overlapping questions (Bush, 2006). If higher-order thinking skills are to be consistently checked, the time requirements for constructing the MC items might increase the effort and expertise needed. For example, in order to test the aptitude to solve complex problems and exercises, it has been proposed to fragmentize the initial problem into questions, so that the reasoning path that a student would take could be checked (Stergiopoulos, Tsiakas, Triantis, & Kaitsa, 2006a). Obviously, it is a challenge to construct a variety of such “linked” questions, per teaching unit. The other category of drawbacks of MCQs tests concerns issues related to the scoring rules that are used for grading the MC items of the test. The simplest grading scheme uses the “positive-grades-only” scoring rule, according to which students collect positive points for correct answers, while no losses are incurred for wrong answers or omissions. The influence of guessing in choosing an answer results in collecting some partial scores in the final score, by answering questions by chance, without possessing knowledge of the questioned material. For remedying this problem a variety of marking schemes for written MC tests have been introduced. In this perspective, “mixed-scoring” rules are used, i.e., rules in which students gain points for correct answers and lose points for incorrect answers and, in a stricter variant of the scoring rule, also for omissions (Scharf & Baldwin, 2007). These rules penalize guessing and seem to drastically reduce the gaining of scores through guesswork. On the other hand, the use of such rules have been shown to dissuade examinees from answering a question for which they might possess an intermediate or low level of knowledge, concerning especially those examinees who anticipated a high grade in

182

the examination. More generally, the behavior of examinees in MCQs tests using mixed-scoring rules might be influenced by factors that affect decision making under uncertainty. In turn, this might produce variance in the test scores that is related to the expectations of the examinees and not to the knowledge that is tested (Bereby-Meyer, Meyer, & Flascher, 2002; Bereby-Meyer, Meyer, & Budescu, 2003). Other scoring rules that have been used include the order-of-preference scheme (Bush, 1999; Wood, 1991). According to this, the MCQs used belong to the “single-best-answer” scheme, i.e., there is only one correct answer per question and examinees have to assign an order of preference to every answer for each question. The score per MC item is dictated by the preference they assign to the correct answer. Another MCQs examination variant requires the examinee to associate a level of confidence with each selected answer, so that their degree of certainty in answering is monitored (Pressley, Ghatala, Woloshyn, & Pirie, 1990; McKenzie, Wixted, Noelle, & Gyurjyan, 2001). In this kind of scoring method a wrong answer that is associated with a high degree of certainty is penalized much more than if the wrong answer was related to lack of confidence. Still another variant are the so-called “liberal” tests (Bush, 2006), which enable the choice of more than one answer per question, if the examinee is uncertain which is the correct answer. It might be assumed that the examiner might be able to test the level of knowledge of the examinee since the examinee has the possibility to select between a lot of alternative responses (Bush, 1999). Such alternative MCQs examination variants present to the person who will construct the tests the problem of formulating a suitable grading scheme, since the variability in selections that an examinee might choose when answering a question is much greater than in simple “single-best-answer” schemes. Furthermore, the use of these testing methods might distract the examinee from focusing on answering correctly the questions, when his/her confidence to the answer given is requested, or when having

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

to decide to select alternative answering tactics in “liberal” tests (Bush, 2006). In a previous work, an alternative MCQs examination method has been presented (Ventouras et al., 2010). According to this method, the MC items given in the test constituted pairs, referred as “paired” MCQs. MCQs in each pair concerned the same topic, but this similarity was not evident for a student who did not possess adequate knowledge on the topic addressed in the questions of the pair. The questions’ answers were graded in pairs, providing a bonus (penalty) if both (only one) answers of the pair were correctly answered. If both answers were wrongly answered no marking was collected by the examinee. The use of this scoring rule aimed at penalizing guesswork, without inducing the “hampering” effects produced by the negative marking part of the commonly used mixed-scoring schemes. In that study a paired MCQs test was given to 63 examinees, in the framework of a computer-based learning system. The same examinees took also a CRQs examination. Students were examined in a PC laboratory room, using an electronic examination platform (details of the platform are given in the following “Methodology of the Examination” section). The results of the paired MCQs examination, when using the pair-wise scoring rule, were statistically indistinguishable with the grades produced by the CRQs examination method, when made to the same sample of students, οn the same topics and with the same levels of difficult. Both the results of the paired MCQs examination, when using the pair-wise scoring rule, and the CRQs examination results differed significantly from those obtained by scoring the same MCQs using a positive-grades-only scoring rule that ignored the pairing of MCQs (referred to as the PSRMCQs examination method). Specifically, with score range (0-100), the mean of the scores of the examinees for the PSR-MCQs method was 58.25 (Standard Deviation, S.D.=18.65), while the mean for the paired MCQs and the CRQs examination methods were 45.95 (S.D.=20.28)

and 46.24 (S.D.=20.37), respectively. Another interesting finding of that study was that, when the CRQs method was compared to the PSR-MCQs method, in an another examination where no paired MCQs were used, the statistically significant difference between the scores of the CRQs and the PSR-MCQs method was greatly reduced when the set of students comprised only those students who got a score greater than 70. Specifically, the difference of the mean scores of the two methods was reduced from 12.42 (p=0.001) for the whole set of students to 5.37 (p=0.047) for the reduced students set. It seems that the higher-grade effect of the PSR-MCQs method was reduced for that set of students. A possible reason for that might be that students who got a higher score than others, were, on average, better prepared and qualified for the examination, and therefore did not apply, or greatly reduced, guessing in selecting the answers to the MCQs. Consequently, there are indications that the methodologies used for alleviating the positive grade bias induced by the positive-grades-only scoring rule, are more important for reducing the amount of students, who by guesswork could take a “pass” note (in the case of that study 50), although their level of knowledge acquisition, recall and/or reasoning capabilities was not sufficient for taking the examination with success.

COMPARISON OF CONSTRUCTEDRESPONSES QUESTIONS AND PAIRED MULTIPLECHOICE QUESTIONS Methodology of the Examination In order to expand the investigation of the comparison of the paired MCQs examination method to the CRQs examination method (taken as a gold standard) and the PSR-MCQs method, mainly concerning the effects of the parameters controlling the bonus/penalty mechanism of the scoring rule

183

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

of the paired MCQs method, in the examination period of February 2010, a course was selected, in order to compare the three examination methods. The examination of the course was implemented according to the following procedure.

The Examined Course and the Sample of Students The course that was selected for comparing the scores of the three examination methods was an introductory electronics course entitled “Electronic Physics”, which belongs to the group of core background courses and is taught at the Electronics Department of the School of Technological Applications of the Technological Educational Institution (T.E.I.) of Athens. The course contains four teaching units. 30 students participated in a CRQs examination and a MCQs examination. Their answers to the MCQs examination were scored according to the two different scoring variants mentioned in the Background section, i.e., the pair-wise scoring rule and the positive-gradesonly scoring rule. All students had previously been given information about the examination procedure, both in its CRQs and MCQs form and were familiarized with the electronic examination platform used in the MCQs examination, so that they would not be distracted from the presentation format and the requirements for the use of the electronic system. Additionally, they had available related study material, in both printed and electronic polymorphic form (Stergiopoulos et al., 2006a).

Formulation of the Questions and Examination Procedure The CRQs examination was taken by the students in its traditional paper-and-pencil form. The MCQs examination took place in a PC laboratory room, through the “e-examination” application developed at the Technological Educational Institution of Athens (T.E.I.-A.). The development

184

of that application is incorporated in a wider effort to further disseminate the use of PCs in the educational process through the creation of an institution-wide LMS (Triantis et al., 2007; Tsiakas et al., 2007; Stergiopoulos et al., 2006a; Stergiopoulos et al., 2006b). The LMS uses the eClass platform, developed and provided by the Greek Academic Network (GUnet Asynchronous eLearning Group, 2010), based on the Claroline open-source Course Management System (Claroline, 2010). “e-examination” is mainly a managing and editing tool that enables the teacher to create electronic assessment tests that can be used either for examining students or for self-assessment by the students. The electronic tests can be displayed by a web browser, ensuring their portability across PC platforms. The user interface consists of a series of questions, which can be either of the MC or the CR type (Stergiopoulos et al., 2006a; Triantis, Stavrakas, Tsiakas, Stergiopoulos, & Ninos, 2004). For the CRQs examination, a set of three test subjects was created. Two subjects contained 3 CRQs and one subject contained 2 CRQs. Therefore, the examinee had to answer 8 CRQs. The CRQs were short essay subject development questions, of either theoretical or exercise form. The distribution of CRQs was designed so as to cover each teaching unit proportionally. For the MCQs examination, a database was constructed containing a large number (N=400) of MCQs, covering the whole range of the subject material of the course. By using “e-examination”, a first set of MCQs {qa1, qa2, …, qak} (k=40) was randomly selected from the database, taking care to cover each teaching unit proportionally. A weight wai was assigned to each question i=1,…,k, depending on its level of difficulty. With those questions as reference, a second set of new MCQs {qb1, qb2, …, qbk} (k=40) was composed, with each question qbi possessing a similarity to question qai (i=1,…,k), constituting a pair of MCQs, according to the following rationale: a) both questions referred to the same topic and b) the knowledge of the

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

correct answer for question qai, from a student who had proceeded to a systematic study and was cognizant of the topic, implied the knowledge of the correct answer for qbi and vice versa. The distribution of the weights was as follows: 31 questions had weight 1, 31 questions had weight 1.5, 16 questions had weight 2 and 2 questions had weight 3. For most pairs, the questions of the pair had the same weight, i.e., it was wai=wbi. In some cases though, the weight of the two paired questions differed slightly (i.e., by 0.5). This was done because, for some topics, it was difficult to create a pair of questions that both respected the two requirements (a) and (b) stated above and were also absolutely equal in their level of difficulty. The presentation of the 2k=80 questions that the students had to answer in the PC screen was designed so that the questions were given with a random sequence, taking care that each question qbi was presented after a lapse of at least 10 questions after the presentation of question qai. Questions were automatically given through the “e-examination” software system. In the first phase of the examination, students were given the CRQs. In the second phase of the examination, the MCQs were given to students, in the PC laboratory room. After the end of the pre-determined MCQs examination duration time, a results’ page was produced for each student. The results’ page included all questions with the correct answer and the indication of whether it was correctly or wrongly answered, as well as the final score. One copy was given to the student and one to the examiner, for processing the scores. Care was taken so that the MCQs were, overall, of equivalent level of difficulty with the questions posed to the examinees in the CRQs examination. This enabled the comparison of the scores that would be achieved after the students would have given their answers in the two forms of examinations, i.e., the one using MCQs and the CRQs.

Scoring Methodology For the MCQs, 2 scores m1 and m2 were computed. Score m1 was computed as follows: for each MCQs pair i=1,…,40, the “paired” partial score pi was computed as pi = (qai wai + qbi wbi ) (1 + kbonus )

(1.a),

if both qai and qbi were correct, in which case qai=qbi=1, or pi = (qai wai + qbi wbi ) (1 − k penalty )

(1.b),

if either qai or qbi was correct, in which case it was either qai=1 and qbi=0, or qai=0 and qbi=1, or pi = 0

(1.c),

if both qai and qbi were incorrect, in which case qai=qbi=0. Parameters kbonus and kpenalty were variables that controlled the bonus and penalty mechanism, respectively, of the scoring rule. It should be noted that in the study of Ventouras et al. (2010) kbonus and kpenalty had fixed values of 0.25 and 0.5 respectively and it was wai=wbi for every i. The total score m1, with maximum value equal to 10, was then computed as: 40

m1 =

∑p i =1

40

∑ (1 + k i =1

bonus

i

⋅ 10

(2)

)(wai + wbi )

Therefore, to produce score m1, a bonus is given to the student if he/she answered correctly both questions of the MCQs pair (qai, qbi) and a penalty if he/she answered correctly only one question of the pair. Therefore, through this scoring algorithm, the final score extracted corresponds to the paired MCQs examination method.

185

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Statistical Evaluation Rationale

Score m2, with maximum value equal to 10, was computed according to 40

m2 =

∑ (q

ai

i =1

wai + qbi wbi )

40

∑ (w i =1

ai

⋅ 10

(3)

+ wbi )

where qai = [

1 if the question was correctly answered 0

if the questiion was wronlgy answered



(4)

and qbi = [

1 if the question was correctly answered 0 if the questiion was wronlgy answered

(5) This method of scoring ignores any similarities existing between the questions of the pair and does not impose a penalty to the student by negative marking for incorrect answers. Therefore, through this scoring algorithm, the final score extracted corresponds to the PSR-MCQs examination method. Finally, for the CRQs examination category, each of the eight questions j=1,…,8, was graded, with minimum question grade gj being 0 and maximum grade varying from 0.4 to 2.5, according to the importance of the respective question. The sum of the maximum grades was 10. The overall CRQs examination score, m3, was extracted as the sum of the question grades: 8

m3 = ∑ gj

(6)

j =1

In all examination categories 5.0/10.0 was the minimum score required for passing the examination.

186

The comparison of the paired MCQs examination method to the CRQs examination method and the PSR-MCQs examination method has as an objective to provide indications for accepting the paired MCQs examination method as an alternative for CRQs examinations, according to the rationale stated in the Background section. Such an indication might be provided, if the scores obtained through the paired MCQs, using the pair-wise scoring method, are statistically indistinguishable from the scores obtained from the CRQs examination method. Therefore, the hypothesis H0 to be tested in comparing the paired MCQs, the PSR-MCQs and the CRQs examination methods could be stated as: “The mean of the distribution of scores m1, obtained using the paired MCQs examination method, the mean of the distribution of scores m2, obtained using the PSR-MCQs examination method, and the mean of the distribution of scores m3, obtained using the CRQs examination method, are equal”. If hypothesis H0 is rejected, i.e., the overall differences between the three means are significant, then post-hoc pair wise comparisons, with adjustment for multiple comparisons, should be used, in order to check the three “secondary” hypotheses, namely H0(paired MCQs to PSR-MCQs) (i.e., “The mean of the distribution of scores m1, obtained using the paired MCQs examination method is equal to the mean of the distribution of scores m2, obtained using the PSR-MCQs examination method”), H0(paired MCQs to CRQs) (i.e. “The mean of the distribution of scores m1, obtained using the paired MCQs examination method is equal to the mean of the distribution of scores m3, obtained using the CRQs examination method”), and H0(PSR-MCQs to CRQs) (i.e., “The mean of the distribution of scores m2, obtained using the PSR-MCQs examination method is equal to the mean of the distribution of scores m3 obtained using the CRQs examination method”).

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Scoring Results and Statistical Evaluation In Table 1 the descriptive statistical values related to the three examination methods used in the examination are presented, i.e., the mean values ( m1, m 2, m 3 ) and standard deviation (S.D.) of the scores m1, m2 and m3 taken from the three methods for the whole set of students examined, and for two sub-sets of students, per examination method: for those students who passed the examination (score >= 5.0) and for those who failed the examination. Furthermore, the number and the percentage of students belonging to each sub-set is given. A value of kbonus=kpenalty=kcommon=1/3 was chosen. The Kolmogorov-Smirnov goodness-of-fit test showed that the distributions of the scores m1, m2 and m3 were each consistent with a normal distribution. The same held for the two sub-sets of students. For the whole set of students who took the examination, repeated-measures ANOVA with one within-subjects factor (method of examination, three levels) indicated that the within-subject effect was significant (F1.266,36.725=273.975, p<0.001, degrees of freedom were corrected for non-sphericity according to the GreenhouseGeisser procedure). Therefore, the hypothesis H0 to be tested, that the overall differences between the means of the scores m1, m2 and m3 are not significant, has to be rejected at p=0.001 level of significance. ANOVA was followed by planned

comparisons between each of the examination methods, assessed with post-hoc Bonferroni pair wise comparisons at 0.01 level of significance. Significant differences existed between m1 and m2 (p<0.001) and between m2 and m3 (p<0.001). Therefore the secondary hypotheses H0(paired MCQs to PSR-MCQs) and H0(PSR-MCQs to CRQs) had to be rejected at p=0.001 level of significance. If the level of significance of the post-hoc test is set to 0.05, then the difference between m1 and m3 reaches significance, albeit at a barely adequate level of p=0.043. These results indicate that the paired MCQs examination method with bonus/penalty adjustment (resulting in score m1) is statistically equivalent to the CRQs examination method (resulting in score m3), if a strict criterion (p<0.01) is selected for rejecting the secondary null hypothesis H0(paired MCQs to CRQs). Both methods differ significantly from the MCQs, which do not use a negative marking “penalty” procedure, i.e., the PSR-MCQs examination method (resulting in score m2). Even in the case where a lax criterion of p<0.05 was chosen, so as to reject the secondary null hypothesis H0(paired MCQs to CRQs), the difference between m1 and m2 and between m2 and m3, as quantified by the absolute differences of the means of their distributions (i.e., | m1 − m 2 | =0.92 and | m 2 − m 3 | =0.86, respectively, see Table 1), is much higher than the difference between m1 and m3 (i.e., | m1 − m 3 | =0.06). Therefore, present results provide support fort the following infer-

Table 1. Descriptive statistics of the three examination methods Set of students

m1 (mean, S.D.)

m2 (mean, S.D.)

m3 (mean, S.D.)

Number (percentage) of students per examination method Paired MCQs

PSRMCQs

CRQs

Whole set (n=30)

5.09 (1.75)

6.01 (1.59)

5.15 (1.77)

Set of students who passed successfully the examination (score ≥ 5.0)

6.65 (1.09)

6.69 (1.24)

6.5 (1.20)

14 (47%)

22 (73%)

16 (53%)

Set of students who failed the examination

3.72 (0.77)

4.15 (0.59)

3.60 (0.71)

16 (53%)

8 (27%)

14 (47%)

187

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

ences: (i) that the evaluation of the students with PSR-MCQs gives greater success rates and scores than both the CRQs examination method and the paired MCQs examination method and (ii) that the bonus/penalty marking scheme of the paired MCQs makes the MCQs examination method scoring results much similar to that obtained through the CRQs examination method. This bias of the PSR-MCQs method to higher scores can also be deduced from the inspection of Figure 1 and the regression line of m2 to m3 (Figure 2). One of the most probable explanations that could be put forward as reason for this characteristic of the PSR-MCQs method is the absence of any mechanism to correct the influence of guesswork on selecting the correct answers. This is in agreement with the results presented in Ventouras et al. (2010). The introduction of bias by the PSRMCQs method, due to the lack of correcting for the “guesswork effect”, seems to be removed by the paired MCQs examination method with bonus/ penalty adjustment, as indicated also by compar-

ing the regression line of m2 to m3 (Figure 2) with the regression line of m1 to m3 (Figure 3). The regression of m1 to m3 was chosen to be expressed by an equation of the form y=ax and the resulting parameters that were computed, i.e., a=0.9992 and R2=0.9945, comforted inference (ii) stated above. The alleviation of the positive grade bias seems to be achieved through the bonus/ penalty pair-wise scoring rule of the paired MCQs examination method. It should be stressed that this improvement is gained while avoiding explicit negative marking for incorrect answers, so that examinees might not be dissuaded from answering questions for which they might possess an intermediate level of knowledge. The above results, concerning the comparison of the CRQs and the two MCQs examination methods, stood also when the comparison was restricted to the two sub-set of students. In the statistical evaluations, in order to keep the validity of the repeated-measures ANOVA methodology, the groups were composed with those students

Figure 1. Number of students graded in the respective score ranges, per examination method. Dotted bars: Paired MCQs examination method. Black bars: PSR-MCQs examination method. Gray bars: CRQs examination method.

188

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Figure 2. Regression line of score of PSR-MCQs (m2) to score of CRQs (m3)

who failed or passed at the CRQs examination method. For both groups of students, repeatedmeasures ANOVA with one within-subjects factor (method of examination, three levels) indicated that the within-subject effect was significant, therefore the null hypothesis H0 that was tested in comparing the paired MCQs, the PSR-MCQs and the CRQs examination methods had to be rejected, and for both students’ groups, at a high degree of significance (p<0.001). Again, with the level of significance set to 0.01, significant dif-

ferences existed between m1 and m2 and between m2 and m3. In the present study the effects of varying parameter kcommon were investigated, concerning the “fit” of the scores of examinees in the paired MCQs and the CRQs examination methods. This fit can be quantified using parameters a and R2, which are related to the regression line of m1 to m3. Their best possible value is 1. As can be seen from Figure 4, R2 remained at very high levels (> 0.99) for 0.27 ≤ kcommon ≤ 0.47. As can also be

Figure 3. Regression line of score of paired MCQs (m1) to score of CRQs (m3)

189

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

seen from Figure 4, the function of R2 to kcommon is smoothly varying, having a global maximum R2=0.9950 at kcommon=0.37. The same characteristics of a smooth evolution are presented by parameter a, which, as can be seen in Figure 5, for the range of kcommon values that were investigated (0.23 to 0.47) varied from 1.0337 to 0.9587, having the values nearest to 1 between kcommon=0.32 and kcommon=0.33. Another metric whose relation to the variation of kcommon was investigated was the mean of the square root of the sum of squared differences (RMS value) of the scores that the students took in the CRQs and the paired MCQs 30 2 1 examinations: ε = m 3 j − m1j ) . Met( ∑ 30 j =1 ric ε also describes, in an overall way, how well the grades provided by the paired MCQs examination method come close to the grades of the CRQs method. This “closeness” is a goal of the introduction of the paired MCQs method. The best value for ε would be 0. As can be seen from Figure 6, the function of ε to kcommon is smoothly varying having a minimum at kcommon=0.34. The smoothness of the variation of the three parameters, in relation to kcommon, provides an indication that the proposed bonus/penalty grading scheme of the paired MCQs does not present abrupt fluc-

tuations, if the value of the parameter controlling the bonus/penalty process is varied.

FUTURE RESEARCH DIRECTIONS Based on the results presented in this chapter, future research should firstly include the thorough investigation of the effects of using the “bonus giving” and penalizing parameters kbonus and kpenalty, respectively, which control the scoring procedure for the paired MCQs examination method. In the work of Ventouras et al. (2010) two different values for kbonus and kpenalty were used, with kbonus=0.5kpenalty, while at the novel results presented in this chapter a common value kcommon for both kbonus and kpenalty was used. Although in both cases the introduction of the paired MCQs procedure “attracted” the MCQs examination scoring much closer to the CRQs “gold standard”, with a clear differentiation to the positive bias producing PSR-MCQs method, when a kcommon was used, there remained a level of statistically significant difference (p=0.043) between the paired MCQs and the CRQs examination methods, which did not exist in the results of Ventouras et al. (2010). Therefore, it would be interesting, for refining the scoring rule, to investigate the effects of us-

Figure 4. Variation of R2 (of the regression of m1 to m3) as related to k(=kcommon)

190

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Figure 5. Variation of a (of the regression of m1 to m3) as related to k(=kcommon)

ing different values for kbonus and kpenalty, by testing schemes where the ratio of kbonus to kpenalty could be either augmented (reaching 1 as was the case for the novel results of the present chapter, or even surpassing it) or reduced. The use of ratios kbonus/ kpenalty<1 is expected to make the effects of the penalizing procedure more pronounced. It would be interesting to see whether this strengthening “protects” the scoring against the influence of guessing better than the equalization of kbonus and kpenalty. It should also be stressed that the application

of the above investigations to a greater number of students and more than one course would add value to the results presented. Another area of future research could be the use of the paired MCQs examination method in conjunction with the MCQs examination variant, presented in the Background section, which requires the examinees to associate a level of confidence with each selected answer so that their degree of certainty in answering is monitored. As stated above, in this kind of scoring method

Figure 6. Variation of ε (mean of the square root of the sum of squared differences of m1 and m3) as related to k(=kcommon)

191

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

a wrong answer that was associated with a high degree of certainty is penalized more severely than if the wrong answer was related to low levels of confidence. If such a scheme would be used in conjunction with the paired MCQs bonus/penalty procedure for the paired questions, a second penalizing factor would be introduced and it would be interesting to explore the influence of the two penalizing parameters to the alleviation of the positive grade bias introduced by the PSR-MCQs method. The additional resources that examinees need to devote to making the confidence judgments raise concerns over possible negative effects to the performance of the examinees, but there exist indications that the addition of such questions in the examinations do not penalize performance (Carvalho, 2009). An important topic related to electronic examination software, is its ability to create applications (i.e., electronic examination platforms) that can both be easily accommodated/combined with already existing tools and provide for future developments that will follow common structured formats. With this aim in scope, in the last decade, the IMS Question & Test Interoperability (QTI) specification has been developed by the IMS Global Learning Consortium. IMS QTI describes a data model for the representation of question and test data and their corresponding results reports. Therefore, the specification enables the exchange of this item, test, and results data between authoring tools, item banks, test constructional tools, learning systems, and assessment delivery systems. (IMS Global Learning Consortium Inc., 2006a). The software used in the work described in the present chapter has been developed independently of the IMS QTI specification. Therefore, the developed MCQs are currently “locked” to the specific e-examination application developed at T.E.I.-A. This application provides “examination packages” to the institution-wide LMS, but

192

the IMS QTI compatibility and capabilities of the Claroline platfrom, upon which the LMS was built, are not utilized. Nevertheless, due to the characteristics of the data model of the specification, basic features of the e-examination application inherently fit classes of the IMS QTI specification. For example, each MCQ item and the choice of response to each item fit the “single assessmentItem” and the “choiceInteraction” classes of the specification. On the other hand, the extent of compatibility of the novel features of the proposed MCQs tests, i.e., the selection of “paired” questions from an item bank and the grading of the responses, to the model described by IMS QTI specification is a matter that needs investigation. Among other topics, future work should clarify whether the pair of questions will be combined into a “composite” item, although this should not be made evident to the examinee, and whether the “generalized response processing model” of the specification has to be extended to include the paired grading scheme, pertaining to the “responseRule” abstract class of the model and its derived and associate classes (IMS Global Learning Consortium Inc., 2006b). The proposed electronic examination scheme will surely benefit if it is re-developed in a form compatible to the IMS QTI specification. Specifically, the question bank and grading procedures, upon which the electronic examination is based, will be smoothly handled and incorporated into LMS supporting the specification and this will consequently enable the wide dissemination of its use in LMS.

CONCLUSION The use of electronic examinations in the framework of LMS provides a valuable tool for educators, since they enable the automated quantification and processing of students’ assessment. Grading and test results tracking, accompanied with performance indicators, can be smoothly merged in the learning management and instructors-students

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

interactions and procedures that the LMS supports. MCQs are inherently suited for electronic examinations. Therefore, it can be conjectured that the adoption of LMS will be enhanced, if electronic examinations based on MCQs could overcome problems related to their use in classroom practice. Simple positive scoring rules (PSR) of the MCQs introduce bias in the scores gained by the examinees, while mixed-scoring rules might introduce “hampering” effects to examinees, who are not sufficiently confident of their knowledge on certain topics. The paired MCQs scoring methodology has been proposed as an alternative scoring rule. The relation of the paired MCQs and PSR-MCQs test performance scores to the scores of the examinees using the CRQs method, in the results presented above, as well as in previous research, provide indications about the superiority of the scoring process adopted in the paired MCQs, as compared to the PSR-MCQs examination method. The grading process of the paired MCQs method does not introduce a positive grade bias, as is the case with the PSR method. Furthermore, with paired MCQs there is no need of negative markings for each wrongly answered question. The elimination of significant drawbacks of existing MCQs scoring methods might enable the unimpeded exploitation of the main advantage of MCQs-based examinations, concerning speed and objectivity in grading and the extent of subjects that can be examined in a single examination session. MCQs, being an important part of examination methods, and, in their electronic form, of LMS, it should nevertheless be stressed that their initial introduction into the assessment procedure of an educational institution incurs a significant amount of workload to those who will construct the question bank, from which the MCQs will be selected for each examination session. This is due to the requirements for the question bank, irrespective of the grading scheme that will be used, i.e., that it should be sufficiently large to cover the whole range of the material to be examined, that there

should be no overlap between questions and that the difference between choices to select must be clear to a sufficiently prepared student. In the case of the proposed method, these requirements are stressed a little further, since, for each topic that needs to be examined, a pair of questions of equivalent level of difficulty should be constructed, in such a way that the fact that each MCQ of the pair concerns the same topic should not be evident to a student, who does not possess adequate knowledge on the topic addressed in the questions of the pair. By judiciously planning and allocating the initial human capital resources that would be needed for the creation of such paired question banks, it is conjectured that electronic MCQs examinations, used as “building blocks” of combined LCMSs and LMS, will improve the proper quantification of the level of knowledge and competences of the examinees. Taking into consideration that automated assessment and testing, as well as handling of grade information in the form of performance metrics, is one of the most valuable features available to users of LMS, the improvement of available electronic examination tools, such as those presented in this chapter, might contribute in enhancing the value of LMS to the users and, thereby, promoting the wider adoption of LMS.

REFERENCES Anastakis, D. J., Cohen, R., & Reznick, R. K. (1991). The structured oral examination as a method for assessing surgical residents. American Journal of Surgery, 162, 67–70. doi:10.1016/00029610(91)90205-R Bennett, R. E., Rock, D. A., & Wang, M. (1991). Equivalence of free-response and multiple-choice items. Journal of Educational Measurement, 28, 77–92. doi:10.1111/j.1745-3984.1991.tb00345.x

193

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Bereby-Meyer, Y., Meyer, J., & Budescu, D. V. (2003). Decision making under internal uncertainty: The case of multiple-choice tests with different scoring rules. Acta Psychologica, 112, 207–220. doi:10.1016/S0001-6918(02)00085-9 Bereby-Meyer, Y., Meyer, J., & Flascher, O. M. (2002). Prospect theory analysis of guessing in multiple choice tests. Journal of Behavioral Decision Making, 15, 313–327. doi:10.1002/bdm.417 Bridgeman, B. (1991). Essays and multiple-choice tests as predictors of college freshman GPA. Research in Higher Education, 32, 319–332. doi:10.1007/BF00992895 Buchanan, T. (2002). Online assessment: Desirable or dangerous? Professional Psychology, Research and Practice, 33, 148–154. doi:10.1037/07357028.33.2.148 Bull, J., Conole, G., Davis, H. C., White, S., & Sclater, N. (2002). Rethinking assessment through learning technologies. In A. Williamson, C. Gunn, A. Young, & T. Clear (Eds.), Proceedings of the 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE2002) (Vol. 1, pp. 8-11). Auckland, New Zealand: UNITEC Institute of Technology. Bush, M. (1999). Alternative marking schemes for on-line multiple choice tests. Paper presented at the 7th Annual Conference on the Teaching of Computing, Belfast. Retrieved August 14, 2010, from http://www.caacentre.ac.uk/resources/bibliography/biblio2.shtml. Bush, M. E. (2006). Quality assurance of multiplechoice tests. Quality Assurance in Education, 14, 398–404. doi:10.1108/09684880610703974 Carvalho, M. K. F. (2009). Confidence judgments in real classroom settings: Monitoring performance in different types of tests. International Journal of Psychology, 44, 93–108. doi:10.1080/00207590701436744

194

Chu, L. F., Young, C. A., Ngai, L. K., Cun, T., Pearl, R. G., & Macario, A. (2010). Learning management systems and lecture capture in the medical academic environment. International Anesthesiology Clinics, 48, 27–51. doi:10.1097/ AIA.0b013e3181e5c1d5 Claroline Consortium. (2010). Claroline documentation. Retrieved December 28, 2010, from http://doc.claroline.net/en/index.php/Main_Page. Coombs, C., Milholland, J., & Womer, F. (1956). The assessment of partial knowledge. Educational and Psychological Measurement, 16, 13–37. Curzon, L. B. (1997). Teaching in further education: An outline of principles and practice (5th ed.). London, UK: Cassell. DeBord, K. A., Aruguete, M. S., & Muhlig, J. (2004). Are computer-assisted teaching methods effective? Teaching of Psychology, 31, 65–68. doi:10.1207/s15328023top3101_13 Ellis, R. K. (2009a). Field guide to learning management systems. Learning circuits, American society for training and development. Retrieved December 4, 2010, from http://www.astd.org/ NR/rdonlyres/12ECDB99-3B91-403E-9B157E597444645D/23395/LMS_fieldguide_20091. pdf Ellis, R. K. (2009b). Learning management systems 2009. Learning circuits and e-learning news, American society for training and development. Retrieved December 4, 2010, from http://www. astd.org/LC/2009/0509_LMS2009.htm. Ellis, R. K. (2010). Learning Management Systems 2010. Learning Circuits and E-Learning News, American Society for Training and Development. Retrieved December 4, 2010, from http://www. astd.org/LC/2010/0910_LMSsurvey.htm. Evans, L., Ingersoll, R. W., & Smith, E. J. (1966). The reliability, validity and taxonomic structure of the oral examination. Journal of Medical Education, 41, 651–657.

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Feldstein, M. (2004). What’s important in a learning content management system. eLearn Magazine. Retrieved December 28, 2010, from http://www.elearnmag.org/subpage.cfm?section =tutorials&article=4-1

Manning, S., & Dix, A. (2008). Identifying students’ mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives. Computers & Education, 51, 1154–1171. doi:10.1016/j.compedu.2007.10.010

Freeman, R., & Lewis, R. (1998). Planning and Implementing Assessment. London, UK: Kogan Page.

Mattheos, N., Stefanovic, N., Apse, P., Attstrom, R., Buchanan, J., Brown, P., & Walmsley, A. D. (2008). Potential of Information Technology in dental education. European Journal of Dental Education, 12(Suppl. 1), 85–91. doi:10.1111/j.16000579.2007.00483.x

Ganguli, S., Pedrosa, I., Yam, C.-S., Appignani, B., Siewert, B., & Kressel, H. Y. (2006). Part I: Preparing first-year radiology residents and assessing their readiness for on-call responsibilities. Academic Radiology, 13, 764–769. doi:10.1016/j. acra.2006.02.057 GUnet Asynchronous eLearning Group (2010). Open e-class asynchronous e-learning platform Platform description. Greek Academic Network. Retrieved December 28, 2010, from http://eclass. gunet.gr/manuals/OpeneClass23_en.pdf. IMS Global Learning Consortium. (2006a). IMS question and test interoperability overview, version 2.1 public draft (revision 2) specification. Retrieved December 5, 2010, from http:// www.imsglobal.org/question/qtiv2p1pd2/imsqti_oviewv2p1pd2.html IMS Global Learning Consortium. (2006b). IMS question and test interoperability assessment test, section, and item information model. Version 2.1 public draft revision 2 specification. Retrieved December 5, 2010, from http://www.imsglobal.org/ question/qtiv2p1pd2/imsqti_infov2p1pd2.html Lukhele, R., Thissen, D., & Wainer, H. (1994). On the relative value of multiple-choice, constructed response, and examinee-selected items on two achievement tests. Journal of Educational Measurement, 31, 234–250. doi:10.1111/j.1745-3984.1994.tb00445.x

McCoubrie, P., & McKnight, L. (2008). Single best answer MCQs: A new format for the FRCR part 2a exam. Clinical Radiology, 63, 506–510. doi:10.1016/j.crad.2007.08.021 McKenzie, C. R. M., Wixted, J. T., Noelle, D. C., & Gyurjyan, G. (2001). Relation between confidence in yes-no and forced-choice taks. Journal of Experimental Psychology. General, 130, 140–155. doi:10.1037/0096-3445.130.1.140 McLean, L. D., Dauphinee, W. D., & Rothman, A. (1988). The oral examination in internal medicine of the Royal College of Physicians and Surgeons of Canada: A reliability analysis. Annals of the Royal College of Physicians and Surgeons of Canada, 21, 510–514. Pressley, M., Ghatala, E. S., Woloshyn, V., & Pirie, J. (1990). Sometimes adults miss the main ideas and do not realize it: Confidence in responses to short answer and multiple-choice comprehension questions. Reading Research Quarterly, 25, 232–249. doi:10.2307/748004 Race, P. (2005). Making Learning Happen. London, UK: Sage. Scharf, E. M., & Baldwin, L. P. (2007). Assessing multiple choice question (MCQ) tests - A mathematical perspective. Active Learning in Higher Education, 8, 31–47. doi:10.1177/1469787407074009

195

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Stergiopoulos, C., Tsiakas, P., Kaitsa, M., Triantis, D., Fragoulis, I., & Ninos, C. (2006b). Methods of electronic examination applied to students of electronics. Comparison of results with the conventional (paper-and-pencil) method. In J. A. Moinhos Cordeiro, V. Pedrosa, B. Encarnação, & J. Filipe (Eds.), Proceedings of the 2nd International Conference on Web Information Systems and Technologies: Internet Technology / Web Interface and Applications(WEBIST 2006) (pp. 305-311). Setubal, Portugal: INSTICC Press Stergiopoulos, C., Tsiakas, P., Triantis, D., & Kaitsa, M. (2006a). Evaluating electronic examination methods of students of electronics. Effectiveness and comparison to the paper-and-pencil Method. In Proceedinsg of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC 2006) (Vol. 2, pp. 143-151). Los Alamitos, CA: IEEE Computer Society. Thomas, P. G. (2003). Evaluation of electronic marking of examinations. In D. Finkel, (Ed.), Proceedings of the 8th ACM Annual Conference on Innovation and Technology in Computer Science Education (ItiCSE 2003) (pp. 50-54). New York, NY: ACM. Triantis, D., Anastasiadis, C., Tsiakas, P., & Stergiopoulos, C. (2007). Asyncronous teaching methods and electronic assessment applied to students of the module “Physics of semiconductor devices”. Journal of Materials Education, 29, 133–140. Triantis, D., Stavrakas, I., Tsiakas, P., Stergiopoulos, C., & Ninos, D. (2004). A pilot application of electronic examination applied to students of electronic engineering: Preliminary results. WSEAS transactions on advances in engineering education, 1, 26-30.

196

Tsiakas, P., Stergiopoulos, C., Nafpaktitis, D., Triantis, D., & Stavrakas, I. (2007). Computer as a tool in teaching, examining and assessing electronic engineering students. In Proceedings of the International Conference of Computer as a Tool (IEEE-EUROCON 2007) (pp. 2490-2497). Van der Linden, W. J., & Glas, C. A. W. (2000). Computerized adaptive testing: Theory and practice. Heidelberg, Germany: Springer. Ventouras, E., Triantis, D., Tsiakas, P., & Stergiopoulos, C. (2010). Comparison of examination methods based on multiple-choice questions and constructed-response questions using personal computers. Computers & Education, 54, 455–461. doi:10.1016/j.compedu.2009.08.028 Wainer, H., & Thissen, D. (1993). Combining multiple-choice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education, 6, 103-1 18. Wainer, H., Wang, X.-B., & Thissen, D. (1994). How well can we equate test forms that are constructed by examinees? Journal of Educational Measurement, 31, 183–199. doi:10.1111/j.1745-3984.1994.tb00442.x Wass, V., Wakeford, R., Neighbour, R., & Van der Vleuten, C. (2003). Achieving acceptable reliability in oral examinations: An analysis of the Royal College of General Practitioners membership examination’s oral component. Medical Education, 37, 126–131. doi:10.1046/j.13652923.2003.01417.x Wood, R. (1991). Assessment and testing: A survey of research. Cambridge, UK: Cambridge University Press.

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

ADDITIONAL READING Downing, S. M. (2003). Validity: On meaningful interpretation of assessment data. Medical Education, 37, 830–837. doi:10.1046/j.13652923.2003.01594.x Downing, S. M. (2004). Reliability: On the reproducibility of assessment data. Medical Education, 38, 1006–1012. doi:10.1111/j.13652929.2004.01932.x Haladyna, T. M. (2004). Developing and validating multiple-choice test items. Mahwah, NJ: Lawrence Erlbaum Associates. Hautau, B., Turner, H. C., Carroll, E., Jaspers, K., Parker, M., & Krohn, K. (2006). Differential daily writing contingencies and performance on major multiple-choice exams. Journal of Behavioral Education, 15, 259–276. Kennedy, P., & Walstad, W. B. (1997). Combining multiple-choice and constructed-response test scores: An economist’s view. Applied Measurement in Education, 10, 359–375. doi:10.1207/ s15324818ame1004_4 Leung, S. F., Mok, E., & Wong, D. (2008). The impact of assessment methods on the learning of nursing students. Nurse Education Today, 28, 711–719. doi:10.1016/j.nedt.2007.11.004 Muijtjens, A. M. M., van Mameren, H., Hoogenboom, R. J. I., Evers, J. L. H., & van der Vleuten, C. P. M. (1999). The effect of a ‘‘don’t know’’ option on test scores: Number-right and formula scoring compared. Medical Education, 33, 267–275. doi:10.1046/j.1365-2923.1999.00292.x Norton, J. M. (1996). A comparison of methods for dealing with troublesome examination questions. Advances in Physiology Education, 271, 55–60.

Philip, A., & Arnold, M. M. (2007). How many questions should I answer? Using bias profiles to estimate optimal bias and maximum score on formula-scored tests. The European Journal of Cognitive Psychology, 19, 718–742. doi:10.1080/09541440701326121 Schubert, S., Schnabel, K. P., & Winkelmann, A. (2009). Assessment of spatial anatomical knowledge with a ‘3-dimensional multiple choice test’ (3D-MC). Medical Teacher, 31, e13–e17. doi:10.1080/01421590802334325 Scouller, K. (1998). The influence of assessment method on students’ learning approaches: multiple-choice question examination versus assignment essay. Higher Education, 35, 453–472. doi:10.1023/A:1003196224280 Scouller, K. M., & Prosser, M. (1994). Students’ experiences in studying for multiple choice question examinations. Studies in Higher Education, 19, 267–279. doi:10.1080/0307507941233138 1870 Tan, L. T., & McAleer, J. J. A. (2008). The introduction of single best answer questions as a test of knowledge in the final examination for the fellowship of the Royal College of Radiologists in Clinical Oncology. Clinical Oncology, 20, 571–576. doi:10.1016/j.clon.2008.05.010 Tarrant, M., Knierim, A., Hayes, S. K., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education Today, 26, 662–671. doi:10.1016/j.nedt.2006.07.006 Traub, R. (1993). On the equivalence of the traits assessed by multiple choice and constructedresponse tests. In Bennett, R. E., & Ward, W. C. (Eds.), Construction versus choice in cognitive measurement (pp. 29–44). Hillsdale, NJ: Lawrence Erlbaum Associates.

197

Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

Valdivia, R., & Nussbaum, M. (2009). Using multiple choice questions as a pedagogic model for face-to-face CSCL. Computer Applications in Engineering Education, 17, 89–99. doi:10.1002/ cae.20196 Van der Watering, G., Gijbels, D., Dochy, F., & van der Rijt, J. (2008). Students’ assessment preferences, perceptions of assessment and their relationships to study results. Higher Education, 56, 645–658. doi:10.1007/s10734-008-9116-6 Wallace, M., & Williams, R. L. (2003). Multiplechoice exams: Explanations for student choices. Teaching of Psychology, 30, 136–139. Walubo, A., Burch, V., Parmar, P., Raidoo, D., Cassimjee, M., & Onia, R. (2003). A model for selecting assessment methods for evaluating medical students in African medical schools. Academic Medicine, 78, 899–906. doi:10.1097/00001888200309000-00011

198

Zeidner, M. (1987). Essay versus multiple choice type classroom exams: The students’ perspective. The Journal of Educational Research, 80, 352–358.

KEY TERMS AND DEFINITIONS Constructed-Response Questions: A question that requires the respondent to create his/her own response. Electronic Examinations: A formal test involving answering questions and/or solving exercises using electronic media. Multiple-Choice Question: A question that is accompanied by a list of possible responses, from which the respondent selects the answer(s). Scoring Rules: The procedures used in the process of assigning a score for evaluating the performance in a test.

199

Chapter 10

Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility Lourdes Moreno Universidad Carlos III de Madrid, Spain Ana Iglesias Universidad Carlos III de Madrid, Spain Rocío Calvo Universidad Carlos III de Madrid, Spain Sandra Delgado Universidad Carlos III de Madrid, Spain Luis Zaragoza News Service, Radio Nacional de España, Spain

ABSTRACT Currently, the great majority of institutions of higher education use Learning Content Management Systems (LCMSs) and Learning Management Systems (LMS) as pedagogical tools. In order to make these systems accessible to all students, it is important to take into account not only educational standards, but also standards of accessibility. It is essential to have with procedures and well-established method for evaluating these tools, so in this paper we propose a method for evaluating the accessibility of LCMSs and LMS based on a consideration of particular accessibility standards and other technological and human aspects. The method application is for all LMS, in order to illustrate the effectiveness of the evaluation method, we present a case study over the widely-used LMS Moodle1. In the case study, the accessibility of Moodle is evaluated thoroughly from the point of view of visually-impaired persons. The results obtained from the case study demonstrate that this LMS is partially accessible. The evaluation shows that the tool provides poor support to the authors of accessible educational contents. DOI: 10.4018/978-1-60960-884-2.ch010

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Disability Standards and Guidelines for Learning Management Systems

INTRODUCTION Information and Communication Technology (ICT) plays a key role in people’s daily lives (Rößling G. et al, 2008), a fact that is equally true of people with and without disabilities. Over the past few years in the education sector and, more specifically, in institutions of higher education, Learning Management Systems (LMS) and Learning Content Management Systems (LCMSs) have become extremely popular pedagogical tools for teachers and students. Such is the current popularity of LMS in these institutions, in fact, that LMS are oftentimes the only tool given to students for communicating with peers and teachers or for accessing particular learning resources. Therefore, the negative impact of an inaccessible LMS on the learning experience of students with disabilities would be large indeed. In order to provide equal opportunities to all students, it is necessary to improve the learning environment by removing all barriers to accessibility. LMS, LCMSs and their learning contents should be available to all students and teachers, including those with disabilities and regardless of their particular accessibility needs. In the effort to make software completely accessible to all types of users, it must be taken into account that certain individuals require the use of Assistive Technologies (ATs) such as screen readers, refreshable Braille displays, speech synthesizers, magnifiers, adaptable keyboards or voice recognition software in order to see, hear, move or interact with the system and its contents. In addition to covering the widest range of user abilities, software should also take user preferences and learning styles (e.g., visual, auditory or tactile) into account. The development of software in this way would allow all users, not just individuals with disabilities, to universally benefit from system contents (Moreno, L. et al, 2008). Therefore, in order to ensure the achievement of this goal in the context of institutions of higher learning, it is necessary to design and develop LMS and LCMSs according to standards that facilitate

200

universal access and, at the same time, promote correct technological growth (Fichten, C.S., 2009). Moreover, evaluations of the accessibility of these LMS and LCMSs and the certification of their compliance with accessibility standards should also be required. In the following section of this chapter, specific technologies, accessibility standards and previously published work regarding LMS accessibility is discussed at length. In the third section, a new method for the evaluation of the compliance of an LMS with previously examined accessibility standards is proposed. This evaluation method is then put into practice in the fourth section for the Moodle LMS whose accessibility, specifically for visually-impaired individuals, is tested by an accessibility expert and a visually-impaired enduser (with the assistance of JAWS screen reader). Finally, the fifth section presents briefly general conclusions from the case study as well as areas for future research.

BACKGROUND For the present study, we have considered a wide variety of previous published works on accessibility standards and regulations, LMS incorporating accessibility requirements into their design, as well as studies of LMS accessibility evaluation methods. With regard to this last point, the usercentered design (UCD) approach is considered and developed here.

E-Learning and Accessibility Standards In order to make educational resource applications and web sites universally accessible for all users, not only educational standards like the Sharable Content Object Reference Model (SCORM), but also accessibility standards like the Instructional Management System (IMS) guidelines for developing accessible learning applications2, the World

Disability Standards and Guidelines for Learning Management Systems

Wide Web Consortium (W3C) standards3, and the Web Accessibility Initiative (WAI) accessibility guidelines4 must be followed. More specifically, in order to ensure the accessibility of authoring tools like LMS or LCMSs, WAI Web Content Accessibility Guidelines (WCAG) (W3C, 2008) and Authoring Tool Accessibility Guidelines (ATAG) (W3C, 2010 a) should be closely followed, as well. The WCAG guidelines are the most important components of the WAI. They are considered the official standard in the European Union and are referenced in most legislation worldwide. Other important initiatives exist, as well, such as U.S. legislation (29 U.S. Code §794d) requiring conformance with U.S. Section 508 technical standards, some of which relate specifically to web accessibility. Although less extensive, these standards are nevertheless quite similar to the WCAG and studies exist in the accessibility literature that map the section 508 with the WCAG. While most current regulatory frameworks still reference the WCAG version 1.0, in December 2008 the WCAG 2.0 was published as a new W3C Recommendation. According to the W3C, the ATAG Recommendation is the method for evaluating accessibility requirements in LMS. The ATAG directs web designers and developers in producing accessible output (i.e., web pages) that meets standards and guidelines. Moreover, they provide guidance for prompting the content author (i.e., the authoring tool user) for accessibility related information. Furthermore, they provide ways of checking and correcting inaccessible content, and integrating accessibility into the overall “look and feel” of the final software. Finally, these guidelines provide help and documentation to make the authoring tool itself accessible for people with disabilities. While the version ATAG 2.0 is has been developed (as a working draft) to be compatible with WCAG 2.0, ATAG 1.0 continues to be the current W3C Recommendation. According to ATAG 2.0, the LCMS user interface should be accessible and,

at the same time, support the production of accessible content as specified in WCAG. We use ATAG 2.0 in sections three and four of this study.

Accessibility Metadata and Models Considering adaptability as an additional resource of accessibility, some learning platforms are based on the use of metadata, such as that of the Dublin Core Adaptability Statement and IMS Global Learning Consortium5. With respect to this adaptability approach, the most important educational standards are ISO/IEC JTC1/SC36/ WG7 “Access for All” and the IMS AccessForAll Metadata Specification (AccMD)6.

User-Centered Design The user-centered design (UCD) approach is a process in which the needs and preferences of end-users of a product are taken into account when that product is designed. The approach is particularly useful to consider in evaluations of the accessibility of LMS, and its inclusion in the evaluation process is given methodological support by standards such as ISO 9241-210:2010 (which replaced ISO 13407:1999)7 . In order to fully consider the access-specific characteristics of persons with disabilities and the diverse contexts of use that may create barriers to web accessibility (Newell, AF. & Gregor, P., 2000), the UCD approach may be further extended to that of Inclusive Design (Henry, S. 2007). The WAI indicates following this approach in the accessibility evaluation process (W3C, 2010 c). The evaluation process presented in this chapter uses this approach, promoting the participation of end-users with disabilities as well as accessibility experts.

201

Disability Standards and Guidelines for Learning Management Systems

Assistive Technology Disabled users interact with computers through Assistive Technologies (ATs) and, as mentioned earlier, it is essential to take ATs into account when the UCD approach is followed in the evaluation process. Many different types of ATs exist to respond to the diversity of needs of users with disabilities. An example of an AT for users with vision impairment is a screen reader which describes in audio the visible information appearing on a computer screen. For blind users, some specific User Interface (UI) features are required, such as providing a full control of interface elements, as well as easy and rapid navigation via keyboard. If web sites are designed in accordance with WAI Guidelines, the accessibility requirements for ATs will be included. In such cases, therefore, ATs should successfully interpret and relay back to the user the semantic mark-up included in the web pages with WAI-ARIA (W3C, 2010 d). The case study presented later in this chapter considers an LMS accessed with a screen reader. Screen readers, such as JAWS8 or Window-Eyes9, are commercial, special-purpose software programs. The Linux Screen Reader10, the Orca screen reader11 for the GNOME platform and the NVDA screen reader for Windows12 are free alternatives to these commercial products. Additionally, Fire Vox13 is a free extension to the Firefox web browser that provides screen reading functionality, the HearSay web browser14 is a standalone self-voicing web browser, and aiBrowser15 is a self-voicing web browser attempting to make multimedia web content accessible to blind users. Nevertheless, due to the expense of commercial screen readers and the general lack of knowledge in the non-visually-impaired community about their importance or the availability of free alternatives, screen readers are seldom installed on computers not normally and predominantly used by blind individuals.

202

For the evaluation presented later in this study, a JAWS screen reader was used. JAWS is a screen reader which is able to recite aloud almost everything encountered on a web screen. It is compatible with the Microsoft Windows operating system and IBM Lotus Symphony. It uses an adjustable synthetic voice to give feedback in the form of aural translations of user keyboard commands, as well as of text/graphics depicted on the Internet. Any text, graphics or links are automatically recited by JAWS at variable speeds and volumes. JAWS was chosen here due to its wide use, configuration options according to the needs and preferences of each user, and versatility in achieving better functioning and monitoring of various applications.

LMS AND ACCESSIBILITY LMS Accessibility Studies Some LMS or LCMSs such as Moodle, dotLRN16 or Atutor17, are designed with consideration for accessibility standards and problems. Most often, such systems are based on learning standards like SCORM and IMS and on accessibility standards like ATAG and WCAG. Moreover, some important initiatives carried out in the area of accessibility and online learning systems have also been developed (Martin, L et al, 2007), (Iglesias, A. et al, 2009). Other LCMSs found in the literature include Dokeos18, Docebo19, Sakai20, and Blackboard21. For the evaluation of the accessibility of an LMS presented in section four below, Moodle was chosen due to the fact that it is one of the most widely-used LMS in the world (the Moodle. org website indicates that there are currently over 49,000 registered Moodle sites in 210 countries with over 34 million total users). Developed at the Open University of the United Kingdom which includes close to 9,300 individuals with disabilities among the total number of students

Disability Standards and Guidelines for Learning Management Systems

enrolled, Moodle is an open-source, flexible software focused on providing accessibility to all users. The software offers a large spectrum of available tools including, but not limited to, course resources, assignment modules, blogs, chats, forums, glossaries, interaction, SCORM packages, surveys, quiz modules, and wikis. Due to the complexity of LMS, very few accessibility evaluations have been conducted to date. Of the evaluations conducted, however, the problems identified were often similar to those of static web sites. These problems are compounded by a poor navigational structure, color contrast problems and other significant problems related to a mark-up which conflicts with user agents and ATs. Furthermore, only a few studies have been undertaken to determine the general types of accessibility barriers common to most LMS and considered as authoring tools. Most of these studies present the following two particular drawbacks. First, since the majority of these studies are based on survey and interview data rather than on empirical results, it has been difficult to make a convincing case at institutions of higher learning for improved accessibility awareness, training and procedures. Second, while all of the studies found present evaluations with respect to the WCAG, none do the same with respect to the ATAG. Regarding particular studies, in Kalnins-Cole & Peters (Kalnins-Cole, T. & Peters, D., 2007) an evaluation of dotLRN is presented along with recommendations for development changes that would make dotLRN compliant with the W3C international accessibility standards. Results from the study show that dotLRN nearly possesses Level A compliance (according to the international standards) and that full Level A compliance would be easy to achieve. The study identifies Level AAA compliance as a desirable long-term goal. In Power et al. (Power, C. et al., 2010), an initial empirical investigation into accessibility

problems present in three different virtual learning environments – Moodle, dotLRN and Blackboard – is conducted. Nevertheless, the study is rather limited in that it only takes into account some examples of web pages and only evaluates a small subset of Moodle tasks with respect to an equally small subset of WCAG 1.0 checkpoints. In Santos et al. (Santos, O.C. et al, 2007), results obtained in the study of an e-learning framework with an initial end-user evaluation are presented. Additionally, very few studies exist in the literature that consider the use of screen readers by the visually-impaired. In (Buzzi C. et al, 2009), the compliance of Moodle software with WCAG 1.0 and the use of WAI-ARIA Suite is evaluated; however, and as in Power et al., only a small subset of Moodle tasks is taken into account. Finally, the study does not check for Moodle’s compliance with the ATAG. To our knowledge, no evaluation studies of the accessibility of LMS or LCMSs exist using the ATAG. This paper tries to address the limitations of previous studies, therefore, by presenting an evaluation process for the Moodle LCMS as accessed with a JAWS screen reader. Moreover, the study is based on accessibility criteria in instructional design and universal design principles (Elias, T., 2010).

LMS Accessibility Evaluation Methodology The ATAG offer a resource for evaluating the extent to which accessibility requirements are met by authoring tools. An LMS or LCMS is an authoring tool for building educational software, so ATAG should be taken into account if the software is to be accessible to all students. As discussed in the prior sub-section, while a number of studies of evaluation methodologies for the WCAG can be found in the literature and the WAI, such is not the case with the ATAG. To our

203

Disability Standards and Guidelines for Learning Management Systems

knowledge, a formal evaluation methodology for authoring tools needs to be proposed. It is the main aim of the present paper: to offer a new proposal of accessibility evaluation methodology for LMS. The new proposal presented in this paper extrapolates the WCAG evaluation methodology from WAI (W3C, 2010 c) to be applied while taking into account the ATAG guidelines. It is important to note that ATAG 2.0 includes testable success criteria, techniques, and references to WCAG 2.0, so such an extrapolation is possible. While software tools exist that allow for the automatic checking of the WCAG conformance level, no such tools are available for the ATAG. Even with the existing tools, however, there are many aspects of accessibility that can only be evaluated manually. Thus, the combination of manual and automatic methods, together with end-user testing following the UCD approach, guarantees the detection of barriers to accessibility as well as the later implementation of more effective accessibility solutions. In general, three types of accessibility testing techniques exist (Abou-Zahra, S. 2008) which obtain the best results when combined in such a way that capitalizes on the specific advantages of each. These three techniques are the following: •

•

204

Automated Testing. This technique is carried out by software tools without the need for human intervention. In automated testing, the syntactic structure of web content is analyzed (e.g., checking for alt attributes in HTML elements). The testing technique is quite cost-effective. However, an important drawback is that it only addresses a subset of the accessibility provisions set out by most standards. Manual Testing. This technique is carried out by human evaluators who could be experts or novices, depending upon the nature of the accessibility test. In practice, most tests carried out by human evaluators are also frequently guided or supported by

•

automated software tools. The principal advantage offered by manual testing is that it covers a broader range of accessibility provisions than that considered by automated testing. For example, manual testing can check if the alt attribute accurately describes the purpose of its corresponding image or contains typical default texts such as ‘‘image’’. End-User Testing. This technique is carried out by real-world end-users. Logically, the technique focuses on the end-users and how well specific technical solutions match their needs in a specific context. This testing technique easily complements the other techniques previously described.

The goal of accessibility standards is to isolate accessibility problems and to define provisions for avoiding them. Some accessibility provisions, but not all, can be detected by using automated testing. Moreover, it is important to identify the underlying causes of an accessibility problem. Sometimes accessibility problems exist with web content, but other times they originate with the browser, ATs or even the user’s inability to handle the tool. With the use of these different techniques, a particular method for the evaluation of authoring tools as LMS is proposed and described in the following section.

Accessibility Evaluation Method for Lms This chapter presents a new method of LMS evaluation based on WAI recommendations. Following the consideration of the many different existing techniques for manual and automated testing, it was concluded that a modified version of the WCAG evaluation process would be most appropriate here. This modified evaluation method includes the following steps and elements:

Disability Standards and Guidelines for Learning Management Systems

• •

•

•

Set the ATAG 2.0 conformance level for which the evaluation is to be carried out. Identify web pages and tasks to be selected in the evaluation process, checking the main functionalities and contents of the LMS with manual testing by people most likely to enter the site. In LMS, these people can be classified into three groups or profiles: administrator, teacher and student. Combine and apply the distinct evaluation methods listed below and described in greater detail in the following sub-sections: ◦◦ ATAG Expert Evaluation Method (with automatic and manual testing) ◦◦ Expert Evaluation Testing ◦◦ End-User Testing Draw conclusions with a discussion of the types of problems encountered, as well as the best practices to be continued or extended. Indicate the method used to identify the problems discussed and finally, to recommend follow-up steps to be taken for every accessibility barrier found during the process (e.g., markup validation and other tests). This would help to test for full conformance and address any problem identified previously.

ATAG Expert Evaluation Method The method demonstrates compliance with the ATAG 2.0 Working Draft and the WCAG 2.0 Recommendation. The method comprises automated and manual tests, the latter of which being supported by automatic tools. The steps of the method are as follows: •

Automated evaluation. Use two general accessibility evaluation tools at least and note any problems indicated by the tools. These tools only exist for checking the WCAG.

•

Manual evaluation. Complete the following steps: ◦◦ Examine a selection of web pages using ATAG 2.0. As an additional resource for the checklist offered in the official documentation for the ATAG, this chapter has defined agile guides for the application of the ATAG, which are described in greater detail in the following section. ◦◦ Use several user agents to browse and check the following: ▪▪ Turn off images and check whether the appropriate alternative text is available. ▪▪ Turn off the sound and check that the audio content is still available through text equivalents. ▪▪ Use the browser controls to vary font-size, checking that the font size changes on the screen accordingly and that the page is still usable with larger font sizes. ▪▪ Test with different screen resolutions (i.e., by setting the application window to less than the largest available size) in order to check that horizontal scrolling is not required. ▪▪ Change the display color to grayscale and check whether the color contrast is adequate. ▪▪ Unplug the mouse and check that all links and form controls can be accessed by tabbing through them. Additionally, check that the links clearly indicate what they lead to. ▪▪ Read over the pages and check whether the text is clear and simple.

205

Disability Standards and Guidelines for Learning Management Systems

▪▪

◦◦

Use a text browser (e.g., Lynx) and check if the information available through the GUI and text browsers is the same. Also, check if the information in the text browser is presented in a meaningful order if read serially. Summarize the results.

As discussed in previous sections of this chapter, the ATAG establish the accessibility guidelines for the design of web content authoring tools. These guidelines are organized into two parts which specify the following: (1) the authoring tool should be accessible and usable for authors with disabilities and (2) the authoring tool should permit, support, promote and guarantee the production of accessible web content. Similar to the WCAG, the ATAG establish three different levels of conformance: Level A, Level AA and Level AAA. For the elaboration of agile guides for accessibility evaluation process, an analysis of official documentation for the ATAG was carried out. Results revealed that the application of the guidelines

in the order given in the documentation was quite heavy-handed. The evaluation process was quite long and tedious and required the repetition of particular tasks at distinct moments. The performance of these tasks could be easily improved, with the definition of concrete application paths and guidelines. Using the Working Draft of July 8th, 2010 (W3C, 2010 a) an agile guide for the ATAG was elaborated and is presented below in Table 1. As can be observed, each of the different groups presented are applied sequentially. The process described above has been created to optimize and make more agile the accessibility evaluation process of authoring tools. Relative to evaluations carried out prior to those discussed in this chapter, the application of these agile guides resulted in a considerable reduction in evaluation times and a greater understanding of the guidelines.

Expert Evaluation Testing The method includes manual testing carried out by evaluators (1) with intimate knowledge of how people with disabilities use the web and (2) who

Table 1. Agile guidelines to evaluate ATAG 2.0 GROUP

DESCRIPTION

SPECIFIC GUIDELINES (ATAG)

Part A: Make the authoring tool accessible for the user interface. Group #1: Content accessibility guidelines

Necessary for the evaluation of conformity with the WCAG. The first guidelines to be followed if the LMS authoring tool is to be accessible.

A.1.1.1, A.3.1.2, A.3.2.1, A.3.2.2, A.3.3.1, A.4.1.1, A.4.1.2, A.1.1.2, A.3.1.3, A.4.1.3, A.1.1.3, A.3.1.4

Group #2: Accessibility standards: Clients

Guidelines corresponding to the evaluation of accessibility requirements concerning the authoring tool user interface

A.2.1.1, A.3.1.1

Group #3: Accessibility standards: Non-web tools

Guidelines corresponding to the evaluation of accessibility requirements for non-web tools.

A.1.2.1, A.2.2.2, A.2.2.3, A.2.2.1

Group #4: Accessibility in configuration & structure

Guidelines corresponding to the evaluation of accessibility requirements regarding the configuration and structure of the tool.

A.2.3.1, A.3.2.2, A.3.2.3, A.3.4.1, A.3.4.2, A.3.4.3, A.3.4.4, A.3.5, A.3.6.1, A.3.6.3, A.3.6.3, A.3.7.2, A.3.7.1

Group #5: Documentation accessibility

Guidelines related to accessibility documentation facilitated by authoring tool.

A.4.2.1, A.4.2.2

Part B:Support the production of accessible content. Group #6: Production of accessible content

206

The same ATAG 2.0 guidelines are included and sequentially applied to groups with attention paid to the conformity level divisions A, AA and AAA.

Disability Standards and Guidelines for Learning Management Systems

can accurately identify problems related to user interaction. Similar to walkthroughs and heuristic evaluations in the field of usability engineering, expert evaluators anticipate the issues that endusers may encounter in the content. In this method, manual evaluation is conducted using simulation (e.g., testing keyboard access, the same view in several user agents) and screening techniques (e.g., testing accessibility using a screen reader).

and using shortcut keys, are able to avoid accessibility barriers identified by accessibility experts. Additionally, accessibility experts may discover particular accessibility barriers previously passed over unnoticed by end-users due to the latter’s extensive experience with screen reader software.

End-User Testing

In this sub-section, the main results obtained following the execution of the evaluation method are presented. The evaluation of the compliance of the Moodle LMS with ATAG 2.0 presented here was performed by two accessibility experts supported by the automatic tools TAW22, AChecker23, and the Web Accessibility Toolbar (AIS for Internet Explorer24 and the Web Developer extension for Firefox25). In the evaluation, a direct installation of the official Moodle version 1.9 packages was made, and all basic web pages resulting from the installation were analyzed. While the inclusion of all possible content in Moodle web pages was not tested here, some different types were included and tested in order to verify Moodle’s ability to create accessible content (Part B of the ATAG). A more detailed description of this evaluation is presented below. In Part A of ATAG 2.0 Working Draft, all success criteria must present authoring tool user interface-related accessibility problems, this main that the authoring tool user interface must conform to WCAG. In Part B, all success criteria must present accessible web content production issues. The result obtained in the accessibility evaluation of Moodle indicates that the lower accessibility level “A” of ATAG is not achieved. In relation to the evaluation of compliance of Moodle with WCAG 2.0, if Conformance Level “A” is obtained, then the success criteria A.1.1.1 of ATAG is fulfilled, but the A.1.1.2 and A.1.13 are not. Figure 1 and Figure 2 show the several levels of conformance resulting from the evaluation of Moodle with respect to ATAG 2.0 and WCAG 2.0

Testing here is carried out by real-world end-users. It includes the evaluation of accessibility with ATs (W3C, 2010 b). User testing distinguishes between formal and informal checks, with the former usually being carried out by professionals following well-established procedures (surveys, interviews …) and the latter by non-experts. In the following section, the method outlined here is used in a practical example of LMS accessibility evaluation.

CASE STUDY: EVALUATION OF ACCESSIBILITY IN MOODLE LMS This section presents the method and the results obtained from an evaluation of the accessibility of the Moodle LMS. The evaluation of the LMS is performed with respect to the ATAG and the use of the JAWS screen reader AT resource. Following the method described in the previous section, sub-section 4.1 examines the evaluation method using the ATAG, while the sub-section 4.2 presents an expert evaluation and user testing using both JAWS resource and combining the results in order to obtain tighten conclusions. In this case, the combined results improve the conclusions due to the fact that the problems and difficulties founded by end-users and experts accessing a product via an AT are not the same.. For example, end-users accessing an LMS via JAWS, due to their experience using non-accessible web-sites

Evaluation Method: Checking Compliance with ATAG 2.0

207

Disability Standards and Guidelines for Learning Management Systems

Figure 1. Number of ATAG (Part A and B) guidelines: fulfilled (Yes), did not fulfill (No) and did not apply (N/A)

Figure 2. Number of WCAG guidelines: fulfilled (Yes), did not fulfill (No) and did not apply (N/A)

One feature obtained of evaluation of the compliance of Moodle with WCAG 2.0 is that in the user interface of the Moodle authoring tool for non-text content present to the user, Moodle shows an alternative text that serves the same purpose (Success Criteria 1.1.1. WCAG 2.0 [Level A) is fulfilled). The functionality of the Moodle interface is operable through a keyboard (Success Criteria 2.2.1 of WCAG 2.0 [Level A] is fulfilled and Success Criteria 3.3.1 of ATAG 2.0 [Level A] is fulfilled too).

208

It is important to emphasize that Moodle in the assistant form with the help of users is able to avoid and correct mistakes. If an input error is automatically detected, the presenting the accessibility error is identified and then the error is described to the user in text as shown in Figure 3. In the same way, labels or instructions are provided when content requires user input (Success Criteria 3.3.1 and 3.3.2 of WCAG 2.0 [Level A] is fulfilled). However, if an input error is automatically detected and suggestions for correction are known, the suggestions are not

Disability Standards and Guidelines for Learning Management Systems

Figure 3. Assistance in Moodle interface for forms: Help users correct mistakes

provided to the user (Success Criteria 3.3.3 of WCAG 2.0 [Level AA] is not met). Moodle uses tables for the layout for the main content of the web page. However, the content is understandable when it is read line to line, so there are no accessibility problems in this case. Additionally, there is a mechanism to bypass blocks of content that are repeated on multiple web pages (see Figure 4 the link: “Skip to main content”). The most important accessibility barrier found during the accessibility evaluation of Moodle’s taking into account ATAG 2.0 is the failure to meet Guideline B.2.5: “Assist authors with accessible templates and other pre-authored content (Level AA)”. Moodle offers multiple types of themes (i.e., templates) to assign, but it does not provide a default theme with an accessible interface in compliance with WCAG 2.0 Level AA. Therefore, B.2.5.8 is not fulfilled. Additionally, Moodle does not provide support for creating accessible web content. Therefore some guidelines of Part B of ATAG 2.0 are not fulfilled. Moodle provides a default WYSIWYG editor. The main problem found in the default editor used by Moodle is that the colors and fonts are included in HTML with the HTML element, rather

than by using CSS styles. Consequently, some WCAG 2.0 guidelines are not fulfilled. Additionally, if a user wants to add an image, the editor allows the user to include an alternative description which satisfies point 1.1.1 (Level A) of WCAG 2.0. Nevertheless, Moodle does not provide warning advices if the user does not include an alternative description for the image. Moodle does not prevent the user from making accessibility errors, so it does not satisfy the B.2.2 and B.2.3 guidelines from ATAG 2.0. Other observed feature is about “lang” HTML attribute, Moodle WYSIWYG editor include it indicating the language of the system’s contents. Nevertheless, Moodle incorrectly uses the

HTML element, using it with the sole intention of achieving a visual effect and not for the real aim of the tag, which is to identify a text block as a quote. It does not comply with the 3.1.2 (Level AA) success criteria of WCAG 2.0. One of the most important problems related to the use of JavaScript in Moodle is that when it is disabled in the user agent, it is not possible to use the WYSIWYG editor. As an alternative, Moodle gives a text area to edit the content. However, although a text area for plain text is an alternative

209

Disability Standards and Guidelines for Learning Management Systems

Figure 4. Mechanism in Moodle to bypass blocks of content that are repeated on multiple web pages

for editing contents, users usually make mistakes due to their lack of knowledge of accessibility guidelines resulting in an editing process which may not be accessible (2.3.3 of ATAG cannot be fulfilled). For instance, by disabling the use of its default editor, Moodle permits one to choose the background color of the editor’s text area, an interesting option if it were necessary to achieve an ideal contrast of colors. In the present evaluation, Moodle demonstrates only partial accessibility and does not comply with the WCAG or ATAG. Accessibility strategies must, therefore, be included in the design of future versions in order to obtain an LMS that is more accessible for all users.

Expert Evaluation and User Testing The evaluation of the accessibility of Moodle was performed in July of 2010 by an accessibility expert. End-user testing was conducted by a visually-impaired person (since childhood) and habitual user of JAWS. Both the expert and end-user made a direct installation from official

210

Moodle packages. Moodle 1.9 and JAWS 10.0 versions were used here. A testing procedure with a list of tasks was completed in the evaluation process, checking the main functionalities and contents of Moodle. Both end-users and the accessibility expert interacted with Moodle – with the expert checking the list of tasks with and without JAWS – and in the same manner so that interaction results across the LMS would be comparable. Appendix 1 shows a table (Table 2) with the tasks evaluated in column 3. This table presents the type(s) (i.e., profile) of user with the ability to execute the task (e.g., administrator [A], teacher [T], student [S]). With regard to this final category, while all these tasks can be performed by a Moodle user with an administrator profile, users with teacher or student profiles may execute only a subset of tasks. In the evaluation of accessibility, the complete set of Moodle functionalities comprising the tasks that every user can perform was checked. Also, this table shows the functionality group in other column and the name of the task evaluated in another column. The fourth column offers brief descriptions of accessibility problems

Disability Standards and Guidelines for Learning Management Systems

where they arose. And finally, the last column shows if the individual tasks could be executed using JAWS and, in affirmative cases, whether such executions involved accessibility problems (Yes*) or no accessibility problems (Yes). In cases where accessibility problems using JAWS prevented the successful execution of the task, a different symbol (No) is used. As opposed to the related work previously described, the accessibility analysis presented in this study was performed on all basic web pages resulting from the direct installation of Moodle technology. This allowed us to assess Moodle’s full capacity for accessibility for visually-impaired users. As was briefly discussed earlier in the study, there is sometimes an overlap between barriers to accessibility and usability problems (Moreno, L. et al, 2009), entailing cognitive barriers to the successful use of the virtual learning environment. Some major usability problems directly affecting accessibility have been detected here and have been included in Table 2 together with accessibility barriers. Following the exhaustive evaluation process, it is quite clear that the combination of accessibility expert and end-user evaluation is extremely helpful for the detection of the principal accessibility problems present in Moodle. Furthermore, many of these accessibility problems such as the lack of descriptive tags and section headings for web pages and forms, the dependency upon the non-accessible Windows editor for long strings in the forms (e.g., descriptions), the changes to the look and feel of different tasks and the redirection to other web page or the refreshing of web page without previously notifying the user are repeated in multiple Moodle tasks. In sum, the errors detected are those presented below: • •

E1: Not all text and combo boxes contain associated descriptive texts. E2: Pages refresh without previously consulting the user.

• • • • •

•

• • • •

•

• •

•

E3: The user is redirected to another page without prior warning. E4: The look and feel of Moodle changes in some tasks. E5: Tables are used for layout. E6: Text images are used to convey information. E7: Information regarding how to complete a task is confusing for the user or difficult to obtain. E8: Some text in English appears even when Spanish is the selected language of the tool. E9: There is no button allowing the user to cancel the operation. E10: Tables are not well structured, creating problems for the screen reader. E11: There are no page or table headings. E12: The many rows in the table make the table difficult to read and require the user to memorize the table structure. E13: Text appears that can only be modified with Windows tool and it do not access by keyboard. E14: Some text descriptions are incorrect. E15: The application does not check data inserted into the form, and the errors messages are not useful for the users, making it difficult for users to imagine the cause of particular problems. E16: The screen reader does not always read the text correctly.

Table 2 presents the detail of all these errors including the tasks were they are found. As it can be observed, the tool contains many different obstacles to accessibility, the most commonly produced of which being E4 and E1. With these errors, the user may be confused since the appearance of the website is not always the same and the content is not always clear. The least frequently produced errors from the list are E12, E14, E15 and E16. Thus, while important, these errors

211

Disability Standards and Guidelines for Learning Management Systems

are nevertheless relatively insignificant insofar as they appear only once in the tool. It is very important to understand that these accessibility problems can seriously and negatively affect any type of Moodle user (i.e., administrators, teachers, and students), keeping the user from correctly and completely accessing the LMS.

CONCLUSION Learning Management Systems (LMS) and Learning Content Management Systems (LCMS) are nowadays necessary in institutions of higher education and should be accessible for users with and without disabilities. For this reason, accessibility standards must be considered. In this chapter, technology and disability standards for LMS have been examined in detail. In order to fully develop and use a resource to evaluate one such system, the most important LMS-related standards such as ATAG and WCAG have been described. In addition, the UCD approach has been introduced and recommended in order to take into account the participation of users with disabilities in the LMS accessibility evaluation process. In addition to the review of related literature, a new general method for evaluating the accessibility of an LMS based on automatic, expert and enduser testing has been proposed. The combination of several techniques has returned more complete result in this experimentation. With the objective of presenting a practical application of the method, a case study focusing on the Moodle LMS and visually-impaired end-users (i.e., those requiring screen readers) has been carried out and analyzed. The results obtained from the case study demonstrate that the Moodle LMS is only partially, not completely accessible. Moreover, the evaluation shows that an LMS or LCMS does not usually support the task of authoring, reusing or re-purposing contents as well as virtual spaces for student interaction. For instance, Moodle does not offer sufficient support to teachers, despite the fact that it is they who are ultimately responsible for 212

the contents of the LMS and their accessibility. Therefore, it is very important to follow accessibility strategies in Moodle module sites. Lastly, this study has pinpointed specific contexts in which a particular user group cannot access certain features and contents of the LMS.

FUTURE RESEARCH One of the most consistent problems with modern learning management systems is their failure to comply with international standards for accessibility. There is a clear need for fully accessible LMS to be made available, so that educational institutions can meet the needs of all students, including those with disabilities. Future LMS development should be required to meet specified levels of accessibility and future LMS research should define methodological frameworks to be used to achieve this accessible development. Furthermore, additional work must be done to develop accessibility policies that can be implemented by institutions of higher learning. In this way, not only may the needs of individuals with disabilities be completely addressed, but the greater university community may also be educated about these needs in the process.

ACKNOWLEDGMENT This research work was supported by the Research Network MA2VICMR (S2009/TIC-1542) (see www.mavir.net/), GEMMA (TSI-0203022010-141) and SAGAS (TSI-020100-2010-184) research projects.

REFERENCES Abou-Zahra, S. (2008). Web accessibility evaluation. Human-computer interaction series, 2008, Part II, 79-106.

Disability Standards and Guidelines for Learning Management Systems

Buzzi, C., Buzzi, M., & Leporini, B. (2009). Accessing e-learning systems via screen reader: An Example . In Jacko, M. J. A. (Ed.), HumanComputer Interaction, Part IV, HCII 2009, LNCS 5613 (pp. 21–30).

Moreno, L., Martínez, P., & Ruiz-Mezcua, B. (2009). A bridge to Web accessibility from the usability heuristics, 5th Annual Usability Symposium USAB 2009. Usability & HCI eInclusion, LNCS 5889 (pp. 290-300). Berlin, Germany: Springer.

Elias, T. (2010). Universal instructional design principles for Moodle, Canada International Review of Research in Open and Distance Learning, 11(2). May – 2010.

Newell, A. F., & Gregor, P. (2000). User sensitive inclusive design: In search of a new paradigm. CUU 2000 First ACM Conference on Universal Usability, 2000.

Fichten, C. S., Ferraro, V., Asuncion, J. V., Chwojka, C., Barile, M., Nguyen, M. N., & Wolforth, J. (2009). Disabilities and e-learning problems and solutions: An exploratory study. Journal of Educational Technology & Society, 12(4), 241–256.

Power, C., Petrie, H., Sakharov, V., & Swallow, D. (2010). K. virtual learning environments: Another barrier to blended and e-learning. Miesenberger et al. (Eds.), ICCHP 2010, Part I, LNCS 6179 (pp. 519–526).

Henry, S. L. (2007). Just ask: Integrating accessibility throughout design. Madison, WI: ET\ Lawton. Available at www.uiAccess.com/justask/

Rößling, G., Joy, M., Moreno, A., Radenski, A., Malmi, L., Kerren, A., & Iturbide, J. Á. (2008). Enhancing learning management systems to better support computer science education (pp. 142-166). Paper presented at the SIGCSE 2008, 4, Nov. 2008.

Iglesias, A., Moreno, L., Revuelta, P., & Jiménez, J. (2009). APEINTA: A Spanish educational project aiming for inclusive education in and out of the classroom. ITiCSE ‘09, Paris, France (p. 393). New York, NY: ACM, NY. Kalnins-Cole, T., & Peters, D. (2007). Towards an accessible learning management system. International Journal of Continuing Engineering Education and Lifelong Learning, 17(1), 84–96. doi:10.1504/IJCEELL.2007.013232 Martin, L., Gutiérrez-Restrepo, E., & Barrera, C. Rodríguez-Ascaso1, A., Santos, Olga C. & Boticario, J. G. (2007). Usability and accessibility evaluations along the e-learning cycle, IWWUA WISE 2007 Workshops, LNCS Volume 4832/2007, Nancy, Francia (pp. 453—458). Moreno, L., Iglesias, A. M., Castro, E., & Martinez, P. (2008). Using accessible digital resources for teaching database design: towards an inclusive distance learning proposal. ITICSE 2008, Madrid, Spain (pp. 32–36). New York, NY: ACM.

Santos, O. C., Boticario, J. G., Del Viso, A. F., De La Cámara, S. P., Sánchez, C. R., Gutiérrez & Restrepo, E. (2007). Basic skills training to disabled and adult learners through an accessible e-learning platform. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (LNCS 4556, part 3, pp. 796-805). World Wide Web Consortium. (W3C) (2008). Web accessibility initiative (WAI), Web Content Accessibility Guidelines (WCAG) 2.0. Available at http://www.w3.org/TR/WCAG20 / World Wide Web Consortium. (W3C) (2010). Web Accessibility Initiative (WAI), Authoring Tool Accessibility Guidelines (ATAG) 2.0. Available at http://www.w3.org/TR/ATAG20/ World Wide Web Consortium. (2010). WAI-ARIA Overview. Available at http://www.w3.org/WAI/ intro/aria

213

Disability Standards and Guidelines for Learning Management Systems

World Wide Web Consortium. (W3C). (2010 b). Involving users in evaluating Web accessibility. Available at http://www.w3.org/WAI/eval/users. html World Wide Web Consortium. (W3C). (2010 c). WAI, evaluating Web Sites for accessibility: Overview. Available at http://www.w3.org/WAI/eval/

ADDITIONAL READING W3C, WAI, Authoring Tool Accessibility Guidelines (ATAG) Overview (2010). Available at http:// www.w3.org/WAI/intro/atag.php W3C, WAI, Introduction to “How People with Disabilities Use the Web” (2010). Available at http:// www.w3.org/WAI/intro/people-use-web.php W3C, WAI, Web Content Accessibility Guidelines (WCAG) Overview. (2010). Available at http:// www.w3.org/WAI/intro/wcag.php W3C, Web Accessibility Initiative (WAI). (2010). Available at http://www.w3.org/WAI/ Abascal, J., Arrue, M., Fajardo, I., & Garay, N. (2004). The use of guidelines to automatically verify Web accessibility. Universal Access in the Information Society, 3(1), 71–79. doi:10.1007/ s10209-003-0069-3 Alan, F. Newell & Peter Gregor (2000). User sensitive inclusive design in search of a new paradigm. In Proceedings on the 2000 conference on Universal Usability (CUU ‘00) (pp. 39-44). New York, NY: ACM. Cann, C., Ball, S., & Sutherland, A. (2003). Towards accessible virtual learning environments., TechDis Web site. Available at http://www.jisctechdis.ac.uk/techdis/home Cook, A. M., & Polgar, J. M. (2007). Cook and Hussey’s Assistive Technologies: Principles and Practice. Mosby.

214

Dunn, S. (2003). Return to SENDA? Implementing Accessibility For Disabled Students In Virtual Learning Environments In UK Further And Higher Education. Retrieved September 1, 2004, from personal Web site: http://www.saradunn. net/VLEreport/ Henry, S. L. (2007). Just Ask: Integrating Accessibility Throughout Design by, Lulu.com. February 2007, ISBN-10: 1430319526. Available at http:// uiaccess.com/accessucd/about.html Iglesias, A., Moreno, L., Jiménez, J., & Revuelta, P. (2010). Evaluating the Users’Satisfaction Using Inclusive Initiatives in Two Different Environments: The University and a Research Conference. Computers Helping People with Special Needs, 12th International Conference, ICCHP 2010, Vienna, Austria, July 14-16, 2010. LNCS, 2010, Volume 6179/2010, 591-594 Isaila, N. & Nicolau, I. (2010). Promoting computer assisted learning for persons with disabilities. Procedia - Social and Behavioral Sciences. 2(2),, 4497-4501. Kelly, B., Phipps, L., & Howell, C. (2005). Implementing a holistic approach to e-learning accessibility. In: ALT-C 2005, 6-8 September 2005, Manchester, UK. Mankoff, J., Fait, H., & Tran, T. (2005). Is your Web page accessible? A comparative study of methods for assessing Web page accessibility for the blind. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ‘05). ACM, New York, NY, USA, 41-50. NCAM. (2006). CPB/WGBH National Center for Accessible, Media (NCAM), Accessible Digital Media Guidelines. Available at http://ncam.wgbh. org/invent_build/web_multimedia/accessibledigital-media-guide

Disability Standards and Guidelines for Learning Management Systems

Pearson, E., & Koppi, T. (2002). Inclusion And online learning opportunities: Designing For accessibility. Association for Learning Technology Journal, 10(2), 17–28. doi:10.1080/0968776020100203 Preiser, W. F. E., & Ostroff, E. (Eds.). (2001). Universal design handbook. New York, NY: McGraw-Hill. Rice, W. (2006). Moodle E-Learning Course Development. Packt Publishing. Rose, D., & Meyer, A. (2000). Universal design for learning. Journal of Special Education Technology, 15(1), 67–70. Seale, J. (2006). E-learning and disability in higher education: accessibility research and practice. UK: Taylor & Francis. Vanderheiden, G. C. (2009). Accessible and usable design of Information and Communication Technologies. In C. Stephanidis (Ed.), The Universal Access Handbook. UK: Taylor & Francis Group. ISBN: 0-8058-6280-3. Pp: 3-1, 3-26.

KEY TERMS AND DEFINITIONS Assistive Technology: Software (or hardware), separate from the authoring tool, that provides functionality to meet the requirements of users with disabilities. Some authoring tools may also provide direct accessibility features Authoring Tool: Any software, or collection of software components, that authors can use to create or modify web content for use by other people. Template: A content pattern that is filled in by authors or the authoring tool to produce content for end users (e.g., document templates, content management templates, presentation themes). Often templates will pre-specify at least some authoring decisions. User-Centered Design Process (UCD): “Human-Centred design is an approach to interactive

system development that focuses specifically on making systems usable. It is a multi-disciplinary activity.” In UCD, all “development proceeds with the user as the center of focus.” Web Accessibility: Web accessibility means that people with disabilities can use the Web. More specifically, Web accessibility means that people with disabilities can perceive, understand, navigate, and interact with the Web, and that they can contribute to the Web. Web accessibility also benefits others, including older people with changing abilities due to aging. WYSIWYG: This is an acronym for “What You See Is What You Get”. A WYSIWYG view displays (to authors) the content being edited in a way that is very similar to how it will appear to end users.

ENDNOTES 1



2



3



4



5



6



7



8



9



Moodle, a Learning Management System (LMS). Available at http://moodle.org/ IMS Global Learning Consortium. Available at http://www.imsglobal.org/ W3C: The World Wide Web Consortium. Available at http://www.w3.org/ WAI: Web Accessibility Initiative from W3C. Available at http://www.w3.org/WAI/ Dublin Core Metadata Initiative. Available at http://dublincore.org/ IMS AccessForAll Meta-data. Available at http://www.imsglobal.org/accessibility/ accmdv1p0/imsaccmd_bestv1p0.html ISO (International Organization for Standardization). Available at http://www.iso. org/iso/home.html Freedom Scientific - JAWS Screen Reading Software. Available at http://www.freedomscientific.com/products/fs/jaws-productpage.asp Window-Eyes. Available at http://www. gwmicro.com/Window-Eyes/

215

Disability Standards and Guidelines for Learning Management Systems

10



11



12



13



14



15



16



17



216

Linux Screen Reader. Available at http:// live.gnome.org/LSR Orca. Available at http://live.gnome.org/ Orca NonVisual Desktop Access (NVDA) . Available at http://www.nvda-project.org/ Fire Vox: A Screen Reading Extension for Firefox. Available at http://www.firevox. clcworld.net/ HearSay project. Available at http://www. cs.sunysb.edu/~hearsay/ ACTF, Accessibility Internet Browser for Multimedia (aiBrowser). Available at http:// www.eclipse.org/actf/downloads/tools/ aiBrowser/index.php dotLRN, An open source E-learning. Available at http://www.dotlrn.org/ ATutor, Learning Content Management System. Available at http://www.atutor.ca/

18



19



20



21



22



23



24



25



Dokeos, an Open Source E-Learning. Available at http://www.dokeos.com/es Docebo, an Open Source E-Learning platform. Available at http://www.docebo.org/ doceboCms/ Sakai Project, an Open Source suite of learning. Available at http://sakaiproject.org/ Blackboard. Available at http://www.blackboard.com/ TAW, a tool for the analysis of Web sites. Available at http://www.tawdis.net/ingles. html?lang=en AChecker, an open source Web accessibility evaluation tool. Available at http://www. atutor.ca/achecker/ AIS, Web Accessibility Toolbar. Available at http://www.visionaustralia.org.au/ais/ toolbar/ Firefox Accessibility Extension. Available at http://firefox.cita.uiuc.edu/

Disability Standards and Guidelines for Learning Management Systems

APPENDIX 1: TABLE 2 WITH SUMMARY OF EXPERT AND USER EVALUATIONS This table presents the type(s) (i.e., profile) of user with the ability to execute the task (e.g., administrator [A], teacher [T], student [S]), the functionality group, the name of the task evaluated. The fourth column offers brief descriptions of accessibility problems where they arose. And finally, the last column shows if the individual tasks could be executed using JAWS and, in affirmative cases, whether such executions involved accessibility problems (Yes*) or no accessibility problems (Yes). In cases where accessibility problems using JAWS prevented the successful execution of the task, a different symbol (No) is used. Table 2. Summary of expert and user evaluations User Profile

Functionality (group)

Task Name

Errors

Can it be completed?

A/T/ S

General

A/T/S

General

Login user

E1

Yes*

Change language Moodle

E1/E2

Yes*

A

Users/ Authentication

A

Users/ Authentication

Manage authentication

E8 /E10

Yes*

Email-based self-registration

E5

Yes*

A

Users/ Authentication

A

Users/ Authentication

No login

E7

Yes*

Manual accounts

E5

A

Users/Accounts

Browse list of users

Yes*

E7/E5/E9/E10

Yes*

A

Users/Accounts

A

Users/Accounts

Bulk user actions

E2/E11

Yes*

Add a new user

E6/E8/E11/E13

Yes*

A A

Users/Accounts

Upload users

--

Yes

Users/Accounts

Upload user pictures

--

Yes

A

Users/Accounts

User profile fields

E3/E7/E13

Yes*

A

Users/Permissions

Define roles

E7/E8/E13

Yes*

A/T

Users/Permissions

Assign system roles

E1/E9

Yes*

A

Users/Permissions

User policies

E8/E9

Yes*

A/T*

Courses

Add /Edit courses

E4/E13

Yes*

A

Courses

Enrollments

E9/E11

Yes*

A/T/S

Courses

Participants

--

Yes

A/T

Courses

Backup

--

Yes

A/T

Courses

Restore a course

E5/E7/E9/ E10/E11

No

A/T

Courses

Import

E4/E5

Yes*

A/T

Courses

Reset course

E4

Yes*

A

Grades

My preferences grader report

E1/E3/E4/E6/ E7/E11

Yes*

A/T/S

Grades/View

Overview report

E1/E4

No

A/T

Grades/View

Grader report

E1/E4

Yes*

A/T/S

Grades/View

User report

E1/E4/E10

Yes*

A/T

Grades/Categories and Items

Simple view

E1/E4/E10

Yes*

A/T

Grades/Categories and Items

Full view

E1/E4/E8/E10/ E12

Yes*

A/T

Grades/Scales

View

E1/E4/E10/E13

Yes*

A/T

Grades/Letters

View

E1/E4/E16

Yes*

A/T

Grades/Letters

Edit

E1/E4

Yes*

continued on following page

217

Disability Standards and Guidelines for Learning Management Systems

Table 2. Continued

218

User Profile

Functionality (group)

Task Name

Errors

Can it be completed?

A/T A/T

Grades/Import

CSV file

E1/E4/E9

Yes*

Grades/Import

XML file

E1/E4

Yes*

A/T

Grades/Export To

Open doc spreadsheet / Plain text file/Excel spdsht/ XML file

E1/E4/E9

Yes*

A/T

Reports

Filter logs

E1/E4

Yes*

A/T

Reports

Activity report

E4/ E14

Yes*

A/T

Reports

Participation report

E4/ E8/E11

Yes*

A/T

Questions

Questions bank

E3/E8/E13

Yes*

A/T

Reports

Live logs from the past hour

E2

No

A/T

Questions

Import

E4/E7

Yes*

A/T

Questions

Export

E4/E9

Yes*

A/T

Files

List of files

E1/E4/E7/E10/ E11

Yes*

A/T

Files

Upload a file

E3/E4/E8/E11

Yes*

A/T

Files

Make a folder

E1/E11/E15

Yes*

A/T

Groups

Create group

E4/E6/E11/E13

Yes*

A/T

Groups

Delete group

E4/E11

Yes*

A/T

Groups

Add/Remove users

E1/E4

Yes*

A/T/S

New event

New event

E11/E13

Yes*

A/T/S

Export calendar

Export calendar

E11

Yes*

A/T

Forums

Add / Edit a new topic

E1/E4/E11/E13

Yes*

A/T

Forums

Delete topic

E4

Yes*

A/T

Forums

Reply

E1/E3/E4/E11

Yes*

A/T/S

Profile

Change password

E4/E8/E11

Yes*

A/T/S

Profile

Edit profile

E4/E8/E11/E13

Yes*

219

Chapter 11

The Technological Advancement of LMS Systems and E-Content Software Dorota Dżega West Pomeranian Business School, Poland Wiesław Pietruszkiewicz SDART Ltd, UK

ABSTRACT This chapter will present the practical aspects of Learning Management Systems adoption by describing this process from the perspective of evolution, observed for LMS and e-content software at West Pomeranian Business School. The chapter will address issues and found solutions relating to LMS deployment and evolution, noticed during the management of e-learning studies. In its first part, chapter will explain the requirements for different types of studies and how they influenced the shape of LMS systems. In the following sections, the chapter will analyze different technologies and software used in the e-learning process. This analysis will also describe how efficiently use the functionality of e-learning software in relation to the users’ requirements. The last part of chapter will present SPE - SDART Presentation Engine, being an innovative e-learning presentation engine, developed in form of Rich Internet Application, to overcome the limitations observed for the previously used presentation engines.

INTRODUCTION The use of e-learning in higher education in Poland is regulated by a set of legal acts. The most important one among them is the order concerning conditions which should be fulfilled

to make conduction of the classes that use the distance-learning methods and techniques possible. Every school that offers this kind of studies must have at its disposal a well-trained teaching staff prepared to conduct classes by the means of blended-learning. Moreover, school must assure:

DOI: 10.4018/978-1-60960-884-2.ch011

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

The Technological Advancement of LMS Systems and E-Content Software

•

• • •

• •

An access to IT systems and the software which enables the synchronous and asynchronous interaction between students and e-tutors. Didactic materials prepared in an electronic version. The possibility of personal consultations with e-tutor at school. The regular verification of students’ progress, knowledge and skills, including conducting examinations at the end of each course at school. The regular verification of tutors’ activity. The trainings for students on the e-learning platform.

The above-mentioned list neither impose on schools what kind of software they have to use, nor it limits the form of IT systems usage (for example: the purchase of the license, the lease, etc.). The number of classes conducted with the usage of distance-learning methods and techniques is limited to 60% of the entire amount of classes planned for each specialization and level of education, which is defined by core curriculum, excluding the practical and laboratory classes. As far as the postgraduate studies and trainings are concerned, there are not any special legal regulations – it is possible to use e-learning methods without limits. The main idea of this chapter is to present the process of evolution for LMS using an example of West Pomeranian Business School. The chapter will explain the requirements for different types of studies and how they influenced the Learning Management Systems. The next part of chapter will include information about different technologies and software that were and are currently used in the e-learning process. The last part of chapter will introduce SPE (SDART Presentation Engine) being an innovative e-learning presentation engine developed as an Rich Internet Application by SDART Ltd and this software was designed to overcome the problems observed for the previously

220

used presentation engines. In each section we will discuss relevant issues and results of observations that shaped the process of LMS evolution. Hence, the chapter delivers information meaningful for the practice of e-learning management.

EXPERIENCE The West Pomeranian Business School is one of the first universities in Poland, which uses methods and techniques of distance learning. Chronologically, e-learning experience in West Pomeranian Business School (ZPSB as a University official abbreviation) were as follows: • • • • •

In 2006 started first blended-learning study, the first LMS was Oracle iLearning™. In 2007 LMS was changed from Oracle iLearning to Moodle. In 2008 started own production of e-learning content and e-books. In 2009 started e-learning postgraduate studies and e-learning business courses. In 2010 started e-tutoring in science courses (for example: mathematics, statistics, econometrics).

The future plans include introduction of virtual practice. Currently, ZPSB is offering three main products using the technology of e-learning. These products are presented in Figure 1. ZPSB e-students may achieve Bachelor degree and Master degree. During their studies they receive help in form of e-Tutoring associated with their courses. After a Bachelor degree or Master degree they can continue study at e-Postgraduate studies. The graduate and postgraduate students may take part in e-courses (business-oriented free courses). Additionally, the traditionally learning students may also take a part in the e-courses as well and they receive an access to e-content, and they may expect help via e-Tutoring. After graduation students may continue learning at

The Technological Advancement of LMS Systems and E-Content Software

Figure 1. e-Learning products offered in West Pomeranian Business School

Master degree or Postgraduate studies. At the moment in ZPSB resource there are: • • •

e-Content (e-learning content for over 100 courses) and e-books (over 30 titles). e-Postgraduate studies (e-learning content for 18 courses). e-Business courses (e-learning content for 8 business courses).

These resources will be analyzed in the following sections. The preparation and implementation of each product is a complex operation and requires the involvement of specialized personnel and technology. Typically, this action is a complex of four processes: the process of preparing e-learning platform, the process of preparing teaching materials, educational process and quality assurance processes. The process of e-learning platform preparation consists in selecting the appropriate LMS system and configure it according to the needs

of the product. The process of preparing teaching materials in the form of e-contents generates the highest costs, and is characterized by the highest workload. The costs of this process are mainly associated with the purchase of the copyright of teaching materials, salary of the production team and the costs of adjustment. It is worth noting that the quality of teaching materials play a very important role in the implementation of the educational process. Educational process is the preparation and implementation of the course, during which there is transfer of knowledge, learning and check of knowledge. At this stage, available functionality of LMS is very important. The last process is the process associated with providing an adequate level of quality of individual products. Not only technology and publishing teaching materials are examined, but also the quality of teachers’ work.

221

The Technological Advancement of LMS Systems and E-Content Software

Educational Process The selection of LMS require a prior analysis of the e-learning educational process, to choose a proper e-learning products and matching technologies. The common elements are: the registration procedure for studies and courses, the documentation of studies (courses) and students’ (participants’) results. Therefore in the overall educational process a set of three phases may be distinguished: the preliminary phase, the intermediate and the final one. The Figure 2 presents a detailed analysis of these phases. The differentiation of the educational process appears in its preliminary phase. At this stage blended-learning and full e-learning differs. The blended-learning path is realized for the diploma studies (where students get a Bachelor or Master degree) and it is closely dependent on law in force

in the area of Polish higher education. The full e-learning path is realized in the case of postgraduate studies and business courses. The final part of the preliminary phase is the end-users training of LMS. The e-learning platform trainings are the fixed element of at the beginning of studies. They take place in the computer room and last at least for two hours. Alongside with trainings done for students, at the beginning of each course special trainings of e-tutors are held – they learn how to conduct e-learning classes and how to make use of all tools offered by the e-learning platform. In the case of the full e-learning path the training of end-users is through the e-materials. The main part of the educational process is held in the intermediate phase. In the blendedlearning path there is a division of courses into two parts: the one realized in the traditional way

Figure 2. Educational process of blended-learning and e-learning paths

222

The Technological Advancement of LMS Systems and E-Content Software

and the other realized by means of the LMS. This article contains the detailed description of the use of the e-learning platform in the educational system. According to law established in Poland all exams have to take place via the traditional way at school. Such a requirement doesn’t exist in the case of the postgraduate studies and short courses. Thus, in the e-learning path the intermediate phase includes learning, finishing the course through the on-line test or exam and the preparation of the final thesis. When it comes to the business courses, the intermediate phase only includes learning and consultations with e-tutor. The final phase is also diverse in regards to the extent of the use of e-learning technology in the educational process. In the case of blended-learning path the personal contacts between student and his/her thesis’ tutor, as well as diploma exam are emphasized. In the case of postgraduate studies, on the other hand, the only activity that takes place in the final phase is the thesis evaluation. The short courses are even more organizationally simplified and the only thing that takes place in the final phase is final on-line exam through the LMS with the usage of built-in test module. Taking this description into consideration, it is obvious that the blended-learning path is more complicated than the e-learning path as far as the organization is concerned. On the contrary, the full e-learning path requires more commitment into precise and faultless preparation of the resources that end-users have at their disposal. The same observation applies to the e-tutors’ engagement into the educational process – the more involved e-tutors are, the more effective blended-learning path is. Along with this commitment comes the necessity of computer knowledge, suitable skills for conducting on-line classes, such as the skill of motivating students to work by means of the computer and e-learning platform, the ability of talking with students on chats without losing their attention, the regularity in checking of the students’ progresses. The realization the above-mentioned educational process requires diversity not only

in the field of organization, but also in the area of used technology and the technologies used by ZPSB will presented in the following section.

E-Learning Technologies In this section we will analyze the LMS changes, their reasons and the functional requirements for LMS relating to different type of e-learning studies. The first LMS system applied by ZPSB for the online learning was Oracle iLearning platform. The learning materials have been created with use of various technologies such as Adobe Flash™, Adobe FlashPaper™ and PDF creator. Currently all the e-learning products are based on Moodle LMS and it was configured for the presentation of e-learning materials using Adobe Learning Suite™ packet and SDART Presentation Engine. It must be explained why a major technological change was done and Oracle platform was replaced by Moodle LMS. Oracle iLearning was a closed solution with limited capabilities (comparing to modules available for Moodle LMS) and due proprietary character it wasn’t possible to extend it by developing additional features adjusting the platform to meet the University requirements. Hence, Moodle being Open Source software was selected. Figure 3 presents the overall process of evolution of LMS systems as well as technologies relating to the production of e-content that took place in e-learning services offered by ZPSB. The solution (1) was an archetype for the future products development. Solutions (2) and (3) are combined into a hybrid of the new LMS system and small modifications of the technology producing e-content. The solution (4) is the most mature of the presented solutions. The LMS system and technology producing e-content fulfils all the needs and requirements established by West Pomeranian Business School. The change of the LMS system was required by the need to enhance the functionality of e-learning platform as well as expansion of the flexibility when modules are

223

The Technological Advancement of LMS Systems and E-Content Software

Figure 3. The development of LMS and e-content

prepared by the developers. The Moodle platform is one of the most popular LMS platforms used by the Polish universities. One of the major benefits of a web platform is that it does not require any installation steps from its users (students, teachers, business participants). An access to the platform is fully distributed and delivered via Internet browsers. Benefiting the modular architecture of the Moodle platform, it is possible to adjust its functionality according to the needs of the bespoke product. The Table 1 contains information about the used functionality of the Moodle platform accordingly to the different types of courses. The presence of activities i.e. obligatory,

224

optional and unavailable, depends on the organization of studies imposed by the law and internal University regulations. In case when the e-studies are delivered in blend-learning form, each registered LMS user has an access to the central service as well as to the courses one should attended. The central service contains activities such as: announcements from and online chat with the e-Learning Centre and news section. Each course published on the e-learning platform is divided into different activities (on-line, off-line) relating to the learning and communication.

The Technological Advancement of LMS Systems and E-Content Software

Table 1. Basic functionality of learning management system on different e-learning products Activity of e-learning Learning

Knowledge check

Communications

Functionality

e-Studies

•

•

e-Business courses

„Traditional” studies

e-learning content

obligatory

obligatory

obligatory

obligatory

e-books

obligatory

obligatory

obligatory

obligatory

Additional teaching materials

optional

optional

optional

optional

Additional course pages

optional

unavailable

unavailable

optional

Glossary

obligatory

unavailable

unavailable

unavailable

Quiz

optional

obligatory

obligatory

optional

Assignments

optional

unavailable

unavailable

optional

WIKI

optional

unavailable

unavailable

optional

Games (crosswords, Sudoku)

optional

unavailable

unavailable

optional

Discussion via forum

obligatory

optional

optional

unavailable

Chat

obligatory

unavailable

unavailable

unavailable

Forum

optional

obligatory

obligatory

optional

eMail (internal email)

obligatory

obligatory

optional

optional

Tutor’s announcement

obligatory

obligatory

obligatory

optional

Internal messengers

optional

optional

optional

optional

Within the activities relating to the learning it is possible: To deliver lectures with use of: •

e-Postgraduate studies

the studies using multimedia teaching materials provide for each course in SCORM form. Additionally users have a fast access to the teaching materials in a printable form (e-books); design of the additional pages for the course, where it is possible to publish additional teaching materials relating to the subject; by building of the dictionary containing keywords for the course. This allows to define expressions relating to the course subject, which may be done by the students or teachers (teachers additionally have a right to accept and approve the keywords added by the students).

Motivation for students can be improved by: •

•

• •

delivering a moderated discussion forums relating to the subject, where students as well as teachers can follow up a conversation and exchange opinions; delivering the workshops based on the exchange of files, where students can assess and mark each other’s work; enabling content entries to the dictionary; gaming solutions, these are very attractive form testing the basic knowledge or the combined activities such as dictionaries and tests. The examples of games are crosswords and Sudoku. To test the learning progress using:

•

the evaluation of posts made by the students on discussion forums. Only specifically re-

225

The Technological Advancement of LMS Systems and E-Content Software

•

•

•

lating announcements from the period of time may be selected for the evaluation; micro testing relating to a particular subject or part of the material, typically tests cover one module. It is possible to build a database of questions (with division into different categories) for each course and to implement following questions within the e-learning platform: single questions or multiple choice questions, true/false questions, questions with selections of the answers (to choose a relation between a couple), calculations, essay questions. All the types of questions mentioned above are accompanied with explanations, the variants of answers or their number of points and rules to calculate the students’ marks allowing to formula and the conclusion for students; it is possible to create a home work and mark teamwork using a Wiki mechanism, where the articles are build by multiple cooperating people; it is possible to give online space for the student, place where generated work can be placed or exchanged.

Finally, an electronic logbook for each course may be created. Each task, apart of the number of points, can be commented. This is very important for the students to fully understand the course and received marks. As a part of the communication between the lecturer and students it is possible to use following features: •

226

Real-time conversation delivered by the online chats. This form of communication is rather restrictive as it’s time-consuming and a minimal length is imposed, according to the number of hours available for the course. The information about the time for chats sessions is announced in three plac-

•

•

•

es: the central service webpages, the main page of course as well as in the calendars of students and lecturers. Publishing of the lecturers’ announcements. This is a place where public announcements about main issues can be distributed, it is recommended to use it mainly for the general announcements e.g. About the sessions and conditions relating to passing a particular exam. Real-time conversations (or leaving a message) using a communicator implemented on the platform. This is a direct one-to-one communication. E-mail exchange, within each of the courses it is possible to create internal e-mail groups. This is a part of the e-learning platform.

The lecturers as well as the students have capability to publish additional materials. The e-learning platform supports text files, spreadsheets, images, audio and video. A user can choose preferred language of interface: Polish, English or German. For the postgraduate studies delivered in the e-learning form each registered user has an access to the course published on LMS system. The postgraduate students learn the course materials individually without any supervision. The tutor role is restricted only to consultations and answering students’ questions. Because the tests at the end of course are delivered in the e-learning form it was required to develop a database of the questions aimed at logical thinking and ability to link the facts and observations. The main reason for that was to restrict the effects of repetitive knowledge absorption. That demands the additional efforts during the tests creation and extended instructions in the online help. The answers and evaluation of the online tests is done fully with use of the LMS System. Information about the achieved marks is passed

The Technological Advancement of LMS Systems and E-Content Software

directly to the students and employees of customer department. In a very similar manner, the examination results are delivered after the business e-learning course.

THE EXTENSION OF LMS The functionality mentioned in the previous section was not sufficient and herein we will describe additional LMS extensions developed to provide the features requested by users. Among the additional LMS modules, which were reported by users as desired ones were: Equation Editor, Audio/ Video Chat and Whiteboard. These modules were not available for Moodle LMS or their capabilities did not match the requirements. Therefore, these three modules were developed by West Pomeranian Business School. The WYSIWG (What You See Is What You Get) editors available for Moodle did not offer equation relating features. The developed Equation Editor allows one to construct any mathematical expression and then insert it into the course content, at any place where it is possible to edit the text using an HTML WYSIWG editor. The developed module contains an editor, where a following set of symbols and operators are used: •

•

• • • • • •

Basic binary operator such as multiplication, addition, subtraction, equality, membership sets, factorial, etc. Special visual fields: such as neighboring expressions, fractions, roots and exponentials, superscripts, subscripts, equations, matrices, etc. Brackets and parentheses: square, triangular, etc. Math functions: trigonometric, logarithmic, statistical functions, etc. Integrals, boundaries, sums, differentials, etc. Greek letters. Arrows (as binary operators). Other symbols and accents.

Inserting math symbols in WYSIWG HTML editor is very intuitive and involves a mechanism of drag&drop for items or it could be done by inserting the current items at current cursor position. In addition to the graphical equation editor it is also possible to insert any math equation in the LaTeX form. The developed module for audio/video chat contains three forms of video communication (or audio-only when user does not have a webcam): •

•

•

Individual interview – an interview can take place only between two users (P2P technology is used). Group discussion – everyone can speak at any time, hence it is a solution suitable rather for a group with a small number of members. Moderated group discussion – the moderator has special privilege comparing to the rest of the members and primarily controls entire conversation and gives a chance to speak to persons who are waiting in a queue. The main difference between the group conversation and moderated group conversation is that in a moderated conversation only 2 users may speak simultaneously and one of them is the moderator.

Interview audio/video is always initiated in a similar way as a text conversation. The Audio/ Video Chat Module has been developed using Laszlo Systems OpenLaszlo™ technology. The whiteboard module is a solution designed to support the cooperation of several users logged on LMS and using one course. As part of this activity it is possible to draw geometric objects and lines with a various thickness and different colors, to write with a pencil or to edit the text by a special editor. The effects of group work can be saved as an image file and send in e-mails to the cooperating users. The basic configuration of Moodle does not offer the functionality of global management of user groups from the main site. It’s possible 227

The Technological Advancement of LMS Systems and E-Content Software

only to manage the groups from the level of each course. This situation causes a major obstacle for system administrators requiring a time-consuming activity to prepare each of course. Hence, the LMS functionality was experimentally extended by introducing an additional feature to Moodle – superior group. The global group management module allows administrator to: • • • •

Define the superior groups. Define the sub-groups. Assign users to the sub-groups. Assign the sub-groups to the courses.

The further development of LMS functionality depends on the verified needs requested by the end-users and system administrators.

Multimedia Teaching Materials The most important part of the e-learning course is the e-content as well as e-books embedded in the LMS system and these elements will be discussed in this section by presenting their structure and process of production, both being the results of observations, analyzes and adjustments that toke place since 2006. That’s why the e-learning materials creation process should be considered as a priority by the teaching organization in the activities relating to the online studies. The team responsible for the production of the e-learning materials must be continuously looking for solutions allowing saving time and reduced the cost of producing materials. The multimedia teaching materials are produced in compliance with rules relating to generating e-content (Horton, 2006) with use of the programming packages such as: Adobe Flash™, Adobe Learning Suite™, Macromedia Flash Paper™, PDF Creator, SDART Presentation Engine. Currently all the multimedia teaching materials were prepared and standardized in the form of:

228

• •

•

•

•

•

• • •

Course program – „List of content” have all the modules including sessions. Course introduction – definition of the aims of the course, introduction of the basic information about the subject, description of the course entry requirements, including general knowledge of the student and skills from the other courses. Modules include: an introduction to module (general explanation about the subject of the module, explanation what student will learn during the course), the main part divided into sessions (containing a theoretical and practical part), a subject summary (conclusions about the main aims of the module). Dictionary – expressions and definitions used within the teaching materials and materials relating to the subject. Recommended literature – a list of most important literature positions as well as additional reading suggestions relating to the course subject. Set of questions and tasks as a part of self testing activity, including questionnaires (single answers multiple answers), true/ false questions, tasks with linked coupled answers, schemas with controllable order of the answers. Open questions to control understanding. Additional materials (presentations, legal acts, etc). Preliminary test from the knowledge learned during the course.

Additionally, the multimedia teaching materials contain images, animations and links to the web pages. The quantity of materials is related to the amount of our hours defined for the course. On average typical course contains 5 modules, each module contains multiple sessions. Every course has on average 140 slides and approx. 20 questions or tasks for self testing. Multimedia teaching materials are not the only source of

The Technological Advancement of LMS Systems and E-Content Software

information presenting the subject. In Table 2 a detailed content of materials is presented and this list is provided for the authors. The authors of the teaching materials are typically lecturers running the classes. Prior to the preparation of e-content all authors ought to be instructed. The instruction contains explanation of the rules and directions for the creation of elearning materials, screens and their contents. The

team responsible for the multimedia materials includes: multimedia teacher, proof-reader, multimedia experts, computer graphic designer. All the multimedia materials, before they are released to the students access, are thoroughly verified: the original materials are analyzed by the multimedia expert, after corrections they are passed to a proof-reader then are tested during the produc-

Table 2. Checklist for e-content authors The organizational structure of course

Amount of screens

Course program

1 screen (obligatory)

Introdution to course

1 screen (obligatory)           Introduction to module 1

Number of screens

1 screen (obligatory)

          Session 1.1           Session 1.2 Module 1

25-30 screens for module

          ………..           Self-check short quiz for module 1

4-5 screens

          Cases and examples for module 1

min. 10 screens

          Summary of module 1

1 screen (obligatory)

          Introduction to module 2

1 screen (obligatory)

          Session 2.1           Session 2.2 Module 2

25-30 screens for module

          ………..           Self-check short quiz for module 2

4-5 screens

          Cases and examples for module 2

min. 10 screens

          Summary of module 2

1 screen (obligatory)

          Introduction to module …

1 screen (obligatory)

          Session 3.1           Session 3.2 Module …

25-30 screens for module

          ………..           Self-check short quiz for module …

4-5 screens

          Cases and examples for module …

min. 10 screens

          Summary of module …

1 screen (obligatory)

Summary of course

1 screen (obligatory)

Literature

1 screen (obligatory)

Glossary

min. 10 keywords

Trial quiz

10 screens

Additional materials

filenames

Database of on-line exam quiz

30 screens

229

The Technological Advancement of LMS Systems and E-Content Software

tion are finally tested by the authors. The detailed tests are as follow: Introduction phase: • •

selection of the author; instruction about the rules and directions for the creation of the e-learning materials, screens and its contents; work on the basic version of the scenarios for the course.

•

Secondary phase: •

preparation of the materials by the author: ◦◦ preparation of the first module of the course, ◦◦ consultation about the first version of module with multimedia teacher, ◦◦ corrections for the first version of module based on the suggestions from multimedia teacher, leading to development of the rest of the modules, ◦◦ approval of the materials by the multimedia teacher, ◦◦ language corrections, ◦◦ approval of the materials after language corrections, ◦◦ approval of the materials to production. production of the materials: ◦◦ preparation of the course contents, ◦◦ preparation of the tests, ◦◦ preparation of the graphics and animations, ◦◦ preparation of the e-books, ◦◦ elements are joined together, ◦◦ preparation of the packet SCORM, ◦◦ beta tests.

•

The final phase: • • •

230

tests done by the author; corrections of the materials; deployment of the materials into LMS.

The LMS system is constantly used during the whole process of e-contents creation. During the introduction phase, the main features of LMS that are used are ones for communication and publication of the materials. In the secondary phase, the main focus is on the part of system enabling exchange of the files and communication between people involved. The file exchange improves the organization and access to the materials as well as distribution of the corrected versions in a real-time. The final tests are also run using LMS and at the end created content is delivered to the users via LMS. One of the most important quality features for the teaching materials is that they shall be upto-date. Delivering the latest/up-to-date content requires an additional effort, not only from the author of materials, but also from all the team involved in their production. Hence, it is essential to ensure the efficiency of technology used. The efficiency influences both costs and time related to the development of course. The materials generated with use of the Flash technology and manual preparation of the documents for the FlashPaper as well as e-books is extremely time-consuming. For the post graduate and short courses it was required to prepare materials only in the e-learning form with a minimum input from the tutor into the teaching process. It was essential to find a more efficient technology. The selected technology was SDART Presentation Engine. Its functionality will be presented in one of the following sections of this chapter.

QUALITY ASSURANCE FOR TRAINING PROCESS The essential activity of each business process is the quality assurance. This section will present this aspect of e-learning management by explaining the guideline used to provide the high quality of educational services. The evaluation of the students’ activity is regularly made by e-tutors

The Technological Advancement of LMS Systems and E-Content Software

or e-Learning Centre staff. The reports generated by the LMS seem to be enough for this purpose. The Moodle platform delivers detailed information about course logs (e.g. visited places, time of entering each element of the course). It also allows filtering of the logs according to the activity within the course (for example: the number of posts added on the forum, the number of browsing e-content, the number of browsing the additional materials, the number of quiz attempts, the number of sent files, etc.). In the case of students such information is sufficient. However, the observation of e-tutors’ behavior is a more difficult task. This difficulty lies in limited possibilities of the e-tutors’ work evaluation only on the basis of the reports generated by the LMS. A detailed survey of the e-learning quality was conducted by (Swedish National Agency for Higher Education, 2008). To assure the training quality the three staged opinion survey was introduced. Students fill in the forms either in the electronic version published in the LMS or in the traditional way (on paper) - of students’ choice.

Stage 1 The survey carried out among first-year students - this one is usually a sort of introduction to elearning platform training. We can find following questions there: • • • •

•

How did you know about our studies? What factors proved to be crucial for you while enrolling in e-learning studies? What are your representations and expectations concerning these studies? What are, in your opinion, the advantages of e-learning studies in the comparison with traditional studies? Do you think studies using the Internet like this are the future?

Stage 2 The evaluation of the courses and e-tutors – every element gets marks from 1 to 5. The frequency of the opinion survey on courses and e-tutors: 4 times during an academic year. The structure of form is as follows:

Part 1: The Course’s Essential and Formal Evaluation •

• • • •

To what extent did classes meet your representations and expectations towards the course? Do you think the presented material is clear and comprehensible? Do you think the presented material is upto date? Do you find yourself active on the e-learning platform during the course? How do you assess your cooperation with other students during the course?

Part 2: The E-Tutor’s Evaluation •

•

•

•

•

How do you assess the e-tutor’s commitment while conducting the course on the e-learning platform? How do you assess the regularity of contacts between the e-tutor and the students – the e-tutor’s availability during the consultation hours? How do you assess the regularity of contacts between the e-tutor and the students (out of the consultation hours) – for example the waiting time for the answer on the forum? How you assess the e-tutor’s activities oriented to make students work systematically (does s/he lead the discussions on the forum and chats, does s/he use the groupwork tools)? To what extent does the tutor make use of the tools available on the e-learning plat-

231

The Technological Advancement of LMS Systems and E-Content Software

form, excluding forums and chats (for example dictionaries, quizzes and on-line exercises, added files, the presentation of websites)?

Part 3: Additional Remarks What Information Would You Like To Pass To The E-Tutor, That Is To Say What Is Praiseworthy And What Should Be Necessarily Changed?

Stage 3 The survey on the functionality of the e-learning platform and the activity of the e-Learning Centre. The following elements are the subject of the survey: •

•

•

The main advantages and disadvantages of the ZPSB e-learning platform, e.g. the intuitiveness, the selection of the activities, the failure rate. Please point out the elements that are praiseworthy and those that should be improved. Please point out the elements that you find difficult to understand or master. Please point out the places/activities/ elements that could be more intuitive in your opinion. The main advantages and disadvantages of the interface, e.g. the color combination of the user interface, the legibility of the graphic elements, the way of text parts’ presentation and functionality of multimedia didactic material, e.g. is the navigation in the material intuitive enough?. The main advantages and disadvantages of the dealing with students’ problems and matters. This evaluation refers to the quality of service provided by the e-Learning Centre staff.

Thanks to this three-staged evaluation system the complex monitoring of training process is possible, the one that takes into account both the

232

education path and the organizational aspect. The students may pass filled survey forms either in the traditional way or by e-mail. The processed results are passed to students, e-tutors and the school authorities as a feedback information. To incorporate the other information, important in the quality assurance process, it was planned to verify the results of questionnaires with the progress of learning. This could protect against unfair opinions and problems reported by students having problems with a particular course. Moreover, to fill an informational gap also tutors will be questioned. This process will be done automatically via Decision Support System, which in this case will be supporting the e-learning management, particularly quality assurance. The system of the opinion survey adopted herein delivers its basics functions. However, when it comes to the questionnaire survey one ought to make allowance for the students’ subjectivity in their attitude toward the courses and the e-tutors. Therefore, at present the research is being carried out on the creation of the e-tutors’ evaluation mechanism which would use the methods of the artificial intelligence. Data used in this solution originate mostly from the LMS. Comparing the teachers to students it must be noted that is difficult to evaluate tutors’ performance. In our other researches we proposed and explained how data mining can be used to observe teachers and to support the management of e-learning platform (Pietruszkiewicz, &Dżega, 2010) and (Dżega, &Pietruszkiewicz, 2010). This process contained five major steps: • • • • •

Generation of reports with events registered for users. Conversion of events sets to time series. Calculation of activity attributes. Grouping users with a similar activity attributes. Identification of users with ‘active’, ‘moderate’ and ‘passive’ profiles.

The Technological Advancement of LMS Systems and E-Content Software

In the mentioned article we have proposed the measures of activity and tested in for Moodle LMS, that were used by data mining algorithms to group users with similar activity profiles. In our opinion the LMS should have more data processing and decision supporting features as the increasing complexity of LMS causes that this process is harder to observe and manage using standard functionality offered by nowadays LMS systems.

Novel Approach to Presentation of E-Content As it was presented in the previous sections, ZPSB used various software solutions to present the e-learning courses, however they caused some problems that needed to be addressed during the latest stage of e-learning technological upgrade – SPE e-content presentation software described in this section. The first issue was the time-consuming modifications of courses that involved edition of graphical files containing screens used in courses. Due to it, the modifications couldn’t be done without an image editor or Flash authoring tool and time required to create or correct a single screen was expected to be reduced. Easy modifications are essential as typing errors or material updates must be frequently done to maintain the quality of courses. The other important problem was a graphical representation of text on screens effecting their size. While this wasn’t decreasing the visual quality of presentation but caused that search functionality was impossible. Hence, users had to navigate through materials using menu without a possibility to look for a particular phrase interesting them. Another important problem was insufficient mechanism of zooming, while this feature is necessary to make the materials available to people with the sight problems. The last important issue related to an expected improvement of e-learning courses production All of these problems were analyzed and solved during design and development of SDART Presentation Engine (SPE later) that was created by SDART Ltd.

The first step of SPE development was a technological analysis that was required to find an appropriate technological solution for materials’ construction that should allow to keep their high visual quality and reduce or even remove the problems mentioned above. Therefore, SDART decided to develop Presentation Engine as RIA (Rich Internet Application) and keep the courses content in a form of styled XML-based files. In this form the software layer was separated from presentation layer and it was planned that update of presentation would be easy, fast and possible using any XML editor. The RIA was selected as this hybrid approach allows to use rich features of web pages – comparing to traditional desktop applications, having also the functionality of applications – comparing to ordinary web pages. Therefore RIA is a hybrid solution that: • •

•

• • •

•

• •

Has a simple installation or even usually does not require any installation steps. Is safe to use, as the service is less vulnerable to threats than the software requiring installation. Is easily available everywhere and at anytime for all users, only if they have an access to a machine connected to internet. Is a universal solution being hardware or operating system independent. Could be easily updated as all changes are immediately available for all users. Is more ergonomic than web pages, as most task could be done on client-side without constant reloads of web pages. Has graphical user interface (GUI) more attractive than a standard HTML-based service. Is easier to develop than pure HTMLbased service. Might process the data, present them in various forms and conduct an interaction with user.

233

The Technological Advancement of LMS Systems and E-Content Software

However, RIA has some disadvantages and their particular effects on a developed application must be analyzed. Thus, we had to remember RIA:

We have established a list of the most important quality factors and were looking for a technology that would:

•

• • • • •

•

• •

Has additional requirements for software i.e. plug-ins or frameworks. Run with a lower speed than desktop applications, as their code is interpreted and converted into a native code on client machine. Forbid access to critical resources to maintain security. Are fully dependent on server and network, any problem with speed or stability of them causes problems with RIA.

More information about RIA quality was presented in (Pietruszkiewicz, &Dżega, 2009). The mentioned advantages and disadvantages were analyzed and addressed during RIA technology selection and SPE design. The technology selection is an essential step for each software project as the selected technology defines the shape of developed software. Currently there exists several RIA technologies on market – Oracle Java™, Adobe Flash™, AJAX, Adobe Flex™, Laszlo Systems OpenLaszlo™, Oracle JavaFX™ and Microsoft Silverlight™ being the most popular ones.

• •

Have a high speed and fast start-up time. Offer multi-platform support. Have prototyping capabilities. Not require any unpopular plug-in. Make it possible to be developed by Rapid Application Development tools. Have easy customized GUI components. Offer attractive GUI.

For this particular project, Adobe Flex™ technology was selected as the best matching requirements for e-learning presentation engine, especially as at runtime it requires Flash plug-in being the most popular web plug-in, has advanced RAD (Rapid Application Development) tools (Flex Builder and Flex plug-in for Visual Studio) and GUI components are easily customized.

Production Process for SPE Software As it was mentioned it the previous sections, the production of e-learning content involves a few steps. For the e-learning materials being produced using SPE these steps are (Figure 4):

Figure 4. The stages of production for SDART Presentation Engine

234

The Technological Advancement of LMS Systems and E-Content Software

•

•

•

•

•

Scenarios production – provided by the authors constitute a base of e-learning production. Parallel production of media files and XML – at this step scenarios are converted to XML and media files (usually graphics) are prepared from sketches or descriptions provided by authors. XML testing and adjustments – involve joining of XML with media files and their testing, some elements on slides may be presented in different forms e.g. large images that do not fit into a screen together with text could be changed into a thumbnail button showing them on a separate popup window only via simple change of one image attribute in XML configuration file. PDF generation – this step is done automatically by a SPEEG program being a part of SPE (the advantages of this program will be explained in the following section) that using XML and media files produces PDF e-book. SCORM packaging – it is the last of stage of production and involves creation of SCORM metafiles and SCORM compression to a ZIP archive.

After the production SCORM package might be deployed on LMS. The most frequent changes to e-learning content that need to be done during materials production involve only edition of XML file. It must be noted that usually, for the other presentation engines, the production must be fully synchronized and text and images cannot be prepared parallelly. This is a problem when images are created by other person than the one preparing XML. In SPE media materials can be embedded in different ways, hence it is possible to decide about it during testing&adjustments phase. The changes of course (typos, formatting, update etc.) that occur after SCORM was produced

and deployed also might be simply done by a reedition of XML file. This is a very easy process and neither SCORM have to be re-packaged (XML replacement in archive is fine) nor any other file apart of XML must be changed. The low maintenance effort for e-learning courses was also a important target to achieve during SPE design&development.

Effects of SPE Deployment and Requirements The critical analysis of solutions used by ZPSB done during SPE development resulted in a software having a few distinguishable advantages that will be analyzed in this section. First of all, it does not require sophisticated software to create e-learning courses or to modify them. The main course part i.e. XML configuration file may be edited by any XML editor or even standard text editor (practically a syntax coloring is sufficient). As the courses may use various media files i.e. JPG, PNG or SWF they also may be produced by many freely available graphical tools. If we consider XML itself, it might said that this form of presentation is human readable and could be easily understood by the staff producing e-learning courses. It is also easy to construct these files as XML schema might be filled by text copied from the scenarios provided by authors. Moreover, commonly to all XML files, the content is separated from the formatting. It is possible to give a special formatting to the elements of presentation but the basic formatting (font, styles, sizes) is provided by engine according the meaning of these elements. Additionally, the courses created with SPE are easy to modify. In the result they may also be easily updated and re-deployed. This feature is important during evaluation of courses as small typing errors are a common situation and presentation mechanism should allow to easily correct them and re-deploy new versions of materials. Using SPE courses might be re-edited by modifying

235

The Technological Advancement of LMS Systems and E-Content Software

XML and replacing them in SCORM packages. Analyzing time require to produce the e-learning courses with the previous presentation engines it was calculated that for ZPSB case SPE reduced time of production by 70%. Further, the process of modification was also significantly reduced. The generation of files is easy to learn as to format text HTML tags may be used. From the functional point of view a textual representation of course and its visual generation by an RIA engine may be extended by search and zooming features. This first feature means that users may look for materials interesting them, while the second one is important for people that does not see standard fonts clearly. SPE was designed to avoid a common mistake of RIA i.e. fixation of fonts’ size that causes problems of people with sight problems. SPE also was build in a way ensuring that different types of materials might be embedded. They include images as well as animations, audio and video files. Using SWF files it is also possible to use interactive elements build for a particular presentation e.g. games or puzzles. The additional part of each course is its PDF e-book being preferred in some situation by users e.g. by users wanting to print the materials or to read them on devices without Internet connection (during the journey). SPE was designed to generate these files automatically by SPEEG (SPE E-book Generator) being an application converting XML course files into PDF e-books. Comparing to the previous presentation engines used by ZPSB, which required a manual generation of PDF files, SPEPG does it in a few minutes. The technical requirements to use SPE could be divided into two groups i.e. those relating to the content production and those required to run e-learning course. Considering the first group, the production of e-learning courses using SPE does not require any unpopular or costly software tools. It might be done using standard and freely available software. The necessary tool is any XML editor. It might be a sophisticated XML editor as

236

well as a basic one from many cost-free editors available or even a text editor with syntax coloring (a popular solution among the web-programmers). Another required tool is a graphical editor or SWF producer. SPE uses bitmap images or vector clips in SWF format. Bitmap images might be produced by many editors including free editors like e.g. popular GIMP, while SWF might be produced by Flash authoring tools e.g. Adobe Flash™ IDE, SWiSH or exported from OpenOffice suite. The high cost of software to create e-learning courses is a common problem of e-learning authoring software. SPE does not have such requirements because this issue was considered during the its design and development and it was conducted not only to provide a high functionality of SPE but also to make e-learning production faster, easier and less expensive. The runtime requirements are practically ensured by each desktop computer. It must provide a web browser equipped with a Flash plug-in, which is available for all major browser. SPE is also hardware and Operating System independent, therefore may be used on any modern computer.

SPE Functionality and Structure On Figure 5 a SPE screenshot and e-book were shown. Both were taken from the same XML course file, while screen version is generated online by SPE each time users watches slides and e-book was generated off-line and added to the learning materials. For the obvious reasons PDF does not contain any interactive elements and primarily can be used to read or printing materials. The main form of e-learning course are interactive SPE materials available via web browser. The online materials presented on the screenshot 5a. reveal a few parts of application: •

Navigation section 1– allowing users to move forward and backward.

The Technological Advancement of LMS Systems and E-Content Software

Figure 5. The example of SPE-based e-learning materials – online course (a) and offline e-book (b)

237

The Technological Advancement of LMS Systems and E-Content Software

•

•

•

• •

Course structure 2– this easy accessible course structure allows users to jump directly to a desired section. Main presentation area 3 – containing the presentation i.e. text, graphics or interactive elements, this area is also used to show the test questions. Search window 4a or search field 4b with the search results – the search field is always visible, while the search window appears after search was started and contains navigable list of screens for which a phrase looked for was found, this window keeps to be open until user decided to close it, hence it is possible to perform a fast verification of found screens without the necessity to re-run searching multiple times. Zooming panel 5 – allowing to increase/ decrease font size. Test exam 6 – is a short exam with interactive tests with a summary at the end of exam.

The structure of SPE presentation window was designed to fulfill the most fundamental software quality factors relating to the user’s experience with the software i.e.: •

• • •

Visual simplicity – users should not be forced read a manual to use software, if it is not really required. Intuitiveness – design software in an intuitive way. Ergonomic – make the functions easily accessible. Functionality – ensure that available functions will be the ones required by user.

The different ways of course navigation are possible in SPE. The first one – a full navigation assumes that user will read screen after screen and is available using standard ‘next’ and ‘previous’ buttons. The second one allows users to navigate directly to a selected section and is done using

238

clickable course structure. The third one searches for user-defined phrase and lead user to the found screens. All of these types of navigation are required for different kinds of users or even the same users taking the course at various moments. Some users prefer learning courses from the first to last screen, the others learn courses during multiple short sessions and there also exists users interested only in screens with information relating to their demand (search phrase). During runtime SPE loads XML course file, while images are load on demand i.e. only if they are required by a slide presented to user. Considering the low size of vector graphics (primarily used on screens) it effect the start-up time of course which is very fast and necessary files are also downloaded shortly. Comparing to the systems with a graphical text representation, the method used by SPE is faster as text contained in XML is available immediately, thus user does not observe any reload gaps between the screens. Figure 6 contains two subfigures with UML diagrams relating to the SPE functionality (a) and structure (b). As presented on use case diagram SPE supports the functionality for e-learning courses including: •

•

•

• •

Navigation – user may navigate through screens (next, previous) or go to a particular section. Presentation – information stored in elearning content are presented by SPE on screens. Knowledge testing – done using test screens during the learning progress, allowing users to examine correct answers, the other way is an preliminary tests evaluating users’ answers and returning a summary without information about correct ones. Search – user may search for a particular phrases and go to found screens. Zooming – changing presentation by adjusting the font size for presented material.

The Technological Advancement of LMS Systems and E-Content Software

Figure 6. Use case (a) and deployment (b) UML diagrams for SPE

These features were observed or reported by users as the most useful ones and required to fully use e-learning materials. The second diagram on Figure 6 explains how SPE-based course might be deployed and run from LMS. ZPSB uses SPE on Moodle LMS but as this software is LMS-independent, SPE might be used also with other LMS. The only requirement for LMS is that such system must use SCORM packaged content, which an obvious requirement for any mature LMS. SPE uses popular web technologies in a way that ensures high quality of e-learning courses and keeping their development simple and immediately understandable to all people with any web-development experience. A proprietary and rare solutions are significant problem for various e-learning systems, that too often reject the developers’ experience. The technologies used by SPE are: Flex, XML, HTML, Flash and standard multimedia files. As it was mentioned earlier Flex is RIA technology based on MXML language., while MXML is a XML application mixed with ActionScript programming language (more generally ECMAScript). However, it is important technical feature

for developers but form user’s perspective it irrelevant, because SPE works as compiled engine that could be deployed on any server or client. For users the more interesting technologies are ones relating to the production of e-learning content. The basic information about e-learning structure are represented by a XML file used by SPE (its structure and capabilities will discussed in the next section). HTML was used as a formatting language for text. It is important feature, because HTML is a very popular language and its knowledge is belongs to the basic skill of people working with web technologies. Thus, materials might be constructed even by a beginning users even with a limited web development experience. The flash technology is used at run-time by SPE as this Flex application was compiled into a Flash code and according to surveys Flash reaches 99% of Internet enabled desktops (June 2010). Due to it SPE might be used with a greater confidence than any presentation engines using different run-time technologies as they are less popular or isn’t a homogeneous on various browsers or Operating Systems as Flash is. However, in the future Flash superiority might change as JavaScript is promoted as a dominant part of HTML 5 but considering

239

The Technological Advancement of LMS Systems and E-Content Software

the past experience with web standardization it might even take a few years until all browsers and OS will behave identically. The other technologies used by SPE include raster graphics and PDF generated automatically from XML-based content. The important feature of Flash runtime is a support for graphic hardware resulting in a high speed of visual effects like glowing, blurring, shades etc which were used in SPE and positively influence users’ web experience. To explain the advantages of RIA-based elearning presentation engine using XML as a representation for courses we will briefly analyze features of XML application used by SPE. There are four main types of screens: •

Textual screens with or without multimedia materials e.g. course content screen, introductions and typical content screen. Interactive presenting screens with popups and links to the dictionary. Interactive testing screens used by user to self-check knowledge. Special screens e.g. title screen or dictionary screen.

• • •

An example of the first type of screen is below: <slide type=”D” title=”Text screen”> <main>Text example <sup>1:

bullet point 1,

<slide type=”G” title=”Popup screen”> <main>Text with
“event:popup1”> popup element 1 ...


Another line of text with popup element 2 ...


<popup name=”popup1”>Popup 1 text

<popup name=”popup2”>Popup 2 text



The third group uses screens like one below: <slide type=”I” title=”Test screen”> A question to answer


wrong=”Incorrect answer1|Incorrect answer 2|...” />





An example of screen from the last group is below: <slide type=”T” title=”Dictionary”>

<element name=”element1”>Explanation

bullet point 2,

of element 1

bullet point 3 with italics style.

And later bolded text

1. Author Title, Publisher ...



For the second group of screens an example was provided below:

240

<element name=”element2”>Explanation of element 2

...



As it can be seen screens have an easy to understand structure and are very informative (a typical characteristic of tagged languages). Additionally, the content may include a subset of HTML formatting tags being the most popular and important ones e.g. italics, bold, anchors, breaks or fonts. Using even a simple text editor

The Technological Advancement of LMS Systems and E-Content Software

with a syntax coloring the person editing course may understand parts of the presentation and fill slides templates. The textual course representation is approx. ten times smaller than the size of bitmap converted text. Considering the future extensions, the XMLbased representation of courses is also safer solution as this is an universal form that might be authored by various tools as well as processed by different programming languages. Even in situation when SPE will be rewritten using a new coming technologies (e.g. HTML5) it will be possible to use old XML files or update XML courses to a new versions schema via conversion scripts. This problem should be remembered by each organization deploying a new software as the data usually live longer than the software that was used to create it. Moreover, XML isn’t a proprietary format and there are several technologies extending it e.g. search language XPath or transformation language XSLT.

SPE Further Development Considering the features presented in the last section it is possible to say that SPE-based e-learning courses, incorporating different technologies, are synergic examples of a modern web application featuring the important aspect relating to the quality of web services i.e. flexibility, easy modification, universality or not limiting the future usage of generated data. However, in the future SDART Presentation Engine will be further developed by the additional features, giving SPE even more functionality. It is planned to add new screens, especially more interactive test slides allowing users to self evaluate learned knowledge. The embedded multimedia materials might be currently included in SPE as SWF files, but it is planned to introduce new tags that will be used to represent audio and video files. As SPE could be used for various e-learning tasks i.e. in the higher education, corporate education or government educational projects, SPE will be

extended by various interactive clips enlarging its interactive capabilities and engaging users more with the courses. At the moment it is possible to adopt SPE to a new visual style via replacement of background images and this feature is planned to be further extended and introduced in the configuration part of XML file. Hence, SPE will be even more easy to adapt to a new style of curses. Another important feature of new generation e-learning courses is intelligence. Therefore, SPE will use Artificial Intelligence methods to analyze the progress of learning and recommend the important materials for user e.g. materials that could be forgotten be user from the last time he/ she was learning or materials relevant to semantic search not only matching given keywords. AI will be also used to analyze users’ behavior during the usage of presentations and to improve courses by emphasizing the important parts or restructuring the courses. The experiments over adaptive elearning process were performed by (Liu, &Yang, 2005) proposing APeLS system, by (Rossi, 2009) proposing e-learning joined with Web 3.0 or using SCO approach as it was presented by (Brkovic, Milosevic, &Krneta 2006). The last possible area of further development are increased features for handicapped people. They will include a voice synthesizer and increased visual accessibility e.g. via switchable high contrast. The analysis of e-learning environment from the perspective of students with sight disabilities was done by (Drigas, Koukianakis, &Papagerasimou 2006).

CONCLUSION The technical advancement and changes in the life style increase the popularity and significance of methods and techniques of distance education. As e-learning becomes more popular also the demands and requirements relating to it are changing. The users expect courses to be tailored to their needs

241

The Technological Advancement of LMS Systems and E-Content Software

and IT services supporting their process of learning. Therefore LMS software must be adjusted to fulfill users’ expectation. These features relate to users’ perspective, but e-learning is also very demanding activity form the business perspective. For the universities the production process of e-learning materials becomes more and more complex, time and resources consuming task, therefore it is essential to use software and technologies offering the high quality for produced materials, keeping the production process to be simple and not expensive. This chapter presented the process of evolution for LMS and e-content software used by West Pomeranian Business School. At the beginning chapter explained the requirements for different types of studies and how they influenced the shape of LMS systems. Later, the chapter analyzed different technologies and software that were used in the e-learning process. In its last part the chapter presented SPE e-content software being an innovative e-learning presentation engine developed in form of Rich Internet Application by SDART Ltd The content of each section included observations and obstacles encountered at different stages of LMS deployment and evolution. The material presented herein does not constitute the final step of LMS evolution as we started the investigation over the further areas of LMS extensions. It is planned to develop a new version of presentation engine SPE e.g. to incorporate artificial intelligence features to make the e-learning content self-adjustable. The AI features are also planned to be used in the module supporting the management of LMS, as the complexity of this process i.e. number of users, observable activities and their character, make this process hard to perform without a supporting software. This module will preprocesses the data, search for the usage patterns and deliver easily understandable comments. In our opinion the further development of Learning Management Systems should and will be done in two major directions – by using intelligent

242

mechanisms, supporting users in the process of learning, being a move towards Web 3.0 applied in e-learning; and by using technologies making e-learning even more available and convenient, moving it from the desktop computers or laptops to mobile devices. Both of these areas are investigation by ZPSB and SDART Ltd.

REFERENCES Brkovic, M., Milosevic, D., & Krneta, R. (2006). SCos for adaptive e-learning. 28th International Conference on Information Technology Interfaces (pp. 29-34). Cavtat/Dubrovnik. Drigas, A., Koukianakis, L., & Papagerasimou, Y. (2006). An e-learning environment for nontraditional students with sight disabilities. 36th Annual Frontiers in Education Conference, San Diego, CA (pp. 23-27). Dżega, D., & Pietruszkiewicz, W. (2010). The support of e-learning platform management by the extraction of activity features and clustering based observation of users. In S. Konstantopoulos, S. Perantonis, V. Karkaletsis, C. Spyropoulos, & G. Vouros (Eds.), Artificial Intelligence: Theories, Model and Applications-SETN 2010 (pp. 315320). LNAI 6040. Heidelberg, Berlin, Germany: Springer-Verlag. Horton, W. (2006). E-learning by design. San Francisco, CA: John Wiley & Sons, Inc. Liu, H., & Yang, M. (2005). QoL guaranteed adaptation and personalization in e-learning systems. IEEE Transactions on Education, 48(4), 676–687. doi:10.1109/TE.2005.858398 Pietruszkiewicz, W., & Dżega, D. (2009). The practical aspects of rich Internet application development and quality factors: RIA–based decision support system. Quality assessment in Web 2009 (Vol. 561). CEUR.

The Technological Advancement of LMS Systems and E-Content Software

Pietruszkiewicz, W., & Dżega, D. (2010). An application of data mining in the management of e-learning platform . In Korczak, J. (Ed.), Business informatics. Data mining and business intelligence (pp. 60–70). Wrocław, PA: Publishing House of Wrocław University of Economics. Rossi, P. G. (2009). Learning environment with elements of artificial intelligence. Journal of Elearning and Knowledge Society - English Version, 5(1), 193-199. Swedish National Agency for Higher Education. (2008). E-learning quality: Aspects and criteria for evaluation of e-learning in higher education. Report 2008:11 R, PA: Swedish National Agency for Higher Education, Stockholm. Retrieved on July 30, 2010, from http://www.hsv.se/downloa d/18.8f0e4c9119e2b4a60c800028057/0811R.pdf

ADDITIONAL READING Allen, M. W. (2007). Designing successful elearning, Michael Allen’s online learning library: Forget what you know about instructional design and do something interesting. San Francisco, CA: John Willey & Sons, Inc. Berggren, A., Burgos, D., & Fontana, J. M. (2005). Practical and pedagogical issues for teacher adoption of IMS learning design standards in Moodle LMS. Journal of Interactive Media in Education, 2005(02). Bersin, J. (2004). The blended learning book: Best practices, proven methodologies, and lessons learned. San Francisco, CA: John Willey & Sons, Inc. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11(2), 65–70. doi:10.1016/j. iheduc.2008.03.002

Boticario, J. G., & Santos, O. C. (2007). An open IMS-based user modelling approach for developing adaptive learning management systems. Journal of Interactive Media in Education, 2007 (2). Retrieved on August 12, 2010, from http:// citeseerx.ist.psu.edu/viewdoc/download?doi=10 .1.1.101.3799&rep=rep1&type=pdf Carliner, S., & Shank, P. (2008). The e-learning handbook: A comprehensive guide to online learning. San Francisco, CA: John Willey & Sons, Inc. Cavus, N. (2010). The evaluation of learning management systems using an artificial intelligence fuzzy logic algorithm. Advances in Engineering Software, 41(2), 248–254. doi:10.1016/j.advengsoft.2009.07.009 Clark, R. C., & Mayer, R. E. (2008). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: John Willey & Sons, Inc. Cole, J. (2008). Using Moodle: Teaching with the popular open source course management system. Sebastopol, CA: O’Reilly Media, Inc. Ćukušić, M., Alfirević, N., & Granić, A. (2010). E-learning process management and the e-learning performance: Results of a European empirical study. Computers & Education, 55(2), 554–565. doi:10.1016/j.compedu.2010.02.017 Del Puerto Paule Ruiz, M., Jesús Fernández Díaz, M., & Ortín Sole, F. (2008). Adaptation in current e-learning systems. Computer Standards & Interfaces, 30(1-2), 62–70. doi:10.1016/j. csi.2007.07.006 Gaeta, M., Orciuoli, F., & Ritrovato, P. (2009). Advanced ontology management system for personalised e-Learning. Knowledge-Based Systems, 22(4), 292–301. doi:10.1016/j.knosys.2009.01.006

243

The Technological Advancement of LMS Systems and E-Content Software

Hannon, J., & D’Netto, B. (2007). Cultural diversity online: student engagement with learning technologies. International Journal of Educational Management, 21(5), 418–432. doi:10.1108/09513540710760192

Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j. compedu.2007.05.016

Horton, W., & Horton, K. (2003). E-learning Tools and Technologies: A consumer’s guide for trainers, teachers, educators, and instructional designers. Indianapolis: William Horton Consulting, Inc.

Rosenberg, M. J. (2001). E-Learning: Strategies for Delivering Knowledge in the Digital Age. New York, NY: McGraw-Hill.

Kritikou, Y., Demestichas, P., & Adamopoulou, E. (2008). User Profile Modeling in the context of web-based learning management systems. Journal of Network and Computer Applications, 31(4), 603–627. doi:10.1016/j.jnca.2007.11.006 Lewis, B. A., MacEntee, V. M., DeLaCruz, S., et al. (2005). Learning Management Systems Comparison. Proceedings of the 2005 Informing Science and IT Education Joint Conference. Retrieved August 02, 2010, from http://www.ultimedia. co.uk/upload/E-Learning%20and%20LMS%20 Comparisons/Learning%20Management%20 Systems%20Comparison%20P03f55Lewis.pdf Oztekin, A., Kong, Z. J., & Uysal, O. (2010). UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. International Journal of Industrial Ergonomics, 40(4), 455–469. doi:10.1016/j. ergon.2010.04.001 Rice, W. (2007). Moodle Teaching Techniques: Creative Ways to Use Moodle for Constructing Online Learning Solutions. Birmingham: Packt Publishing. Rice, W. (2008). Moodle 1.9 E-Learning Course Development: A complete guide to successful learning using Moodle. Birmingham: Packt Publishing.

244

Shen, R., Han, P., Yang, F., & Yang, Q. (2003). Data mining and case-based reasoning for distance learning . Journal of Distance Education Technologies, 3(1). Tang, T.Y., & McCall,a G. (2002), Student modelling for a web-based learning environment: A data mining approach, [in:] Eighteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park. Watson, W. R., & Watson, S. L. (2007). An Argument for Clarity: What are Learning Management Systems, What are They Not, and What Should They Become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-007-0023-y Zajac, M. (2009). Using learning styles to personalize online learning. CampusWide Information Systems, 26(3), 256–265. doi:10.1108/10650740910967410

KEY TERMS AND DEFINITIONS Blended Learning: A mixed form of learning where traditional learning is used together with e-learning form. E-Content: A set of materials containing course information and exercises. LMS: Software system used to manage the process of learning.

The Technological Advancement of LMS Systems and E-Content Software

RIA: Web-based application with advanced interactivity and rich visual style. SCORM: A common standard of data storage used to represent universal courses in e-learning platforms.

SPE: SDART Presentation Engine being a software used to present e-content. XML: A tagged language used to represent data structures with logical meaning of its elements.

245

Section 4

Case Studies

247

Chapter 12

Differences in Internet and LMS Usage:

A Case Study in Higher Education Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues NID-RH, ESEIG, Portugal Carla Teixeira Lopes Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira ISEP, Portugal Ricardo Queirós KMILT, ESEIG, Portugal Mário Pinto KMILT, ESEIG, Portugal

ABSTRACT The Internet plays an important role in higher education institutions where Learning Management Systems (LMS) occupies a main role in the eLearning realm. In this chapter we aim to characterize the Internet and LMS usage patterns and their role in the largest Portuguese Polytechnic Institute. The usage patterns were analyzed in two components: characterization of Internet usage and the role of Internet and LMS in education. Using a quantitative approach, the data analysis describes the differences between gender, age and scientific fields. The carried qualitative analysis allows a better understanding of students’ both motivations, opinions and suggestions of improvement. The outcome of this work is the presentation of the Portuguese students’ profile regarding Internet and LMS usage patterns. We expect that these results can be used to select the most suitable digital pedagogical processes and tools to be adopted regarding the learning process and most adequate LMS’s policies. DOI: 10.4018/978-1-60960-884-2.ch012

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Differences in Internet and LMS Usage

INTRODUCTION In recent years, Learning Management Systems (LMS) has been assumed as the cornerstone of the learning environments wide spread in the majority of the academic institutions. The central role occupied by the LMS in a typical architecture of a learning environment forces us to study its structure and its use by all users, whether they are teachers or students, in the teaching/learning process. In order to improve the LMS usage as a tool to support teaching and learning in an effective way, and since LMS are delivered by Internet, it is important to know the Internet students’ usage habits, concerning the Internet and LMS usage. This information will allow us to adopt the most appropriate learning and pedagogical strategies, according the students’ profile, characteristics and preferences. It is also relevant to know whether these habits of Internet and LMS usage are in line with the following studies presented in this section. This study aims to characterize higher education students’ of Polytechnic of Porto regarding their behavior on the Internet and eLearning platform, and it seeks to investigate how students use the information and communication technologies in their learning activities. Based on these, we propose suggestions to improve LMS usage what will be of interest to the definition of organizational policies. The motivation for this work came from the heterogeneity of students’ profiles found in the Polytechnic of Porto and the need to enhance the teachers’ pedagogical strategy based on the students’ profile. The study is focused on gender, age and scientific field. These criteria were chosen based on several studies regarding LMS and Internet and due to the variety of Polytechnic of Porto’s students characteristics. Several studies across the world have been combining the characterization and analysis of students’ Internet usage patterns and perceptions of how technologies could be used in learning at

248

university: Australia (Kennedy, Dalgarno, Sturt, Bennett, Gray, Waycott, Judd, Bishop, Maton, Krause & Chang, 2009), United Kingdom (Conole, Laat, Dillon & Darby, 2006), USA (Walker and Jorn, 2007). Since these LMS platforms present opportunities previously unavailable for academic institutions to apply their financial, logistical, and instructional resources (Uzunboylu, Ozdamli & Ozcinar, 2006), knowing the students perceptions will allow, not only to enhance Polytechnic of Porto teachers’ perceptions of what has the most impact on their students, but also, to define policies regarding eLearning platforms as a support to traditional education delivery and understand needs to be improved to undertake on distance and blended learning courses offers. The remainder of this paper is organized as follows. Section 2 traces the evolution of the eLearning systems, highlighting the LMS. In the following section we begin by describing the research methodology used for the study such as data collection and population and sample. Then, we present the data analysis and discussion. The last section focuses on the main contributions of this work, more precisely, the results of this study and a perspective of future research.

STATE OF THE ART Learning Management Systems Electronic Learning (eLearning) can be defined as the delivery of educational content via any electronic media, including the Internet, satellite broadcast, audio/video tape, interactive TV, CD-Rom and others (Tastle, White, & Shackleton, 2005). Despite some efforts to improve remote education (Harasim, 2006), the genesis of eLearning can be traced with the development of network communication in the late 1960s, more precisely, with the invention of email and computer conferencing. These innovations con-

Differences in Internet and LMS Usage

tribute to the collaboration between teachers and students and initiate a new education paradigm shift (Williams & Goldberg, 2005). During the 1980s and 1990s, there was a significant growth in the number of students studying part-time and also in non-traditional learners, such as, typical 18 to 24 years old students seeking the university demand and women’s returning to the workforce after child rearing (Williams & Goldberg, 2005). The growth in lifelong learning has made the educational institutions to seek for flexible education delivery to satisfy these non-traditional students. By the end of the century, this delivery has been accentuated by the emergence of new forms of distance delivery based on information and communication technologies advances, such as, the Internet. In their first generation, eLearning systems were developed for a specific learning domain and had a monolithic architecture. Gradually, these systems evolved and became domain-independent, featuring reusable tools that can be effectively used virtually in any eLearning course. The systems that reach this level of maturity usually follow a component-oriented architecture in order to facilitate tool integration (Leal & Queirós, 2010). Different kinds of component based eLearning systems target specific aspects of eLearning, such as student or course management. One such case is the LMS that aims to simplify the management of learning within an organization (Harman & Koohang, 2007).This type of system allows students to plan their learning and collaboration activities with colleagues, while teachers may associate educational content and monitor, analyze and report progress of their students. Most LMS’s are structured around courses rather than courses’ content thus, they only support reusability at the course level, where many learners can enroll on a single course. LMS also don’t support the creation of instructional content. This “issue” implies the use of third part content creation tools. Despite these issues, the LMS are the cornerstone of the learning environments wide spread in the

majority of the academic institutions. Typically, they have two types of users’ groups: learners and teachers. Learners can use the LMS to plan their learning experience and to collaborate with their colleagues; teachers can deliver educational content and track, analyze and report the learner evolution within an organization.

LMS in Higher Education Higher education institutions have been challenged by the emergence of new information and communication technologies (ICT), in the last few years. Among various technologies, eLearning and LMS have been widely adopted at higher education, motivating researchers and teachers to study its influence in the learning process (e.g. Misko, Choi, & Lee, 2004; Gras, 2005). In fact, Mayadas (2007) states that in 2006 there were 3,5 million students enrolled in at least one online course in United States (US); this was about 20% of all university students in the US. Since then until now, online learning is been growing 20% a year. Today, New York University enrolls almost 130.000 online students in LMS platforms (NYU, 2010). A vast number of LMS that provide integrated services exists nowadays, like for instance, WebCT, Blackboard, Centra, Learning Space and Moodle (Botturi, Cantoni, & Tardini, 2006; Babo & Azevedo, 2009). Within these eLearning systems, Moodle assumes an important role (Babo & Azevedo, 2009). It is an open source solution, offers a set of scalable tools and services that support communication and collaboration across students and teachers and there are a lot of components that we can add to the platform (Georgouli, Skalkidis, & Gerreiro, 2008). In early January of 2010, Moodle had a userbase of 46,624 registered sites with 32,464,992 users in 3,161,291 courses in 209 countries and in more than 75 languages (Moodle, 2010). According to some authors (Monsakul, 2007; Moodle, 2010), the following are the most common func-

249

Differences in Internet and LMS Usage

tions of LMS’s that we can find in Moodle: i) course information and documentation; ii) document repository; iii) announcements; iv) discussion board; v) external links; vi) synchronous and asynchronous communication (email, chat room, discussion forum); vii) assignments; ix) users profiles. There is also some plug-ins that we can add to Moodle, in order to install other functionalities like, for instance, hot potatoes and Sharable Content Object Reference Model (SCORM) features. According to Landsberger (2004), LMS’s relevant features in higher education are about: i) courseorganizing functions, such as electronic and multimedia contents, evaluation and assessment tools; ii) dynamic tools to facilitate the learning process, such as synchronous and asynchronous discussion groups; iii) student’s environments to enhance collaboration and cooperation between them. With its various functionalities, LMS’s serve different learner’s characteristics, different learning styles and outcomes (Landsberger, 2004; Monsakul, 2007). To understand all these changes and to see how students are dealing with them, several studies were made to characterize Internet usage patterns and LMS implementations in higher education: e.g. testing for a variety of strategies in an eLearning platform (Oliveira, Oliveira, Souza, & Costa, 2006); other findings point the need to LMS support solving problems and critical thinking functionalities (Andronico, Carbonaro, Colazzo, Molinari, Ronchetti & Trifonova, 2004; Monsakul, 2007); still others conclude that teaching and learning pedagogy should be modified to the new student’s typical profile, which includes the capacity of learning anytime, anywhere, across multi-platforms and with multi-tasking (Kubora, Terashima, Nakahashi, & Morioka, 2008). Botturi, Cantoni, &Tardini (2006) conducted a survey at University of Lugano in order to evaluate students’ usage and satisfaction with eLearning platform, Moodle. The survey indicates that 94,3% of the respondents is satisfied with the platform usage, but only communication forums

250

and online submission have been widely used by students and teachers. On the other hand, only 44% of the respondent noticed positives changes in their learning process after the adoption of Moodle platform. Several studies point to different Internet usage habits, depending on the criteria discussed: male students predominate in the use of Internet; female students prefer contents based on science learning environment while male tend to prefer distracting contents; the Internet usage increases by more 10 percentage points in each age group between 2006 and 2009; the number of messages sent and received is clearly greater in men than women (Kubora et al., 2008). Introducing gender as a criterion in studies in the ICT fields consistently show a male bias towards Internet use (Kubora et al., 2008; Madell & Muncer, 2004; Nachmias, Mioduser, & Shemla, 2001; Sink, Stob & Taniguchi, 2008). And, especially in an education setting and, in order to get equity among graduates facing the labor market, gender is an issue that can’t be dismissed (Madell & Muncer, 2004). Age is another criterion that is very much considered in ICT studies. In recent years there has been widespread interest in studying young people’s relation with ICT. Tapscott (1999) introduced, in the late 90’s, the term Net generation to characterize a whole new population born after 1980 that grew in an environment where technological artifacts and digital culture were a part of their everyday life. The computer, Internet, cellular phone, digital camera, digital games and social networks are, among others, examples of how present is digital technology in this generation’s world, in such a way that these are not viewed as new technologies but as tools that have always been there (Prensky, 2001). This generation tends to work and learn, unlike traditional approach, developing some particular characteristics: multitasking, visual learning, hypertext, short messages, and compact information, among others (Kennedy et al., 2009; Simões, 2009). Burlea (2008) found in her study, that the older students

Differences in Internet and LMS Usage

(from 46-64 years old) found more difficulties of individually using the facilities offered by the information system. For example, Kennedy et al. (2009) found slight differences between generations of students and teaching staff in their study, as they said a little unexpectedly. As Polytechnic of Porto is known for having a large percentage of students who are (or has been in any moment of their lives) professionals (as opposed to students that are continuing their studies after high school without going through the labor market), and these students are older, it was found to be interesting to study the behavior and perceptions of different generation students, in the same learning context, towards Internet and LMS platform supporting face-to-face classes. In the study promoted by The Australian Learning and Teaching Council (Kennedy et al. 2009) they considered “a key aspect of the Investigation stage” (p.17) to analyze the discipline in which they were studying: students were asked to nominate the course and subject in which they were responding to the questionnaire and these were used to classify students into the discipline categories, according to the Australian Standard Classification of Education Codes. Their assumption was to consider how the students’ experiences with technology might differ as a function of the field they were studying. They did not found relevant statistical differences between discipline areas, this is, they reported around the same level of technology use in all five discipline areas (Arts, Sciences or others). However, clear differences between the three universities students’ engaged in the study (Kennedy et al. 2009). In Polytechnic of Porto there are also very different study areas and, there is the common sense perception and, to some extent, empirical evidence, that students that engaged in Informatics courses tend to use LMS more than others, since their teachers do it (Rodrigues, Pinto, Queirós, 2010). Rodrigues et al. (2010) studied Moodle sophistication usage, using Janossy & Hover’s (2008) model in one of Polytechnic of Porto schools (ESEIG1) by teach-

ers and, systematically found that Informatics department teachers used Moodle more and in a more complex manner than teachers from other scientific fields.

Research Methodology To characterize Internet and LMS usage patterns and their role in Higher Education, a descriptive case study with students from Polytechnic of Porto was conducted. In Polytechnic of Porto, all courses are delivered in presential classes and students must enroll in a minimum of classes (except for the working students that have a special authorization). LMS (Moodle) is used as a support to presential classes and, it is not mandatory. The students’ usage patterns were analyzed in two components: Internet usage and LMS in Education. Internet usage patterns is characterized by type (e.g.: location, frequency, type of use, motivation) and by the underneath communication tools (e.g.: social networks, chat, email in frequency, activities, benefits). The role of LMS in education deals with items such as: types of functionalities, their frequency of use and their perceived importance.

Data Collection Data was collected through an online questionnaire distributed to students of five Polytechnic of Porto’s schools: ESEIG2, ESTSP3, ESTGF4, ISCAP5, and ISEP6. This instrument was chosen because it allows the collection of a large number of answers at a reasonable cost. The questionnaire had twenty-seven questions, five of which were open. In the first week of June of 2009 a pilot test was carried out with 15 students from various programs and academic years of ESEIG. This test validated the questionnaire’s objectivity, understanding and also the web form’s accessibility. In the second week of June 2009 began the dissemination of the questionnaire, in the

251

Differences in Internet and LMS Usage

various schools of Polytechnic of Porto. The announcement was the same in all schools and was done by: announcements on the school website, announcements on the school learning management system, email messages sent to students and teachers (so they would ask their students to answer the questionnaire) and requests to teachers, who used computers in classrooms to free class time so that students could complete the questionnaire. The questionnaire answers were collected anonymously, from June 5th until the end of July. In July, the schools’ academic services were contacted in order to gather descriptive data about the population.

Population and Sample Similar to what is done in other studies (e.g. Walker & Jorn, 2007), the case study methodology was adopted for this study. Students from five Polytechnic of Porto’s schools compose the universe of this study. The study included students from technological specialization programs and also from undergraduate and graduate programs. From the 1416 obtained answers, 1397 were considered valid (11% of Polytechnic of Porto’s population).

Figure 1. Sample distribution across schools

252

In Figure 1, it is visible the respondents distribution according to the school where they study. According to the Ministry of Science, Technology and Higher Education (DGES, 2010), Portugal has ten study fields: Science (SCI); Health (HEA); Technology (TEC); Agriculture and Natural Resources; Architecture, Arts and Design (AAD); Education; Law and Social sciences (LSS); Economics (ECO); Humanities (HUM); Physical Education, Sports and Show’s Arts. In Figure 2 it is possible to see the distribution of the population and the sample by scientific field. As observed the majority of the students (population and sample) attend courses in the field of Technology (TEC-51%), Economics (ECO-23%), Heath (HEA-13%) and Law and Social Sciences (LSS-9%) and so, the analysis by scientific fields will be based on these main fields. Most of the students are undergraduate (94.6%), 5.2% are graduate students and students from technological specialization programs are 0.2% of this sample. Concerning gender, 51% of students are female and 49% male. As expected, in the population there are more students born in 1980 and after (86%) than born until 1980 (14%)7. However, the percentage of older students (over 29 years old) is large because

Differences in Internet and LMS Usage

Figure 2. Sample distribution across scientific fields

Polytechnic of Porto’s schools also host working students, which are older than ordinary students. The proportion of working students in the population is 25%, 32% of which answered the survey. The student’s sample regarding open questions analysis was defined considering all students who answered all the open questions (n=121). The sample consisted of 65 female students (53,7%) and 56 male students (46,3%). Considering age groups, there are 102 students (84,3%) born in or after 1980 and 19 (15,7%) born until 1980. Finally considering the scientific fields there are 26 students (21,5%) from Health, 52 students (42,1%) from Technology, 13 students (10,7%) from Law and Social Sciences, 27 students (22,3%) from Economics and 3 students (1,7%) that didn’t answered what was the scientific field that they belonged.

DATA ANALYSIS The data analysis was carried out on two major components: characterization of Internet usage, and the role of Internet and LMS in education. In both cases, we have started by a descriptive analysis of the sample and, then, we studied

which differences were statistically significant through hypothesis tests. Our analysis considered three students features: age, gender and scientific field. In the first two, defined only by two groups, we have used the proportion test using the Chisquare distribution in nominal variables and the Mann-Whitney test in ordinal variables. In the scientific field feature, we have decided to select only the four most representative fields, namely Health, Technology, Law and Social Sciences, and Economics. To study the differences between scientific fields, we have used the Chi-square test on nominal variables. On ordinal variables, we have applied the Kruskal-Wallis test to detect differences between the 4 fields and, when differences were found, we have applied the Mann-Whitney test between pairs of scientific fields adjusting the significance level to the number of tests performed. The open questions analysis was done using QSR NVivo 7. To analyze the significant differences found in NVivo7, we applied the proportion hypothesis test in age and gender, and the Chisquare test in the scientific field.

253

Differences in Internet and LMS Usage

Internet Usage Due to the nature of data this section was divided into two sub-sections: Type of Internet usage and Communication tools usage.

Type of Internet usage The majority of students access the Internet at home (91%) and at school (80%). Only 24% say they have mobile Internet access. Most students access the Internet several times a day (65%), 32% connect daily and the remaining 3% connect monthly, weekly and even more rarely. On average, 51% of students are connected to the Internet between 1 and 3 hours per day (Rodrigues et al., 2010). We found that male students stay connected for longer periods of time (p=0.013) (Babo, Lopes, Rodrigues, Pinto & Oliveira 2010a). Students born in 1980 and afterwards stay connected for longer periods of time (w=103902, p=0.000). An explanation for this factor could be the professional duties of the older students, which prevent them to a more lasting usage (Babo, Lopes, Rodrigues, Pinto, Queirós & Oliveira, 2010b). Analyzing by scientific field we found that students enrolled in Technology degrees stay connected for longer periods of time than Health (w=64028, p=0.019) and Economics (w=143266, p=0.000) students. On average, 48% of the time spent on the Internet is for personal leisure (the mode is 50%). The most used Internet tools are email (95%), search engines (92%), instant messaging (58%) and social networks (52%). Students’ gender is related to the usage of some Internet tools. Male students statistically use more instant messaging (p=0.007), forums (p=0.009), games (p=0.000) and wikis (p=0.000) and female students, statistically, use more social networks (p=0.000) and search engines (p=0.039) (Babo et al., 2010b). Students’ generation is also related with the use of Internet tools. Younger students use more email (p=0.000), search engines (p=0.003),

254

forums (p=0.000), instant messaging (p=0.000), social networks (p=0.000), wikis (p=0.000), games (p=0.000). Only blogs are used similarly by male and female in both students’ generations (Babo et al., 2010a). Scientific field is also statistically related with the usage of email (χ2=52.12, p=0.000), search engines (χ2=33.67, p=0.000), forums (χ2=9.13, p=0.000), blogs (χ2=9.13, p=0.028), instant messaging (χ2=38.61, p=0.000), social networks (χ2=48.41, p=0.000), wikis (χ2=54.70, p=0.000) and games (χ2=14.77, p=0.002). A descriptive analysis shows that health students tend to use more search engines, instant messaging and blogs than students from other scientific fields. Technology students use more email, forums, games and wikis and students from Law and Social Sciences use more social networks than students from other scientific fields. According to Table 1, presented below, the main reasons for Internet usage are research concerning class works/study (94%), accessing documents in Moodle or another LMS (77%) and email exchange (77%) (Rodrigues et al., 2010). An overview of Internet usage’s main reasons by gender (Babo et al., 2010b) shows that female students statistically use the Internet more than male students to research concerning class work/study, email, connect with friends and access friends’ social networks. Male students use theZ Internet more than female students to read newspapers, magazines and portals, download music or films, play games, use forums (post or read) and shopping. In general, it is noticeable that female students use the Internet to study, and to socialize with others. While male students are more engaged in leisure activities and enrolled in activities by themselves that do not depend on other people (Babo et al., 2010b). With the two independent samples t-test, it was also verified that youngest students use more Internet based tools (p=0.000) than students born before 1980 (Babo et al., 2010a).

Differences in Internet and LMS Usage

Table 1. Gender, age and scientific field differences regarding Internet usage’s main reasons Gender

Internet Usage’s Main Reasons

Total

Researching concerning class works/ study

94%

Accessing document in Moodle or another LMS

77%

Email

77%

Connecting with friends

72%

Reading newspapers, magazines and portals

52%

Downloading music or films

50%

Searching info (other than school issues)

47%

Sharing information (documents, music, films, etc.)

28%

Accessing friends’ social networks

24%

Gaming

19%

Using forums (post or read)

17%

Shopping

12%

Using Blogs (post or visit others)

9%

Searching new friends/aquaintances

6%

Age

F

M

96%

90%

Older (Born<1980)

Scientific Fields

Younger Born >1980)

HEA

TEC

LSS

ECO

97%

92%

97%

93%

p≅0.000

p≅0.008 85%

73%

65%

80%

p≅0.001 82%

73%

87%

p≅0.000 78%

66%

p≅0.000 43%

62%

p≅0.000 44%

57%

70%

83%

72%

63%

84%

70%

35%

56%

p≅0.000

p≅0.000

76%

64%

58%

49%

p≅0.000 55%

52%

57 %

p≅0.000 56%

80%

p≅0.000 49%

17%

80% p≅0.001

p≅0.000 68%

72%

p≅0.000

p≅0.000 8%

82%

43 %

40 %

p≅0.000 42%

p≅0.000 25%

3% p≅0.000 30%

18%

14%

24%

6%

17%

14%

22%

p≅0.000 7%

27%

p≅0.000

27%

14%

18%

10%

11%

p≅0.000 9%

2%

39%

p≅0.019 15%

2%

p≅0.000

9%

17%

p≅0.000 17%

1%

p≅0.000 7%

27%

p≅0.000

p≅0.000 31%

21%

2%

p≅0.000

p≅0.000

4%

16%

8%

12%

p≅0.000

10%

p=0.028

(The absence of p-values means there are no significant differences between the groups.)

Data overview shows that both generations of students use the Internet for different reasons (Babo et al., 2010a). Younger students use it for contacting with friends, download music and movies, share information (documents, music, movies, etc.), visit friend’s webpage in social

networks, play games, and participate in forums. Older students use the Internet mainly for visiting the eLearning platform to look for new documents, email, read the news in newspapers, magazines and portals, to get information about various themes not connected with their studies,

255

Differences in Internet and LMS Usage

and to shop. Non working students, namely, the younger students at IPP, do not shop (shop in a lesser extent) because they cannot afford it (Kwak, Fox & Zinkhan, 2002). In general this descriptive analysis shows that, on one hand, younger students are more enrolled in Web 2.0’s activities like socialization and sharing. On the other hand, students born before 1980 engage in activities typical of the Web 1.0 using the Internet as a mean of getting information, more than to interact with others, to share information, namely personal information (Babo et al., 2010a; Kennedy, 2009). Through Table 1 we can see that almost all reasons for Internet use are statistically associated with the scientific field. The exceptions are: searching for information not related to school subjects, sharing information, using blogs and searching for new friends. The trend shows that health students use the Internet more than other students to: research concerning class work/study, email and connect with friends. Technology students use more the Internet to download music and movies, play games, use forums and shopping. Law and Social Sciences students use more the Internet to: research concerning class work/study, visit the eLearning platform to look for new documents, read the news in newspapers, magazines and portals and accessing friends’ social networks.

Communication Tools Usage In general, 74% of Polytechnic of Porto students have a profile in a social network. Of these, 91% have a profile in HI58, followed by Facebook (23%). About 33% of the students use social networks to make new friends. Nevertheless, there is a statistically significant (p=0.002) difference between gender: 78% of the female students have a profile in a social network, while only 70% of the male students have such a profile (Babo et al., 2010b). Regarding generation there are also differences: 77% of the younger students have a profile in a social network. The importance

256

of social networking is minor among the older generation: only 54% have a profile in a social network. Finally, the possession of a profile in a social network is also related with the scientific field of the student (χ2=18.67, df=3, p=0.000). More than 74% of the students from Health, Law and Social Sciences and Economics degrees have a profile in a social network while 69% of Technology students’ have such a profile. Table 2 shows the most engaged activities in social networks by the students: viewing photos (47%), knowing what my friends are doing/what is happening (36%) and commenting (35%) are the most reported activities in social networks. An overview of the data (Babo et al., 2010b) and Table 2 above, shows that female students are, with statistical evidence, the ones most engaged in activities developed in social networks: view photos, try to find out what their contacts are doing, make comments, talk with other people and share information (photos, videos, music). Male students admit more than female students that they meet new people, searching for new friends and acquaintances. Either, male students are meeting other male students or, female students are less frank about their online activity, another explanation could be, female students are not actively trying to meet new people, thus do not acknowledge this activity. Among younger students, it is observed a more frequent use of social networks. In social networks, the younger students are statistically different from the older in what regards the activities: see photos (p≅0.000), find what their contacts are doing (p≅0.000), do comments (p≅0.000), talk with other people (p=0.003) and, share information (p≅0.000): photos, films, music. Older students statistically update their CV (p≅0.000) more than the younger generation and, search for interesting information other than school subjects (p≅0.000). Through Table 2 we can see that almost all the activities in social networks use are statistically associated with the scientific field. The exceptions are: search for information not related to school

Differences in Internet and LMS Usage

Table 2.Gender, age and scientific field differences regarding most engaged activities in social networks Gender

Activities most developed in Social Networks

Total

Viewing photos

47%

Knowing what my friends are doing/what is happening

36%

Commenting

35%

Age

F

M

Older (Born<1980)

Younger Born >1980)

HEA

TEC

LSS

ECO

54%

39%

22%

51%

60%

42%

55%

45%

p≅0.000 41%

31%

26%

77%

56%

p≅0.000 44%

33%

Chatting with other people

31%

Sharing information (documents, photos, videos, music)

24%

Meeting new people (search new friends/ aquaintences)

15%

Informing my friends what I am doing

15%

Searching for information about companines

9%

Updating CV

8%

Setting personal meeting with network contacs

3%

Sending messages

0%

p≅0.000

p≅0.000 44%

32%

p≅0.000

p≅0.000

44%

35%

p≅0.000 39%

16%

49%

28%

p≅0.000 18%

Finding interesting things(search information – other than school issues)

Scientific Fields

43%

34%

p≅0.000 8%

p≅0.000 35%

27%

29%

28%

p≅0.000 30%

19%

19%

39%

p≅0.004 26%

14%

p≅0.000 10%

34%

34%

20%

p≅0.000

p≅0.000

22%

p≅0.000 32%

24%

33%

10%

17%

p≅0.008

8%

19%

p≅0.004 32%

28% p≅0.000

17%

7% p≅0.000

5%

8%

12%

10%

p≅0.040 34%

27% p≅0.033

The absence of p-values means there are no significant differences between the groups

subjects, search for information about companies, setting personal meetings and sending messages. The trend shows that health students use more the social networks to: view photos, try to find out what their contacts are doing, make comments and share information. The Law and Social Sciences students use the social networks to: try to find out what their contacts are doing and update their CV. Finally, Economics students use the social networks to talk with other people and try to meet new friends.

In order to connect synchronously, 98% of the students use MSN while Skype and Google Talk have small shares (15% and 11%, respectively). However, male students are more diverse in their tools’ choice (Google Talk, p=0.001 and Skype, p=0.021). Females are much more loyal to MSN hardly using any other synchronous tool (p=0.028). Data shows (Babo et al., 2010b) that 28% of the students use instant messaging several times a day and 34% use it at least daily. Male students are more frequent users (p=0.004).

257

Differences in Internet and LMS Usage

Instant messaging is used by 28% of the students several times a day and 34% use it at least daily. In what refers to differences between generations, younger students are more frequent users (p=0.000). Email is used more than instant messaging. According to data (Babo et al., 2010b), 52% of students use email several times a day and 39% use it at least once a day. Even though there are not statistical gender differences regarding whether students use email, when it comes to the frequency of email usage male students use email more often (p≅0.000): 59% of male students use it several times a day against 44% of the female students. In terms of scientific field we found that, regarding to the use of email, there are some statistical differences between the several fields considered. Health students use it less than Technology students (w=5718, p=0.000) and less than Law and Social Sciences students (w=13623, p=0.001). Economics students use it less than Technology students (w=116093, p=0.000) and less than Law and Social Sciences students (w=27952, p=0.001). From the data collected we can state that the majority of students use the email several times a day with special importance to Technology (58,3%) and Law and Social Sciences (57,3%) students. Unlike the observed with the results of Internet’s session time, social networks and instant messaging usage, older students do a larger use of email (71% use it several times a day against 48% of the youngest ones; p≅0.000). This might be happening due to the fact that older students usually are also workers and can have email connected all day allowing them to check it several times a day.

Role of Internet and LMS in Education In this section we will analyze the use of the Internet and LMS and their role in education. A descriptive analysis of the sample will be made and significant statistical differences in gender, age

258

group and scientific field will be reported. Table 3 analysis the statistical differences encountered by gender and age group in: a) the students’ access LMS platform; b) Students’ evaluation of usability of the LMS platform and c) Students’ evaluation of the importance of the LMS platforms in their learning process. Analyzing Table 3, by gender, male students visit more often the LMS platform than female students (65% of male students visit the LMS platform in a daily basis against 57% of female students), a difference that is statistically significant (p=0.006). It is important to notice there are no statistical differences between the two age groups regarding the frequency of visits to LMS platforms. When it comes to usability it is important to say that no gender or age statistically significant differences were found related to the difficulty in the use of LMS platforms. Analyzing the importance of LMS platforms in students’ learning process it is important to refer that more than 77% of the students consider LMS platforms to have an important role in their learning process. If we associate this result with the fact that most students access LMS platforms, at least once a day, in order to help students’ learning teachers should improve and promote the use of these platforms. In terms of age group we found that older students feel that this kind of platforms are more important to their learning process than younger students, a difference that is statistically significant (p=0.008). This result is consistent with the fact that most of the older students are employed, attend fewer classes and need the LMS platforms to be updated, inquire teachers and colleagues and access documents on the contents covered in classes. No differences were found in terms of gender. The same analysis was made concerning scientific fields (Table 4). Through Table 4 we can see/observe that the major differences encountered are between Health students and students from other scientific fields.

Differences in Internet and LMS Usage

Table 3. Statistical differences by gender and age group regarding students’ access to LMS platform, students’ evaluation of LMS usability’s platform, and the importance of those platforms to students’ learning process Access

< p-0.006

Usability

NDF

Importance

NDF

Born >1980 Born<1980

Female

Male Access

NDF

Usability

NDF

Importance

> P=0.008

(NDF equals Non Differences Found. The left operands of the signs are the row header. The right operands are the column header.)

In terms of access to LMS platforms Health students have the lowest access to these platforms what is consistent with the fact that Health students have more difficulties when using LMS platforms (only 63% of them say that the use of the LMS platform is easy against more that 78% of the other scientific fields). This last result is statistically significant since the Wilcoxon test shows Health students find it harder to use than Technology students (w=51506, p=0.000), than Law and

Social Sciences students (w=11299, p=0.000) and Economics students (w= 30598, p=0.000). Finally, and in concordance with the previous results, it is important to say that only 59% of Health students say the LMS platform is important to the learning process against more than 79% for students from other scientific fields. This result is statistically significant since the Wilcoxon test shows Health students find it less important than Technology students (w=47211, p=0.000), Law

Table 4. Statistical differences by scientific field in students’ access to LMS platforms and evaluation concerning the usability of LMS platforms and the importance of those platforms learning process HEA Health

Technology

Law & Social Sciences

TEC

LSS

ECO

Access

< w=43761 p=0.000

< w=11348 p=0.000

< w=26457 p=0.000

Usability

< w=51506 p=0.000

< w=11299 p=0.000

< w=30598 p=0.000

Importance

< w=47211 p=0.000

< w=9300 p=0.000

< w=26772 p=0.000

Access

NDF

NDF

Usability

NDF

NDF

Importance

< w=35959 p=0.004

NDF

Access

NDF

Usability

NDF

Importance

> w=26923 P=0.005

259

Differences in Internet and LMS Usage

and Social Sciences students (w=9300, p=0.000) and Economics students (w=26772, p=0.000). According to the students in this study, the tools most used by teachers for educational purposes are: email (88%), Search Engines (36%) and Forums (23%), as illustrated in Table 5. Analyzing the students’ response we can find significant differences in their perceptions about the tools used by their teachers. Only the use of search engines by teachers is not associated with the scientific field. The proportion of students stating that the teachers use the email is larger in health students. This can explain why health students are not so supporters of LMS platforms as students from other scientific fields. As stated by students, search engines are the second most used tool by their teachers. Considering gender, female students, have the perception that their teachers use search engines more than male students (40% against 33%, p=0.008). Tools such as forums (25% against 21%, χ2=3.2482, df=1, p=0.036), blogs (9% against 6%, p=0.032) and Wikis (11% against 7%, p=0.008) are, in male students’ opinion, more used by their teachers.

The trend shows that forums’ are more used by Technology students’ teachers and wikis and blogs are more used by Law and Social Sciences field teachers. These results may be related to the tools that students themselves say they use more frequently: email, search engines and Forums (Babo et al., 2010b). No statistical differences were found in age groups. Regarding the open question where students were asked about what can be done to facilitate or enhance their learning regarding LMS platforms, five dimensions emerged from the qualitative analysis: a) to be used by all teachers (27%); b) update the content available (25%); c) to use a wide variety of functionalities (16%); d) to be used to increase the interaction between teachers and students (9%); e) make easier their access and use (9%) (Table 6). Describing results by gender we can state that female students find that the two most important actions to be performed in the LMS platform to facilitate their learning process are its use by all teachers (40%) and content’s update (27%). For male students the two most important actions are

Table 5. Internet’s tools used by teachers for school purposes Gender

Internet tools used by teachers

Total

Email

88%

Search engines

36%

forums

23%

wikis

9%

F

M

89%

87%

86%

HEA

88%

95%

LSS

ECO

89%

80%

83%

33%

34%

37%

43%

36%

37%

32%

21%

23%

7%

28%

20%

26%

χ2=5.71,df=1 P=0.008 25%

χ =3.25,df=1 p=0.036 2

7%

11%

χ2=45.30,df=3,p=0.000 7%

9%

1%

χ2=5.90,df=1 P=0.008 6%

9%

13%

18%

4%

χ2=57.09,df=1,p=0.000 11%

7%

χ =3.43,df=1 p=0.031 2

(The absence of p-values means there are no significant differences between the groups.)

260

TEC

χ =27.85, df=3, p=0.000

21%

8%

Scientific Fields

Younger Born >1980)

2

40%

blogs

Age Older (Born<1980)

2%

9%

10%

χ2=13.44,df=3,p=0.004

8%

Differences in Internet and LMS Usage

content’s update (31%) and a wider variety of functionalities (29%). Analyzing by age we can state that for older students it is important to enlarge the variety of functionalities used in LMS platforms (33%) and to use them to increase the interaction between students and teachers (20%). Younger students think it is important to spread the use of LMS platforms to every teacher (34%). We would like to note that none of the older students refers difficulties to access or to use LMS platforms. Finally, analyzing by scientific fields we can state that “to be used by all teachers” is the most important action to be taken to almost all scientific fields (Health, Law and Social Sciences and Economics). The exception is the Technology field degree students’. For Technology field students’ the most important action is “update the contents available”. This action also was considered important to Health and Economics. Statistical analysis of these five dimensions and the demographic variables considered in this study there weren’t found any differences with the exception for the first dimension “To be used by all teachers”. This dimension was considered

more important to female students than male students and has statistical relevance (χ2=7.24, df=1, p=0.004). For all the other dimensions considered and demographic variables there weren’t found statistical differences and this means even if there are some tendencies, all of students have similar opinion about what could enhance their learning regarding Moodle. Many students state that it would be important that the LMS platform was used by all teachers because it allows in a single place to have all the information related to a course and stay updated. To illustrate this opinion we transcribe an answer: …it was important that ALL teachers saw the platform as something important for the academic community because it allows students to stay updated and know the material taught in class… (S221) Students, also, think that teachers should use more LMS’s functionalities like quizzes, forums, blogs to improve the learning process and they also think it is important to promote interaction

Table 6. What measures could be used in LMS platforms to facilitate or enhance your learning? What measures could be used in LMS platforms to facilitate or enhance your learning?

Gender Total

To be used by all teachers

27%

Update the content available

25%

26%

To use a wide variety of functionalities

16%

Make their access/use easier To be used to increase the interaction between students and teachers

Age

Scientific Fields

Older (Born<1980)

Younger Born >1980)

HEA

TEC

LSS

ECO

16%

29%

31%

14%

54%

33%

23%

21%

26%

27%

25%

15%

26%

11%

21%

26%

14%

15%

17%

23%

11%

9%

8%

11%

0%

11%

15%

8%

15%

4%

9%

12%

5%

16%

8%

8%

8%

15%

8%

F

M

39%

14%

χ =7.24,df=1 p=0.004 2

(The absence of p-values means there are no significant differences between the groups.)

261

Differences in Internet and LMS Usage

between teachers and students. The next two transcripts illustrate these ideas: …Do diagnostic tests, promote the use of blogs or forums, to share questions and those questions be answered by teacher for all students… (S252) Create a kind of msn that would allow teachers to talk in real time with students in order to answer to student’s questions (S247) Finally students have the opinion that access to LMS platforms must be improved and must be easier to use. The next transcript reflects this opinion, although it doesn’t make any specific suggestion: Easier access and more precise instructions for use… (S110)

RESULTS DISCUSSION The results presented in this study are very important for two reasons. First, they show the state of Internet and LMS platforms usage for higher education learning proposes in a very important Portuguese tertiary level education institution, allowing an analysis and comparison of this important topic across countries. Second, they are important to the LMS policies to be defined at Polytechnic of Porto. Since LMS are available on the Internet it was considered important to know the Internet students’ usage habits. The data collected and analyzed in this study show the state of Internet usage and LMS perceptions at Polytechnic of Porto. From the data analysis, we can state that 91% of the students have Internet access at home and 80% at school. They also show that 90% of students’ access Internet every day. These results are in agreement with a study made by an Australian Universities

262

(Kennedy et al., 2009), in which 95,8% of the students have access to Internet. Such facts could indicate that teachers should exploit the use of teaching and learning methodologies supported on the Internet. Students that stay connected for longer periods of time are male, born until 1980 (also found by Williams and Goldberg, 2005) and studying Technology or Law and Social Sciences. Reading email and search engines are the most used Internet tools by students, furthermore, data shows that there are no significant differences between genders. Male students use more Internet tools (wikis, forums, games, instant messaging), with exception for social networks and search engines than female students. These results are, again, in line with the study of the Australian universities (Kennedy et al., 2009) in which the most used Internet tools are email, search engines and instant messaging. Concerning age gaps it is a fact that younger students use more email, search engines, forums, instant messaging, social networks, wikis, and games. Only blogs are used similarly by both students’ generations. Scientific field is related with the usage of Internet tools. Students from Health statistically use more search engines, instant messaging and Blogs than students from other scientific fields. Students from Technology use more email, forums, games and wikis than students from other scientific fields. Finally, students from Law and Social Sciences degrees use more social networks than students from other scientific fields. When students were asked about the reasons for accessing the Internet, most of them answered that the main reasons are academic. They use it to carry out research to do class work or to study. Other reasons mentioned by females are being connected with friends and access to friends’ social networks. Conversely, men prefer to use the Internet to read newspapers, download music or videos and play games. The reason for this behavior differences can be explained by psychological

Differences in Internet and LMS Usage

differences between the two genders. Women tend to exhibit a more social behavior than men, who usually have a more individual attitude. These results are in agreement with a study made by Madell and Muncer (2004). Both generations of students use the Internet for different reasons (Babo et al., 2010a). Younger students use it to contact with friends, download music and movies, share information (documents, music, movies, etc.), visit friend’s webpage in social networks, play games, and participate in forums. Hence, older students use the Internet mainly to visit the eLearning platform to look for new documents, email, read the news in newspapers, magazines and portals, to get information about various themes not connected with their studies, and to shop, since, unlike the younger students, they have the economic means that enable that attitude(Kwak et al., 2002). In general, this study’s data shows that, on one hand, younger students are more enrolled in activities of Web 2.0 like socialization and sharing. And, on the other hand, students born before 1980 engage in activities typical of the Web 1.0 using the Internet as a mean of getting information, more than to interact with others, to share information, namely personal information. This finding is consistent with several studies conducted in this area (Kennedy et al., 2009). The role of Internet and LMS in education was analyzed in five different perspectives: frequency, difficulty and importance of its usage to the students learning, the kind of functionalities were used by teachers (according to the students perceptions) and suggestions from students in order to improve LMS usage for their learning process. Differences between gender and two age groups (students born before 1980 and in 1980 or after) as well as the scientific field of their degree were analyzed. More than 60% of Polytechnic of Porto’s students checks out the LMS platform at least daily which is similar to what happens in the University of Minnesota: 59% (Walker & Jorn, 2007).

The differences found were regarding the frequency of LMS access: male students access it more frequently then female students. This finding is consistent with the fact that male students stay connected to Internet for longer periods of time (e.g. Minhsein, 2005; Misko et al., 2004; and even in Japan: Kubora et al., 2001). Next studies will allow us to see if the gender gap will decrease, as found in other international studies (e.g. Misko et al., 2004). In what comes to age differences, it was found that older students found LMS more important to their studies than younger ones. This finding goes accordingly with the increase of non-traditional students (over 24 years old) in higher education studies (Williams & Goldberg, 2005). The growth in lifelong learning has made the educational institutions to seek for flexible education delivery and support so they can better host and address these news students’ needs. Regarding scientific fields the access to the LMS platform results are in accordance with the general findings across Polytechnic of Porto except for the Heath degrees’ students. The perceptions of the LMS ease of use by Heath students may explain its lower use: generally they have more difficulty in using the LMS platform when comparing with students from other scientific fields. In general, students find it easy to use Moodle and no differences were found neither between gender and age groups. Heath degrees’ students have an issue using LMS: they use it less, they find it more difficult to and they don’t find it as important for their studies as students in other fields. These might be explained by the low use of Moodle by teachers: the less teachers use it, the less it used by students, the less they find it important and the less they train their abilities in using it, thus they find it more difficult. Regarding students’ perceptions about the tools used by their teachers the use of email is unanimous for gender, age group and for all of the scientific fields but for Heath degrees’ students. Students’ perception is that email is the most common In-

263

Differences in Internet and LMS Usage

ternet tool used by their teachers. This result can also help to explain why Health students are not so supporters of LMS platforms as other students from different scientific fields. Supporting these findings in an eLearning readiness evaluation study carried out in Polytechnic Heath School (Lopes, 2007), it was found that student’s access to computers and Internet, one of the major concerns, was due to a lack of technological infrastructures. This study allowed to identify the professors’ need of ICT training and technological support (72,4% said they had no previous experience with eLearning) and acknowledge professors interest and openness towards e-learning (e.g. 69% value lectures access in the LMS platform). The students’ perceptions of the Internet tools most used by teachers are email (88%), search engines (36%) and forums (23%). Botturi et al. (2006) study at University of Lugano also found similar frequent Internet tools used by teachers (perceived by students). Even though students have classes with the same teachers, we found gender differences in the perceptions of the teachers Internet tools usage. No difference between age groups was found. Analyzing by scientific field the forums’ are most used by the teachers of Technology degree students. Wikis and blogs are more used by Law and Social Sciences teachers. These results may be related to the tools that students themselves say they use more frequently: email, search engines and Forums (Babo et al., 2010b). In realizing that students’ perceptions about the tools used by their teachers may be influenced by their own personal experience led us to two different kinds of conclusions: empirical finding and new course of action. Students tend to say that their teachers use the tools they, themselves use. Along with data from the first part of this study characterizing students’ Internet usage we can conclude, that male students use more diversified Internet tools, than female students. Regarding the course of action we should follow in future researches, if we want to really

264

know what Internet tools teachers are using, we should find the original data from the LMS system administrator, since the perceptions of students may not be reliable. From the suggestions students made regarding what could be done to improve or enhance their learning regarding LMS platforms, emerged five dimensions: (1) to be used by all teachers, (2) update the content available, (3) to use a wider variety of functionalities, (4) to be used to increase the interaction between teachers and students and (5) make access and use easier. Suggestions from students’ point out that a lot of teachers still do not use the LMS platform and, from the ones that use them, some don’t have the (content) updated content, which implies that students would like to base their studies more on the LMS platform. These are the two features female students point out primarily, which goes in accordance with previous results found in this study, which state that female students use more Internet to study than male students. Some other suggestions, request for sophistication, saying that more functionalities should be used, including the interaction between teachers and students. Older students make this request in bigger proportion. This finding reinforces the importance that older students give to LMS platforms as a mean of learning. Male students also advocate the use of other functionalities in the LMS, which is consistent with previous findings in this study. “To be used by all teachers” is the most important action to be taken to almost all scientific fields (Health, Law and Social Sciences and Economics). The exception is Technology field, whose students’ agreed that the most important action is “update the contents available”, followed by Health and Economic students. On one hand there was demand for more sophisticated use of LMS platforms, but on the other hand, a smaller percentage of students (11%) request for more help and direction to access and use the LMS platform. These differences show different levels of LMS users abilities, which suggest

Differences in Internet and LMS Usage

that making training or a specialist available for students would be welcomed. To increase LMS usage some institutional rules should be used along to with training and support to teachers, in order to increase the use of the ones who still do not use or use it barely and to assist the advanced users to sophisticate their usage even more.

CONCLUSION The presented study focused on one of the first studies in Polytechnic of Porto about Internet and LMS usage, as well as the role of LMS in higher education as a tool to support the learning process The data collected was analyzed, considering three dimensions: gender, age and scientific fields, in order to provide a more detailed picture of the student population of Polytechnic of Porto. The results could support the idea that it is important to promote active learning strategies, across students and teachers, which improve the learning process. The results clearly suggest the implementation of methodologies and learning tools (LMS) based on the Internet, since students have acquired consistent habits in using the Internet and LMS platforms. According to the data collected and analyzed, the majority of students have skills and competences to use Internet tools and LMS facilities, in order to improve their learning process. In fact, almost all students access the Internet daily, regardless of scientific field, gender or age. However, some results could indicate the need to introduce some changes in the actual paradigm, particularly in respect of: i) the usage of Internet tools by teachers, shifting from traditional emails to collaborative platforms (Web 2.0), such wikis, blogs, and forums. A significant number of students, particularly the younger ones, use these tools daily so it suggests their ability to explore these tools in an learning environment; ii) a greater usage of the LMS platform, particularly in some scientific areas, such as health field. In fact, one

of the most addressed issues about enhance eLearning is the need to “be used by all teachers”, and “ update the content available”. It suggests the need of a more pro-active attitude by teachers concerning the LMS usage. Another important issue that we could extract from the data collected and discussed is the variety of Internet tools used by students, according to their scientific field. It is possible that the student profile (interests, demands requirements, relationships, etc.) could influence the type of tools and tasks done on the Internet. Thus, it is important to adapt eLearning strategies and learning approaches according the student scientific field and its profile. Students from scientific fields like law, technologies, health or others, have different profiles, different cultures and different approaches in their learning processes. The LMS methodologies and strategies must be centered on the student needs, according to their skills and their learning context. Following the example of the University of Minnesota (Walker & Jorn, 2007), we intend to carry out a bi-annual analysis of the habits of Internet and LMS usage among students of the Polytechnic of Porto to study its development, as well as the evolution of “new technologies” usage in teaching and learning activities. This information may have a relevant role in defining technologies planning and thus, to the success of higher education proposes. In future studies, we aim to study the habits of LMS usage for teaching: what Internet tools, LMS activities and type of learning objects are available to promote learning and interaction among teachers and their students. In future studies, even though “what” is being used is always important, it is our plan to focus on the “how” LMS are being used and perceived by students as well as teachers. With all this data, it will be possible to establish orientation guidelines for the use of technologies in IPP and Higher Education institutions, in general.

265

Differences in Internet and LMS Usage

REFERENCES Andronico, A., Carbonaro, A., Colazzo, L., Molinari, A., Ronchetti, M., & Trifonova, A. (2004). Designing models and services for learning management systems in mobile settings . In Mobile and Ubiquitious Information Access (Vol. 2954, pp. 524–532). Berlin-Heidelberg, Germany: SpringerVerlag. doi:10.1007/978-3-540-24641-1_8 Babo, R., & Azevedo, A. (2009). Learning management systems usage on higher education institutions. In Proceedings of 13th IBIMA Conference - Knowledge Management and Innovation in Advancing Economies: Analyses & Solutions (pp. 883 – 889). Babo, R., Lopes, C., Rodrigues, A., Pinto, M., Queirós, R., & Oliveira, P. (2010a). Gender differences in Internet usage habits - A case study in higher education. 14th International Business Information Management Association, Istanbul, Turkey.

Conole, G., de Laat, M., Dillon, T., & Darby, J. (2006). JISC LXP: Student experiences of technologies: Final Report: Joint Information Systems Committee. Retrieved October 7, 2010, from http://www.jisc.ac.uk/. Georgouli, K., Skalkidis, I., & Gerreiro, P. (2008). A framework for adopting LMS to introduce e-learning in a traditional course. Journal of Educational Technology & Society, , 227–240. Gras, M. (2005). Internet usage and digital divide among Spanish university students. Revista Media e Jornalismo - A Europa e os Media:14(8)(1). Harasim, L. (2006). History of e-learning: Shift happened. The international handbook of virtual learning environment. Berlin-Heidelberg, Germany: Springer-Verlag. Harman, L., & Koohang, A. (2007). Learning objects: Standards, metadata, repositories, and LCMS. Information Sciense Institute, Informing Sciense.

Babo, R., Lopes, C., Rodrigues, A., Pinto, M., Queirós, R., & Oliveira, P. (2010 b). Comparison of Internet usage habits in two generations of higher education students - A case study. 2th International Conference on Computer Supported Education, Valencia, Spain.

Janossy, J., & Hover, T. (2008). Proposed model for evaluating C/LMS usage. In K. McFerrin et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2008 (pp. 2979-2986). Chesapeake, VA: AACE.

Botturi, L., Cantoni, L., & Tardini, S. (2006). Introducing a Moodle LMS in higher education: The e-courses experience in Ticino. Retrieved on June 21, 2010, from http://www.elearninglab.org/ wp-content/ uploads/ 2010/ 04/ jelks_ecourses_ paper.pdf

Kennedy, G., Dalgarno, B., Sturt, C., Bennett, S., Gray, K., Waycott, J., & Judd, T. …Chang, R. (2009). Educating the Net generation. A handbook of findings for practice and policy. Australia: Australian Learning and Teaching Council.

Burlea, A. (2008). The complexity of an e-learning system: A paradigm for the human factor, The Inter-Networked World: ISD theory, practice and education (Vol. 2, pp. 267–278). New York, NY: Springer-Verlag.

266

Kubora, K., Terashima, K., Nakahashi, Y., & Morioka, H. (2008). Analyzing learning environments of distance education in Japan, distance learning and the Internet, Tokyo, Japan. Kwak, H., Fox, R. & Zinkhan, G. (2002). What products can be successfully promoted and sold via the Internet? Journal of Advertising Research, Jan.-Feb.

Differences in Internet and LMS Usage

Landsberger, J. (2004). Thoughts on convergence in instructional settings. TechTrends, 48, 6–8. doi:10.1007/BF02763348 Leal, J. & Queirós, R. (2010). From e-learning systems to specialised e-learning services. Chapter of EduJudge project book - A new learning paradigm: Competition supported by technology. Lopes, C. (2007). Evaluating e-learning readiness in a health sciences higher education institution. In M. Baptista Nunes & M. McPherson (Eds.), Proceedings of the IADIS International Conference e-Learning 2007 (vol. 1, pp. 59-67). Madell, D., & Muncer, S. (2004). Gender differences in the use of the Internet by English secondary school children. Social Psychology of Education, 7, 229–251. doi:10.1023/ B:SPOE.0000018552.85903.4d Mayadas, F. (2007). Online education: Worldwide and moving quickly. In Technology- Enabled Education (pp. 57-60), Proceeding of the 2007 MIT LINC Conference, Dubai, UAE. Minhsein, H. (2005). Integrating Internet into traditional education: A practical study. Misko, J., Choi, J., & Lee, I. (2004). E-learning in Australia and Korea: Learning from practice. Korea Research Institute for Vocational Education & Training. Retrieved on June 19, 2010, from http://www.ncver.edu.au/ research/ core/ cp0306.pdf Monsakul, J. (2007). Learning management systems in higher education: A review from faculty perspective, 4th International Conference on Elearning for Knowledge-Based Society, Bangkok, Thailand. Moodle. (2010). Current Moodle statistics, Moodle.org. Retrieved on May 27, 2010, from http://moodle.org/stats.

Nachmias, R., Mioduser, D., & Shemla, A. (2001). Information and Communication Technologies usage by students in an Israeli high school: Equity, gender, and inside/outside school learning issues. Education and Information Technologies, 6(1), 43–53. doi:10.1023/A:1011367212148 NYU. (2010). New York University. Retrieved from http://www.nyu.edu/ Oliveira, P., Oliveira, C., Souza, F., & Costa, N. (2006). Teaching stategies to promote active learning in higher education: Current develepments in technology-assisted education, FORMATEX 2006, (pp. 636-640). Retrieved on September7, 2010, from http://www.formatex.org/ micte2006/ pdf/ 636-640.pdf Prensky, M. (2001). Digital Natives, Digital Immigrants. MCB University Press. Rodrigues, A., Pinto, M., & Queirós, R. (2010). Grau de Preparação e Utilização do E-learning no Ensino Superior. Estudo de Caso na ESEIGIPPorto. Setúbal, Portugal: Conferência Investigação e Intervenção em Recursos Humanos. Simões, L. & Gouveia, L. (2009). Geração Net, Web 2.0 e Ensino Superior. Cadernos de Estudos Mediáticos, 21-32. Sink, C., Sink, M., Stob, J., & Taniguchi, K. (2008). Further evidence of gender differences in high school–Level computer literacy - . Chance, 21(1), 49–53. doi:10.1007/s00144-008-0011-9 Tapscott, D. (1999). Growing up digital: the rise of the next generation. New York, NY: McGraw-Hill. Tastle, J., White, A., & Shackleton, P. (2005). E-learning in higher education: The challenge, effort, and return of investment. International Journal on E-Learning, 241–251.

267

Differences in Internet and LMS Usage

Uzunboylu, H., Ozdamli, F., & Ozcinar, Z. (2006). An evaluation of open source learning management systems according to learners tools. Online Submission. Retrieved December 28, 2010, from http://www.eric.ed.gov/ ERICWebPortal/ contentdelivery/ servlet/ ERICServlet?accno =ED494265 Walker, J. & Jorn, L. (2007) Net generation students at the University of Minnesota, Twin Cities Student Educational Technology Survey 2007. Williams, J., & Goldberg, M. (2005). The evolution of e-learning. [Global.]. Universitas, 21.

ADDITIONAL READING Al-Smadi & Gutl. (2010). SOA-based architecture for a generic and flexible e-assessment system. In EDUCON’10. Alario, C., & Wilson, S. (2010). Comparison of the main alternatives to the integration of external tools in different platforms, ICERI2010 Proceedings, (pp. 3466-3476). Apostolopoulos, T. K., & Kefala, A. (2003). An e-learning service management architecture. In Proceedings of the 3rd IEEE International Conference on Advanced Learning Technologies (pp. 140-144), Athens, Greece. Bohl, O., Scheuhase, J., Sengler, R., & Winand, U. (2002). The shareable content object reference model (SCORM)-A critical review, Proceedings of the International Conference on Computers in Education (950-951). Bryden, A. (n. d.). Open and global standards for achieving an inclusive information society. Burlea, A. (2008). The complexity of an e-learning system: A paradigm for the human factor: The Inter-Networked world - ISD Theory, Practice and Education, 2, 267-278. New York, NY: Springer-Verlag.

268

Casella, G., Costagliola, G., Ferrucci, F., Polese, G., & Scanniello, G. (2007). A SCORM thin client architecture for e-learning systems based on Web services. [Hershey, PA: IDEA Group Publishing.]. International Journal of Distance Education Technologies, 5(1), 13–30. doi:10.4018/ jdet.2007010103 Dagger, D., O’Connor, A., Lawless, S., Walsh, E., & Wade, V. (2007). Service-oriented e-learning platforms: From monolithic systems to flexible services . IEEE Internet Computing, 11(3), 28–35. doi:10.1109/MIC.2007.70 Davis, B., Carmean, C., & Wagner, E. D. (2009). The evolution of the LMS: From management to learning - Deep analysis of trends shaping the future of e-learning. Sage Road Solutions. LLC. Donello, J. (2002). Theory & practice: Learning content management systems. Ellis, R. K. (2009). Field guide to learning management systems. ASTD Learning Circuits. Friesen, N. (2005). Interoperability and learning objects: An overview of e-learning standardization. Interdisciplinary Journal of Knowledge and Learning Objects. Hall, B. (2003). Learning management systems and learning content management systems demystified. Harasim, L. (2006). History of e-learning: Shift happened - The international handbook of virtual learning environments. Germany: Springer. Harman, K., & Koohang, A. (2007). Learning objects: Standards, metadata, repositories, and LCMS - Informing Science Institute. Edição de Informing Science. Hatala, M., Richards, G., Eap, T., & WilLMS, J. (n. d.). The interoperability of learning object repositories and services: Standards, implementations and lessons learned. Proceedings of the 13th international World Wide.

Differences in Internet and LMS Usage

Holden, C. (2004). What we mean when we say “repositories.” User expectations of repository systems. In: Academic ADL Co-Lab. JORUM team (2006). E-learning repository systems research watch. Retrieved December 13, 2010, from http://www.jorum.ac.uk/ docs/ pdf/ Repository_ Watch_ final_ 05012006.pdf. Kennedy, G., Dalgarno, B., Sturt, C., Bennett, S., Gray, K., Waycott, J., & Chang, R. (2009). Educating the Net generation. A handbook of findings for practice and policy. Australia: Australian Learning and Teaching Council. Martínez, J. Á., & Navarra, P. L. (2007). Content interoperability on e-learning platforms: Standardization, digital libraries, and knowledge management. Revista da Universidad y Sociedad del Conocimiento. Nichani, M. (2009). LCMS = LMS + CMS [RLOs] – How does this affect the learner? The instructional designer? Retrieved from http:// www.elearningpsot.com/articles/archives/lcms_ LMS_cms_rlos. Rehak, D., & Mason, R. (2003). Engaging with the learning object economy . In Littlejohn, A. (Ed.), Reusing online resources: A sustainable approach to e-learning (pp. 22–30). London, UK: KoganPage. Tastle, J., White, A. & Shackleton, P. (2005). E-learning in higher education: The challenge, effort, and return of investment, International Journal on ELearning. Walker, J. & Jorn, L. (2007). Net generation students at the University of Minnesota. Twin Cities Student Educational Technology Survey 2007. Williams, J., & Goldberg, M. (2005). The evolution of e-learning. [Global.]. Universitas, , 21.

KEY TERMS AND DEFINITIONS eLearning: A new form of learning based on technology. A model of teaching and learning based on the online environment, leveraging the capabilities of the Internet for communication and content distribution LMS: System used mainly by teachers and students to manage the teaching/learning process. Nowadays these type of systems occupy a central role in the eLearning realm coordinating and integrating a set of features by third-party systems. LO (Learning Objects): Units of instructional content that can be used, and most of all reused, on web based eLearning systems. Usually it’s a set of files including a XML manifest wrapped with resources and packaged in a ZIP file. Metadata: Data about data. Metadata will allow systems to index, search and retrieve files based on semantic data. Net Generation: For the majority of the authors the “Net generation” are all the people born after 1980. These people access to new technologies as they have always existed showing higher capacity of use and self-efficacy. Web 2.0: A new Web paradigm enhancing the bidirectional communication between client (using a web browser) and web servers. This new paradigm augments the capacity for the user to product, collaborate, share and recommend using Web 2.0 tools. Web 2.0 Tools: Tools that proliferate on the Web that enhance the production, collaboration and sharing of documents. Some examples of these tools are the Wikis, Blogs and Social Networks.

ENDNOTES 3 4 1 2

http://www.eseig.ipp.pt http://www.eseig.ipp.pt/ http://www.estsp.ipp.pt/ http://www2.estgf.ipp.pt/

269

Differences in Internet and LMS Usage

6 7 8 5

270

http://www.iscap.ipp.pt/ http://www.isep.ipp.pt/ These numbers do not include ESTSP. Hi5 is a social networking website (http:// hi5.com/) very popular among Portuguese young people. If the study was conducted

in 2010, in our opinion, Facebook would be the most used social network.

271

Chapter 13

LMS Adoption at the University of Genova: Ten Years After Maura Cerioli University of Genova, Italy Marina Ribaudo University of Genova, Italy Marina Rui University of Genova, Italy

ABSTRACT The last two decades have seen the spread of LMS among schools, universities, and companies to augment the traditional teaching process with ICT and network technologies. This chapter presents the process leading to the adoption of a Moodle based LMS at the University of Genova in the last decade. By analyzing the data collected from the LMS logs and from questionnaires proposed both to students and teachers, we found out that the needs of the stakeholders are largely limited to resource sharing and organizational support, satisfactorily provided by the current service. Further improvements could be achieved by the introduction of a policy encouraging or forcing the teachers to use the provided LMS. A project on instructional design and, as a case study, the evolution of some of the courses involved in it are also presented. Though the redesign of such courses has improved their results, the impact on the overall organization of the degree program has been negative. We infer that this is due to the excessive freedom the students enjoy in taking their exams in Italy.

INTRODUCTION The advent of the Web in the mid-1990s has dramatically changed the way people communicate, look for information, produce and share textual and multimedia content, access to (online) services,

buy goods, etc. The Web – the largest cognitive artefact ever built – offers an easy access to data, their sharing, and collaborative elaboration, and this new opportunity has also an important impact on education. Indeed, the different forms of communication, cooperation, and knowledge building, which are central in modern theories on education,

DOI: 10.4018/978-1-60960-884-2.ch013

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

LMS Adoption at the University of Genova

are not anymore limited to classroom activities. But, they can be moved online thanks to an easy access to network technologies. To prompt these new forms of interaction, in the last two decades, schools, universities, and companies have started offering LMS (Learning Management Systems) to the different actors involved in education. The adoption of LMS at the university level may follow two approaches, most often with the early adopters evolving from the first to the next. From the one hand, single departments or degree programs may develop or customize their own LMS, offering students and teachers a supplement to the traditional educational process. This approach leads in time to a plethora of different systems coexisting in the same institution, often using different technologies, maintained by separate groups of technical staff; thus, it is hardly scalable. This was, indeed, the situation for the case study of the University of Genova, up to 2004. On the other hand, the adoption and management of a LMS may be centralized for all the substructures. At the University of Genova, such a centralized service, called AulaWeb, was introduced in the academic year 2004/05 and is currently supporting education for all the Faculties of the University, as this chapter will illustrate. Measuring the achievements of an institution in implementing technology enhanced learning, being completely online or a blend of face-to-face and Web-based activities, is difficult. Indirect and partial information may be obtained by a systematic analysis of LMS usage. Indeed, if the activities are taken online, the kind of tools used and the interactions between students and system give a rough measure of involvement. Unfortunately, according to (Janossy, 2008), even quantitative analyses of LMS usage are generally difficult to obtain, since those available rely on incomplete log data or ad hoc surveys. One of the major contributions of this chapter is presenting a large amount of data about the usage of LMS across all the Faculties of the University of Genova. Janossy (2008) proposes a five levels

272

model to measure LMS usage. The five levels span from Level 0, corresponding to no use of the LMS, to Level 4 in which students’ activities are recorded and made accessible to all students for review, or made available in real time to remote participants. Surprisingly, this classification does not include collaborative group activities, which indeed started to appear in the recent years. In this metric, the University of Genova is at Level 3. In addition to course syllabus and reading materials (Level 1), assignments and assessments (Level 2), the students also find forums for asynchronous communication and online tests to evaluate their preparation, although with different numbers depending on their curricula. After a short analysis of background literature in Section 2, this chapter describes in Section 3 the adoption of the central LMS software platform at the University of Genova, from the early experiences to the present. Data extracted from the log files of the LMS are presented together with a project, focussing on instructional design and ICT supported learning. In Section 4 the findings obtained by two surveys administered respectively to faculty members and students are discussed. Finally, Section 5 presents conclusions and future research directions.

BACKGROUND Learning Management Systems have become an important resource for Higher Education Institutions willing to modernise their curricula and teaching methods as well as to widen their audience by attracting new students, geographically remote or adults, who could not otherwise attend traditional lectures. This is witnessed by the rich literature on e-learning, ranging from the proposal of different pedagogical models, to the description of LMS technical development, adoption, qualitative and quantitative usage, to the analysis of success case studies at the school, university or company levels.

LMS Adoption at the University of Genova

Although researchers in the field of education claim that technology enhanced learning should not focus on technologies but on learners, this does not necessarily always happen. As pointed out, for example, in (McGee & Diaz, 2007), “technologies are often selected for arbitrary reasons that have little to do with teaching and learning or the students they serve” (p. 38). Research by Dror (2008) supports the same claim: It is naïve and would be a mistake to think, let alone take for granted, that using technology per se will enhance learning. In fact, using technology often hinders learning because it does not match and fit well the cognitive system. (p. 218) When the traditional learning material is simply transformed into digital form and presented to the learners, they remain passive and not really engaged in the process of knowledge acquisition. In contrast, when learners are active and motivated, participating and interacting, then learning is maximised. In this case study, we will see how at the University of Genova the homogeneity of technological support is not entailing similarity of learning models across the different programs. Malikowski, Thompson and Theis (2007) proposed a model to study LMS not only from a technological point of view but also considering the pedagogical aspects. The authors select five categories - transmitting course content; evaluating students; evaluating courses and instructors; creating class discussions; and creating computer based instruction – and describe them introducing the LMS features for each category but also providing recommendations from an educational point of view. They compare the use of LMS (called CMS in their work) in different American institutions and summarize their results as follows: The most prominent use of a CMS is to transmit information to students. A category showing moderate use is creating class discussions, primarily asynchronous discussions. Tools in the category

of evaluating students are receiving low to moderate use, particularly with quizzes. Tools in other categories are receiving very little use. Considering the research described in these categories, it appears that, as a group, instructors are in an early phase of CMS adoption. Arguably, they are using a CMS in familiar ways, such as transmitting information or completing grades. (Malikowski et al, 2007, p.167) As we will discuss later in this chapter, the case study of the University of Genova fits these conclusions. The adoption of a centralized LMS for the whole university poses many challenges. At the technical level, the administration of the single software platform is demanding, because of its large community of users; at the organizational level, proposals, decisions and strategies need to be shared and clearly communicated to the university community; at the educational level, faculty members need to be informed of the advantages they might achieve adopting ICT for education, and those who are convinced need to be trained; at the learning level, even if today’s students are “technology addicted”, or Digital Natives (Prensky, 2001), they do not really seem accustomed to use technology for learning. For example, McNaught, Lam and Ho (2009) analyse the use of a wide range of digital technologies, showing that students are indeed “digitally ready” but “there is no strong empirical evidence that students are committed to eLearning, that is, to using the technology-based tools they know how to use with the intention of maximizing their learning experience and outcomes” (p. 655). Success factors in implementing e-learning programs are deeply investigated in the literature. Volery and Lord (2000) observe that re-implementation of conventional models borrowed from passive face-to-face transmission does not work in the Internet based setting and can bring only minor improvements. Three critical success fac-

273

LMS Adoption at the University of Genova

tors are identified in online delivery: the ease of access to the technology, the instructor attitudes towards students and his technical competences, and the students’ previous use of the technology. O’Neill, Singh and O’Donogue (2004) discuss the issues associated with the global competition of Higher Education that, thanks to the development of new delivery and communication methods, has extended beyond national boundaries. Traditional universities have opportunities to change educational methods, and those who will miss the appointment will become less competitive. Samarawickrema and Stacey (2007) cluster the factors that enable and impede the adoptions of a LMS into five categories: institutional reasons; technology reasons; influence of colleagues and network; influence of research and the literature; and personal reasons. Time constraints, heavy workload, rewards structures, professional development, policies at the university and faculty levels are all issues that should be carefully addressed when deciding to adopt a LMS. MacKeogh & Fox (2009) provide similar conclusions presenting drivers and barriers that could emerge when trying to embed e-learning in the educational strategies. Some of these barriers like the lack of funding, of strategies at the university level, and of recognition in terms of career for the time spent in teaching activities are also preeminent in this case study. The discussion above poses the emphasis on LMS in a context that we can associate with the Web 1.0 that emerged in the mid-1990s and expanded access to (open) information. More recently, new research activities have emerged laying their roots in the so-called Web 2.0, spread out in the past few years starting from the seminal presentation of O’Reilly (2005). If Web 1.0 was about consuming information, Web 2.0 is about producing and sharing it. Tools such as blogs, wikis, social networks, tagging systems, and mashups are examples of user-centric applications that emphasize users’ participation and collaboration. The user becomes active in the process of

274

building knowledge and the network is no longer a mere tool for content distribution but it is rather a facilitator for the interaction among the participants involved in the educational process. Downes (2005) coined the term e-learning 2.0 to denote the need of changes in the pedagogical models in the first decade of the new millennium. The model of e-learning as being a type of content, produced by publishers, organized and structured into courses, and consumed by students, is turned on its head. Insofar as there is content, it is used rather than read— and is, in any case, more likely to be produced by students than courseware authors. And insofar as there is structure, it is more likely to resemble a language or a conversation rather than a book or a manual. (Downes, 2005) Dalsgaard (2006) suggests that it is necessary to move e-learning beyond LMS. These software platforms still maintain the main role of course administrators (e.g., managing students enrolments, exams, assignments, course materials, etc.), but the active learning process can be effectively obtained by using educational social software such as individual collaborative tools to build knowledge and social networks to engage in different forms of communication. Successful case studies exist within this line of research, mostly experiences involving researchers in the educational and technological fields. For instance, Ullrich, Borau, Luo, Tan, Shen, and Shen (2008) analyse technological principles of Web 2.0 from a learning perspective and two case studies, respectively based on micro-blogging and social bookmarking, are presented. To the best of our knowledge, the advanced experiences presented in the literature involve groups of early adopters and the transition from the more traditional use of LMS to real e-learning 2.0 is still in its infancy. Moving from the small scale of the early adopters to the large scale of the whole institutional level will be a slow process.

LMS Adoption at the University of Genova

TEN YEARS OF EXPERIENCE The University of Genova is a traditional university offering face-to-face courses. However, in the last ten years, it has made several efforts to improve the quality of its educational process by the introduction of ICT support. These attempts began as personal initiatives by individual faculty. But, they quickly scaled up to the level of Departments, and finally of the overall University. The many centralized and somehow topdown initiatives notwithstanding, the University of Genova still misses an official policy in ICT educational-support, badly hindering its adoption. Without such a policy, the level of commitment in terms of funding and personnel is unclear, making effective planning impossible. Moreover, as rewards for faculty members spending their time on ICT-supported teaching, so far on a totally voluntary base, cannot be expected, their enthusiasm is inevitably dumped. Thus, the lack of a central policy in ICT educational-support is a main strategic mistake, as discussed in literature (see e.g., (Samarawickrema & Stacey, 2007; MacKeogh & Fox, 2009)) and also pointed out during a certification procedure undergone by the University of Genova on the extent of ICT support (project UNIQUe, http://unique.europace.org/). The results, based on peer review, were mostly good, but for the lack of a strategy of eLearning. For this topic, the need of implementing a central policy plan on e-learning to be integrated into the general University quality plan has been stressed, together with the need of increasing the focus on the quality of communication.

Past Since the late nineties, the University of Genova has hosted and facilitated a certain amount of spontaneous experiences in the field of ICT supported learning. On the one hand, a few individual research groups experimented with the distribution of material via Web, and the use of

forums. Such pioneering efforts involved mainly some faculty members - the early adopters - of Computer Science, Engineering and Pedagogical Sciences. On the other hand, almost at the same time, the University joined two consortia, Tethys (http://www.tethys-univ.org/) and Nettuno (http:// www.consorzionettuno.it/), both promoting the development of distance learning. Though the involvement was officially at the level of the overall institution, the activities on these two networks have been mostly left to the volunteering commitment of individual research groups. The first really centralized effort took place in 2001, when the University participated in the national project CampusOne. This project was focused on two main goals: supporting reorganization of the degree structure in the spirit of the Bologna Declaration, and promoting technological innovation within the educational process. A specific committee was appointed to work on this latter action. At that moment, many universities in Europe were facing the “buy-or-build dilemma” and a survey conducted in more than one hundred European institutions (Paulsen, 2003) revealed the adoption of 52 different commercial and 35 self-developed LMS. On the open source side, Claroline (http://www.claroline.net/) seemed the most eligible, and it was taken in consideration for a while. But, in the end, the committee decided to build from scratch, because the available platforms seemed to be not fully SCORM compliant (http://www.adlnet.gov/Technologies/ scorm/default.aspx), while this was perceived, at that time, as a strong requirement. Moreover, the prospective users judged the available platforms insufficiently appealing. Then, each Faculty appointed a few members as representative users in the user-centered development of a centralized service for ICT support to education. As requirements addressed the teachers’ needs, the platform was expected to be extremely convenient to use by them, hopefully leading to a large number of adopters.

275

LMS Adoption at the University of Genova

In retrospect, the approach was naïve: first of all, students were not included in the stakeholders; secondly, including the requirements of all Faculties inevitably forced the development of a general purpose platform, no more appealing for any specific user than the already existing ones; finally, and most relevantly, the code production exceeded the available resources, so that the tool was not completed within the project, making its adoption problematic, even as an experiment. Besides these “official” initiatives, some other platforms were adopted (or their use was increased) by individual courses or programs, mostly following a blended learning style, since the activity around ICT supported learning was quite lively in those years. These early (official and unofficial) experiences consistently showed both that students greatly appreciated the tool support, and that the (non negligible) effort required on the teacher side had a good return of investment in terms of students’ participation and satisfaction.

Present At the end of CampusOne, in 2004, the University nominated a new committee to supervise the adoption of a sustainable centralized LMS for all Faculties. From the very beginning, it was evident that completing and maintaining the current custom made LMS was not feasible due to limited resources. Hence, the first task of the committee was to select a standard platform to be adopted. The requirements collected from the end users were mostly basic (support for forums, document repositories, polling, quizzes and wikis); thus, all the analyzed LMS met them. On the other hand, some users had a broad variety of very specialized requirements which no single platform was simultaneously able to satisfy. Since the choice of supporting only some of them, privileging some users over others, would be politically unacceptable, the committee elected to ignore the specific requirements in the selection process. Therefore,

276

the final choice was driven by two main concerns: stability and low-cost. Indeed, the end users made clear that further changes of platform in the next few years would not have been acceptable, and the extremely limited economical resources were absorbed by the service technical support. Thus, the committee decided to move to Moodle (Cole & Foster, 2007), because its large community of active developers guaranteed that it would be supported in the years, and because it was free of charge. Moreover, as Moodle is open source, tailoring of, and additions to, the platform could be seamlessly integrated, providing a natural way to address the specialized requirements by the individual end users. In retrospect, the choice made was sound. Indeed, nowadays Moodle is still maintained, its diffusion is growing to the point of both competing with the giant of the LMS market BlackBoard (http://www.itcnetwork.org) and having being adopted by leaders of distance learning, such as The Open University (http://learn.open. ac.uk/mod/oublog/view.php?user=27287). The Moodle-based centralized service was called AulaWeb (Ribaudo & Rui, 2009). Following the same policy already adopted for CampusOne, joining AulaWeb for faculty members has been, and still is, exclusively voluntary. In particular, the production of educational contents is left to the initiative of the individuals, within the scope of their capabilities and time availability. Moreover, a few professors from each Faculty accepted to act as supplementary administrators, to alleviate the burden of the undersized technical staff (initially two people, currently just one) devoted to AulaWeb. These administrators filter the needs of their users, and interpret the peculiarities of their own area of education. Initially, they had also the task of disseminating the use of the platform among their faculty members, while currently they just provide the first level of help desk. The technical staff takes care of the more advanced levels, manages and updates the platform. Moreover, the staff encouraged Moodle adoption by a few seminars

LMS Adoption at the University of Genova

on its tools to facilitate the use of the platform by ICT beginners. These seminars were a concrete help to limit the a-priori rejection of the platform and overcome the heterogeneity of the involved users – different in terms of education, age, and command of ICT technologies. In a short time all the Faculties of the University were present on the portal, although with different numbers of online courses and active users. The participation had a significant growth, as shown in Figure 1: the number of activated courses, in the histogram on the left, increased quickly and, as a consequence, the number of enrolled students grew up continuously. In August1 2009, AulaWeb underwent a structural reorganisation; all users were reset and only the active ones registered again. It is interesting to note that, in a short time, the number of students enrolled on the new portal, depicted by the dashed line in the diagram on the right of Figure 1, reached the same size of those already registered on the old portal. This fact witnesses that AulaWeb has now reached a number of users close to saturation, at least in the current setting. In order to significantly improve the use of ICT for education, a central policy is strongly needed. Indeed, many students, in their questionnaires, presented in Section 4, complain that the courses on AulaWeb are a

minority, and most of those available are poor. This is, however, unavoidable, unless the efforts voluntarily spent on AulaWeb have a place in the evaluation of faculty members, as they ask, or their use of AulaWeb is made mandatory; either action can be taken only on the basis of a central policy and should burst the use of AulaWeb, extending its users’ base to the students of currently unsupported courses.

AulaWeb in 2010 In August 2010, more than two third of the students (26280 out of 36835) were enrolled, along with the majority of faculty members (844 out of 1518, that is, 55.6%). AulaWeb hosted about 120 degree programs out of 143, for 1125 different subjects. During the week there was an average of 3800-4000 distinct users’ accesses in the 24 hours, with more than 100 users simultaneously connected, showing the vitality of its community. The data become, however, less encouraging when we investigate the use of the different Moodle tools. Figure 2 shows the use of AulaWeb: data are split by considering all the Faculties and the two “technological” subgroups formed by Computer Science (part of the Faculty of Science) and ICT Engineering (part of the Faculty of Engineering). The left histogram represents the percentages of

Figure 1. Growth of AulaWeb

277

LMS Adoption at the University of Genova

online courses and online students weighted with respect to the overall number of courses and students of the University. The last two data in the series (the rectangles with thick borders) show the same percentages when considering the whole University, thus providing the average trend. It is clear from the histogram that some Faculties are more “active” than others. If we consider the courses, then we have, as it could be expected, a clear “winner” in the Computer Science program, followed by the Faculties of Economics and Engineering. The numbers of students are not even roughly proportional to those of courses; for instance, Sciences and Engineering swap places. This phenomenon is due to the average number of students per course in different Faculties. Indeed, in many cases the most populated courses were the first to move to AulaWeb, because their large number of students made them difficult to manage by traditional means. So, in the Faculties where few courses moved to AulaWeb the average number of students is higher. Moreover, since each student has many courses, the number of students is naturally logarithmic over the number of courses. Notice that the percentages of students enrolled on AulaWeb for Foreign Languages and Computer Science are higher than 100%. This is due to the fact that, in both cases, also students from other Faculties are attending some of their Figure 2. AulaWeb Usage and course typology

278

courses, because those subjects are of interest for any kind of curricula. So far, we have discussed the quantitative impact of AulaWeb. Let us now analyze the offered courses from a qualitative viewpoint. As shown on the right of Figure 2, content repository courses, that is, those offering only downloading of educational material, are still (in percentage) the large majority, with the Faculty of Law offering only such kind of courses. Only in two cases, Architecture and Computer Science, the percentage of blended courses, that is those offering not only educational material, but also thematic forums and, in many cases, also other types of online activities, exceeds that of content repositories. Some Faculties also offer organizational courses, giving only support for administration of traditional courses, like for instance registrations to in-presence examinations2. In more detail, data collected in Figure 3 show the activities on AulaWeb normalized by the official number of courses for each Faculty. The last columns in the table show the average values and standard deviation. Here and in the following data comparisons, we consider as worth of notice values differing from the average by more than the standard deviation, and show with a gray background those below, and in a larger bold font those over such limits. The significance of this statistical

LMS Adoption at the University of Genova

analysis is somewhat limited by the small number of points in the series and their large spread, as shown by standard deviation measures close or even larger than the average. However, some data are outstanding by any standard, like for instance the number of resources for the Computer Science program, which is four times the average and more than twice the second best value. Some results, like the preeminence of the Computer Science program, or the better use of quiz and thematic forum by the Faculty of Education, which is also consistently over average on all activities, or, vice versa, the poor quality of the Faculty of Literature were to be expected. Some other achievements, like for instance the very good use of assignment and quiz by the Faculty of Foreign Languages, are mostly due to the individual efforts of some enthusiastic early adopters, who not only use the tools, but also disseminate the good practice among their colleagues. Finally, Figure 4 compares how students (S-Quest series) and teachers (T-Quest series) perceive their activities on AulaWeb to the percent-

age of courses actually offering those activities (AW-Log series). The data for students and teachers come from recent questionnaires, described in detail in Section 4. The AW-Log data show that content distribution is indeed predominant (more than 7 times the second higher value), followed by comparatively similar values for activities strongly related to the organization of traditional courses, (i.e., assignments and reservations3), as well as to communication (i.e., discussions in subject related forums4). Activities involving collaboration (i.e., wiki and glossaries) are by far less extensively used. By comparing the different series in the histogram, the users seem to acknowledge more interaction on all activities, but resource downloading, than it could be expected by the percentage of courses offering them (series AW-Log). We interpret this discrepancy as a confirmation that students and teachers using advanced features are more involved in AulaWeb, more familiar with it, hence more willing to participate into the questionnaires. Thus, we assume these statistics to be somehow biased. Another interesting point is that students’ involvement both in thematic

Figure 3. Activities on AulaWeb (empty cells stand for 0)

279

LMS Adoption at the University of Genova

forums and wikis is much lower than teachers’ participation, while for the other features they are comparable. These data confirm the many teachers’ complaints that students are not eager to share and build knowledge and that often collaborative activities are ignored.

The WEL Project As discussed so far, from the very beginning the use of AulaWeb has seen the download of teaching material as prevalent activity. In the winter of 2007/08 a quick survey showed that the main cause of the scarce use of more sophisticated tools was not, in most cases, the “fear of technology”. Indeed, thanks to the basic dissemination made by the AulaWeb staff, the needed knowledge was, in fact, available to all interested faculty members. The main problem appeared to be their teaching style, mostly based on in-presence lessons and totally content-driven. They did not feel the need for technical support of collaborative activities simply because such kind of activities was not part of their teaching process.

Figure 4. Activities on AulaWeb

280

University teachers, in fact, rarely come into contact with educational models, as they are required to be expert only on their own subject, their career is dependent mostly on their research achievements and courses on teaching techniques are not part of any program but those on Education. Thus, they usually come up with personal variations of the educational process they experienced as students, mostly based on in-presence lessons. For a large majority the course design still consists of selecting the topics to be covered and ordering them in a logical stream, without planning activities to better convey the relevant aspects of each topic, or to involve and evaluate students. Therefore, in April 2008 the action WEL (Web Enhanced Learning) was promoted, to improve the quantity and especially the quality of Web-supported teaching in the University. Its main goal was devising a model for the transfer of instructional design knowledge and skills to subject-oriented university teachers. The WEL course, delivered by ITD (the Institute for Didactic Technologies of CNR) experts (Puddu, Repetto & Vallarino, 2008), was driven by hands-on projects: each participant was indeed

LMS Adoption at the University of Genova

required and helped to redesign his/her course, including some form of collaborative activity. WEL has been organized into three distinct phases. In the first phase (May 2008) two plenary lectures showed to university teachers (1) an overview on the use of ICT technologies for educational methods and (2) the instructional design methodology they were going to follow to redesign their teaching strategies (see, e.g., Trentin, 2001). About 100 faculty members attended those sessions and, among them, 48 decided, at the end of the second lecture, to commit themselves to the project. The ICT competencies claimed by the participants were mostly average (64%), with a minority of experts (22%) and a few beginners (13%). The large majority (82%) had previous experiences in using LMS and almost 50% had already used AulaWeb, though mostly without any (formal) formation. There was a high variety of subjects, from humanistic to scientific disciplines; the distribution of participants with respect to their Faculty, normalized by the number of staff members of each Faculty is shown in Figure 5.

In the second phase (May-June 2008) two individual face-to-face meetings with CNR codesigners took place. Each participant had been asked to plan her course in terms of macro-design (i.e., goals and objectives definition, evaluation criteria, types of activities, first version of the course guide) and micro-design (i.e., providing a detailed storyboard of the course with a description of the modules, planned activities, initial messages for the virtual class, activity scheduling). In parallel, on demand technical course have been organized for small groups of participants. Various software tools have been presented, including some for content production, graphic and video editing, as well as some Moodle modules. The third and last phase started in September 2008 with the last face-to-face meeting with CNR co-designers: a sort of audit to check the work done during the summer to create the learning materials/strategies based on the design phase. The third step ended with the launch of the revised first semester courses. A final plenary meeting was organized in February 2009 to share the experiences of the first semester courses, and take into account the

Figure 5. Distribution of faculty members in WEL

281

LMS Adoption at the University of Genova

lessons learned to slightly adjust those of the second semester. It is worth noting that the participation was (again) on a voluntary base. The participants’ number has been limited for two reasons. On the one hand the instructional design experts were able to individually train only a few faculty members. On the other hand, even interested professors were not sufficiently encouraged into participation, because of the lack of recognition, and insufficient communication (both problems due to the missing policy for e-learning). Indeed, among the respondents of the questionnaire T-Quest, described in Section 4, 181 (out of 210) answered that they did not participate in the WEL project, because: “Did not have time” (92, that is, 43.81%), “Did not know about the project” (45, that is, 21.43%), “Thought it was not useful” (20, that is, 9.52%). While the remaining 24 (11.43%) did not answer. Information about WEL disseminated through “word of mouth” within the faculty members community since the majority of the respondents of T-Quest answered that they might consider participating to a second edition of the course, with 59“Yes” (28.10%), and 97“Maybe” (46.19%), while only 35 (16.67%) answered “No”, and the remaining 19 (9.05%) did not answer. Data shown in Table 1 compare aggregate numbers extracted from AulaWeb logs and can be used to get some insights on the changes in the use of AulaWeb in pre- and post-WEL periods. Comparing the data in Table 2, it is quite clear that, though the lion’s share of the online activity

is still given to resources distribution, something has changed when considering collaborative activities. Indeed, we can observe that the usage of tools like wikis and glossaries has increased in 2008/09, in most cases with a percentage of growth much larger than that of the courses and the other resources. Moreover, these numbers suggest that also the approach to the use of the tools is improving. For instance, not only the number of wikis has increased of almost 75%, but the number of pages and versions has dramatically grown, about 4 times the percentage of the wikis themselves! Thus, we can infer that not only do we have a larger number of wikis, but also that they are used differently, with more online activities going on. In general, all the teachers involved in the WEL experiment reported an improvement of their courses in the number of active participants, passed exams and even average scores. Moreover, most of the professors acknowledged that their teaching styles had changed, and this was, in fact, reflected by the numbers on AulaWeb. The findings presented here are consistent with other analysis from the literature. For instance, the interested reader can see (Yueh & Hsu, 2008), where the authors describe the adoption process of a LMS, built by an internal team, and the support offered to professors. Though they are not using Moodle, the approach followed to increase acceptance and usage of their platform is remarkably similar to that adopted in Genova. Their results, showing that most frequently used functions are indeed the

Table 1. Pre- and post-WEL 07-08

08-09

Courses

822

1,016

+23.60%

Resources

10,622

14,954

+40.78%

Forums

1,296

1,452

+12.04%

- Threads

7,917

10,133

- Posts

24,628

Assignments

692

282

07-08

08-09

Quizzes

135

199

+47.41%

Wikis

51

89

+74.51%

+27.99%

- Pages

302

1,115

+269.21%

30,856

+25.29%

- Versions

1,899

8,201

+331.86%

949

+37.14%

Glossaries

5

12

+140.00%

LMS Adoption at the University of Genova

basic ones, and that supporting professors with a team of instructional specialists has increased the use of their LMS, are in accordance with the findings of the WEL project. Comparing the numbers of 2008/09 to those of 2009/10, we can see that courses, resources and forums have slightly increased (respectively from 1016 to 1125 (+11%), from 14954 to 18464 (+23.5%), and from 1452 to 1478 (+2%)). On the contrary, quizzes, wikis, assignments and glossaries have decreased (respectively from 199 to 147 (-26%), from 89 to 62 (-30%), from 949 to 863 (-9%), and from 12 to 6 (-50%)). Part of this failure may be due to the update of the platform, and the resulting cleanup of obsolete data. But, as courses, resources and forums have nevertheless increased, still we can identify a difficulty with the more advanced features, which is not surprising. Indeed, lacking a follow-up of WEL (the promised second edition never started), any reward for the faculty members, and a clear appreciation from the students (who deserted collaborative activities as much as possible), the enthusiasm of teachers deflated, and they went back to their (bad) practices.

(Unwilling) Collaborative Activities It is worthwhile pointing out that during the WEL experience, the students were required to participate into some collaborative activities in order to pass the exams, and those activities were mostly successfully, contributing to the improvement of AulaWeb usage. But, some optional collaborative activities went mostly ignored. For instance, several courses proposed optional wikis for the collaborative production of notes, or collections of solved exercises, which had to be abandoned for lack of critical mass. The students, even those of Computer Science who should not have experienced any technical problem, candidly admitted that “they had no time for activities not directly rewarded in terms of grading”. Thus, it appears that the crucial failure was in motivations,

more than in actual limitations of the tools, or insufficient technical skills of the students. This experience seems to be quite in contrast with the main claims in the literature that students are used to, and insist on, sharing knowledge, as for instance in (Prensky, 2007). However, it seems to be perfectly coherent with the experiments in (Ebner et al., 2008), where optional wikis, with the only reward of being allowed to use them at examination time, were left totally empty, and with the observation in (O’Neill, 2005) that unless there is a prized reward for it, and that clear responsibilities are attributed to individuals, wikis are simply not used by students. Moreover, on the unofficial students’ sites, the personal notes are always provided as documents, sometimes even at payment, and never collaboratively developed. Thus, it seems quite clear that students are simply not interested in cooperative efforts. Sharing knowledge means to them sharing individually developed documents. Collaborative activities are successful only if participation is compulsory, though after being (forcedly) involved, students acknowledge the activity value, agreeing that they reached a level of knowledge much deeper than that they could have acquired working the same time in isolation, and that the social aspects were actually enjoyable.

CASE STUDY: AULAWEB AND WEL FOR THE COMPUTER SCIENCE CURRICULA Faculty members of the degree program in Computer Science heavily use AulaWeb from its first activation, after having been prominent in a few earlier experimentations. Currently, the Computer Science program is by far the best user of AulaWeb, and this could be related to its large participation in WEL, with 13.8% against 2.9% on the overall University. In particular, two mandatory courses expected to be taken in the first semester of the third and last year of the first cycle were part of the

283

LMS Adoption at the University of Genova

experiment: Programming Advanced Techniques (TAP for short) and Software Engineering with project (SE for short). Together, these exams take up 18 out of the 30 credits of the semester. The other 12 credits are evenly distributed between another mandatory course, Enterprise Knowledge for the IT Professional (EK for short), and an open choice from a list. Thus, the analysis of the results for those courses, and those sharing the semester with them, is significant to get an insight on interferences and global trends. In (Cerioli & Ribaudo, 2009) the details of this experience are illustrated. Here we just recall the key points to analyze the impact of WEL. TAP and SE address similar topics, requiring basically the same capabilities to be successfully taken. Both have a major project, which the students find hard to pass, and several preparatory activities, received by students, at best, with lukewarm interest. Moreover, the teams of teachers delivering both courses were almost totally overlapped in 07/08 and 08/09. However, before WEL, TAP and SE were differently organized: SE was strictly structured in a stream of collaborative activities, while TAP was loosely proposing individual tasks. Accordingly, the changes introduced following WEL, were different in nature: SE improved only the ICT support to its well-established organization, while TAP was totally restructured around a set of asynchronous collaborative activities. In 07/08 there were almost twice as many students passing the project and the overall exam of SE than of TAP (in percentage). After the WEL-driven redesign, in 08/09 the percentages of successes for SE increased marginally, while those for TAP more or less doubled, leveling the difference with SE. Though both courses were successful, during the semester the teachers of other courses having the same expected audience gave negative feedbacks. The student participation was decreasing, with peaks in occasion of deadlines for mandatory activities of SE and TAP. It is worth noting that

284

both SE and TAP monitor students’ efforts, not to exceed the expected workload. Moreover, the deadlines of the activities of the different courses for the same year are globally planned to avoid conflicts. Hence the interferences should be null or very limited and the average full time student should be able to manage all the courses at the same time, while it was not so. For instance, in 2008/09 less than half (in percentage) of the student successfully took EK exam, if compared to 2007/08. It is quite clear that improving courses, by encouraging the participation of the students in interesting activities during the semester, has a negative side effect on the other courses, which have a choice between partially losing their audience (like EK) or planning some activity of their own to capture the students, adding to the student burden and competing for their time (like TAP adversus SE). The competition among courses is due to a peculiarity of the Italian system. Indeed, in Italy the concept of passing from a year to the next does not exist. Students simply have to eventually collect positive marks in all the courses in their curriculum in an arbitrary number of academic years. Hence, even mandatory courses may be almost indefinitely postponed. Moreover, each year the same exam may be taken by each student several times (an average of 6-12 possibilities is common all over Italy). Thus, many students take an exam with an incomplete preparation, thinking that if they are not lucky the first time, they can still try it again (and again...) in a very short time, slightly improving their preparation, till they are able (or lucky enough) to pass it. These two mechanisms were established to facilitate the students, by letting them organize themselves. But, it is quite clear that they are, paradoxically enough, causing the students to lose a lot of efforts. Indeed, they encourage the students not to plan, focus and deliver, but to overreach and struggle. Many students are not able, nor forced by the regulations of the University, to complete the exam of some year before passing to the next.

LMS Adoption at the University of Genova

So they add a full workload of new exams, hoping of regaining the lost time. They start to work on all courses, old and new. But, when the effort required becomes too much, they drop some courses, starting from those where they invested less, that is, those without intermediate activities. It is worth noting that the interferences due to the WEL-driven redesign are of the same nature as those experienced in the past every time a course proposed any activity during the lesson time while others limited themselves to standard lessons. So the conflict is between courses with/without activities during the term, not between courses using/ignoring technological support. However, the current trend toward a more participative and technologically supported learning model will amplify the problem, introducing a divide between modern and traditional courses, till the latter will be completely extinct. The delicate point will be to find a balance among the different competing courses to let students participate in all of them.

Feedback on AulaWeb Any online service needs to be evaluated by its users to understand its acceptance. Hence, we have planned and administered two online surveys for the community of AulaWeb users, using the Moodle feedback. Both questionnaires were proposed to the overall population, by email invitations through the official university mailing lists, and feedback submission was not enforced. Thus, the received feedbacks are given by motivated volunteers, who can be assumed to have answered at the best of their capabilities. On the other hand, the surveys had to be light in order to encourage participation, so that in many cases just one choice among a list of possibilities has been proposed, in place of a group of standard Likert scaled questions, one for each item in the list. This approach obviously hinders any statistical analysis.

Teachers Feedback (T-Quest) During the summer 2009, a short survey (just 14 questions, some of them open-ended), in the following called T-Quest, was carried out among faculty members to get their opinions on AulaWeb (and, marginally, on WEL). For about one month it was possible to submit feedback, and in the end 210 valid answers were received (12.9% of the faculty members, 23.5% if we consider only those enrolled to AulaWeb as teachers in 2009). The answers were not categorized by Faculty, so that there is no way to verify if they are representative of the actual ratio of staff. Faculty members were asked about their use (or lack of use) of AulaWeb, and those using the platform were also solicited for their judgment on its ease of use. Only 11 out of 210 teachers (about 5%) said they do not use AulaWeb. Out of the others 199 participants, 67.84% find the platform optimal or good to use, the remaining 32.16% find it fair, satisfactory or scarce. Those who were not totally satisfied left 45 comments, complaining about the rigidity of the user interface; the weak support for group management; the high number of steps necessary to perform simple operations like uploading a single file. Almost two third (134 out of 199, that is, 67.34%) stated that they do not you use AulaWeb only for uploading material for lectures, and further detailed their usage of the tools available on AulaWeb, as already shown in Figure 4. For the other 65, who consider AulaWeb as a mere repository for their notes and slides, the motivation for not using the rich array of available tools was equally distributed among the following answers: “I have no time”, “I do not know how to use them”, “I do not think they are useful for my subject”, “I have tried to use them, but then I gave up”. The final part of T-Quest was about the (dis) advantages introduced in the teaching style by the adoption of AulaWeb, if any. The majority of the 158 comments left recognized among the

285

LMS Adoption at the University of Genova

advantages the ease of distribution of the teaching material and the increase of communication and interaction with the class, as well as the possibility of complementing and enriching face-to-face lessons by asynchronous communications. But, faculty members observed an increase in their workload for preparing the digital material and for timely answering forum posts. Moreover, some perceived AulaWeb as too complex to be used by beginners. According to teachers, most of the students seem satisfied although some faculty members stress that the reason is that they have easy access to the teaching material while it is generally difficult to involve them in any form of interactive activity. They also observe that full-time working students are the most approving, since the platform allows them to (virtually) take part to the lectures. Finally, to the question “Do you think that an adequate training on the use of ICT technologies for education could improve your teaching activity?”, half of the respondents, that is, 103 (49.05%) said “Yes” and another 71 (33.81%) said “Maybe”, giving 72% of positive responses, while only 5.71% answered “No” (and the last 11.43% did not answer at all). Figure 6. Percentage of students’ answers

286

Students Feedback (S-Quest) Analogously to the T-Quest, during the summer 2010, an online survey composed of 28 questions (single and multiple choice, plus some open-ended questions), in the following called S-Quest, was launched among the students. S-Quest has been available for about two months and 1397 valid answers have been collected, slightly less than 4% (9%) of the students enrolled in the University (AulaWeb). Students could choose not to answer specific questions, so that the numbers of answers for the possible choices (and their percentage) very seldom sum up to the number of respondents (100%), though in the following we list for each question the results of all possible answers. The distribution of answers among Faculties, weighted by the number of students officially enrolled to them, sees the Faculty of Sciences (including the Computer Science program) at the top, followed by those of Education, and Foreign Languages, and then by those of Engineering (counting also the technological group), and Economics, as shown in Figure 6.

LMS Adoption at the University of Genova

In Figure 7, the sample is characterized in terms of age, kind of degree and working position. Since our University does not provide these data about students, it is not possible to know if the answering sample is representative, under these respects, of the overall population. 28.42% of the respondents claims optimal skills in the use of the computer, 50.61% good skills, 18.04% sufficient knowledge and only about 2% declare themselves not enough skilled to take full advantage of ICT supported education. The majority of students (66.71%) go to the university every day, followed by 15.75% answering two or three times per week, 3.65% less than once in a week, 1.50% less than once in a month. These data, with more than 80% of frequently presents, confirm the vocation of Genova University as face-to-face educational institution. However, the 8.59% of students is going to the university only

to take exams, suggesting a possible market for e-learning. When asked about their frequency of access to AulaWeb, 19.18% answer every day, 39.73% two or three times per week, 23.91% less than once per week, 6.23% less than once per month, and 9.31% only for exams. The vast majority (86.54%) connect from home, 7.02% from the university laboratories, with a small percentage (3.08%) accessing from work or other means, like internet café (2.29%). The vast majority (82.39%) easily located their courses online, but 194 (13.89%) were not able to find them and many of them blamed their failure on the lack of courses for their program, which agrees totally with the situation shown by our previous analysis for some Faculties. The next questions investigate the ease of use of AulaWeb. Its interface does not pose any problem for more than half of the respondents (59.84%)

Figure 7. Characterization of students’ sample

287

LMS Adoption at the University of Genova

while it is considered too crowded by 11.31%, too static and impossible to personalize by 8.59%, “antique” by 10.02%, leaving a small percentage of 4.65% to the option difficult to browse. The online interaction is generally judged to be fine (by 58.87%), but not efficient (i.e., requires too many clicks) for 16.39% of the students, too slow for 8.38%, too repetitive for 8.16%. A significant percentage of students (11.88%) do not expect any interaction with teachers and even when it was expected, it has been null for 8.23% of the students, scarce for 28.20%, quite frequent for 29.35%, very frequent for only 4.94%. Thus, apparently communication between teachers and students still need improvement. Answers to the question “Has the use of AulaWeb enriched the formative offer of your course?” are shown in Figure 8. Many of those having a negative viewpoint left comments, clustering on two major complaints: the subjects present on AulaWeb are not enough, and many teachers use AulaWeb only for uploading (sometimes after a long delay) a digital version of the material available in the past at the student office. Any form of interaction or collaboration is lacking. Finally, in some cases, now finding the teaching material has become even more difficult Figure 8. Improvement of the formative offer

288

since those teachers who do not use AulaWeb still maintain some Web pages and therefore, from the student’s point of view, multiple URLs are necessary to find all the relevant information. Independently from its actual use, we have asked students whether they think that “The use of AulaWeb could improve their higher education”, and we got a 4.01% of “No” answers, followed by a 19.90% of “Not noticeable”, a 30.21% of “Fairly”, and 42.16% of “Yes”.

CONCLUSION AND FUTURE DEVELOPMENT This chapter has reviewed the last ten years of experience on ICT supported learning at the University of Genova. The adoption of a centralized LMS has been presented, discussing the followed process, the achievements, a formative project on instructional design techniques organized for faculty members, and the feedbacks provided by teachers and students. The numbers presented here clearly show that AulaWeb has reached its limits. It seems indeed difficult to think of further improvements without significant changes in the overall organization.

LMS Adoption at the University of Genova

Some contradictions emerge. The service itself should be enhanced, but the size of the staff barely manages the current situation, and the University is disinclined to spend. At the same time, students ask for a larger number of (traditional) courses to be supported also online, while faculty members seek recognition of their online activity. The University, as institution, needs to improve its rating and financial income, increasing the numbers of students, by catching also remote users and improving its services, so that expanding the ICT support to education becomes crucial, but the University is not disposed to put high priority on the definition of an e-learning policy. The financial side of the problem is amplified by the current general economic crisis: the shortage of resources to invest in e-learning prevents the student numbers, and hence the corresponding revenue, from increasing, in a vicious circle. The claim of faculty members for recognition of their online activity is exasperated by the fact that nowadays their evaluation is based mostly on research production, and marginally on officially recognized activities. So, in terms of working career, if in the past educational efforts paid very poorly, in the current settings spending time on teaching activities is totally detrimental unless officially recognized. Despite all the difficulties, providing online support for education is a must for a 21st Century University aspiring to be world-class. Hence, the definition a policy for e-learning, to be submitted to, and get approved by the central authority, is one of the first priorities of the committee responsible for e-learning and AulaWeb. Once established the policy, it will be interesting – and necessary - to deeply reanalyze user activities and their perception of the service to understand its impact. Meanwhile, the service needs to be maintained and improved to meet the users’ expectations that have emerged from the questionnaires. This affects the technical side of the management of the platform since new Moodle modules or their updating might be required to solve the problems

that have emerged by the feedbacks provided by our community of users. A comparison of our case study with similar research in Europe would help to clarify the negative impact of permissive examination policies. We need to collect data from educational systems where the exams of one year have to be passed before starting the next or, at least, cannot be taken several times a year, and compare the student results and LMS usage in those and our systems. Thus, we could understand if the competition among courses we have discussed actually impairs the adoption of collaborative activities in education.

REFERENCES Cerioli, M., & Ribaudo. M. (2009). Great is the enemy of good: Is perfecting specific courses harmful to global curricula performances? 4th European Conference on Technology Enhanced Learning (EC-TEL 2009) (LNCS 5794). BerlinHeidelberg, Germany: Springer. Cole, J., & Foster, H. (2007). Using Moodle. Teaching with the popular open source course management system. O’Reilly Community Press. Free online edition. Dalsgaard, C. (2006). Social software: E-learning beyond learning management systems. European Journal of Open, Distance and E-learning. Downes, S. (2005). E-learning 2.0, eLearn, 2005 (10). Dror, I. E. (2008). Technology enhanced learning: The good, the bad, and the ugly. Pragmatics & Cognition, 16(2). Amsterdam, Netherlands: John Benjamins Publishing Company. Ebner, M., Kickmeier-Rust, M., & Holzinger, A. (2008). Utilizing Wiki-Systems in higher education classes: A chance for universal access? Springer Universal Access in Information Society International Journal, 7(4). 289

LMS Adoption at the University of Genova

Janossy, J. (2008). Proposed model for evaluating C/LMS faculty usage in higher education institutions. Conference MBAA 2008.

Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon - The Strategic Planning Resource for Education Professionals, 9(5).

MacKeogh, K., & Fox, S. (2009). Strategies for embedding e-learning in traditional universities: Drivers and barriers. Electronic Journal of eLearning, 7(2).

Prensky, M. (2007). Changing paradigms – from “being taught” to “learning on your own with guidance”. Educational Technology. July-August.

Malikowski, S. R., Thomson, M. E., & Theis, J. G. (2007). A model for research into course management systems: Bridging technology and learning theory. Journal of Educational Computing Research, 36(2). doi:10.2190/1002-1T5027G2-H3V7 McGee, P., & Diaz, V. (2007). Wikis and podcasts and blogs! Oh, my! What is a faculty member supposed to do? EDUCAUSE Review, 42(5). McNaught, C., Lam, P., & Ho, A. (2009). The digital divide between university students and teachers in Hong Kong. Proceedings ASCILITE 2009, Auckland, New Zealand. O’Neill, K., Singh, G., & O’ Donoghue, J. (2004). Implementing e-learning programmes for higher education: A review of the literature. Journal of Information Technology Education, 3. O’Neill, M. E. (2005). Automated use of a Wiki for collaborative lecture notes. SIGCSE Bull, 37(1). New York, NY: ACM. O’Reilly, T. (2005). What is Web 2.0: Design patterns and business models for the next generation of software. Retrieved from http://www.oreillynet. com/pub/a/oreilly/tim/news/2005/09/30/what-isweb-20.html (Dec. 2010). Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4).

290

Puddu, F., Repetto, M., & Vallarino, E. (2008). The WEL initiative: A model of Web enhanced learning for faculty members. Journal of E-learning and Knowledge Society, 4(3). Ribaudo, M., & Rui, M. (2009). AulaWeb, Webbased learning as a commodity - The experience of the University of Genova. 1st International Conference on Computer Supported Education (CSEDU 2009). INSTICC Press. Samarawickrema, G., & Stacey, E. (2007). Adopting Web-based learning and teaching: A case study in higher education. Distance Education, 28(3). doi:10.1080/01587910701611344 Trentin, G. (2001). Designing Online Courses. The Web in Higher Education: Assessing the Impact and Fulfilling the Potential. The Haworth Press. Ullrich, C., Borau, K., Luo, H., Tan, X., Shen, L., & Shen, R. (2008). Why Web 2.0 is good for learning and for research: Principles and prototypes. 17th International World Wide Web Conference (WWW 2008). Volery, T., & Lord, D. (2000). Critical success factors in online education. International Journal of Educational Management, 14(5). doi:10.1108/09513540010344731 Yueh, H.-P., & Hsu, S. (2008). Designing a learning management system to support instruction. Communications of the ACM, 51(4). doi:10.1145/1330311.1330324

LMS Adoption at the University of Genova

ADDITIONAL READING Anderson, P. (2007). What is Web 2.0? Ideas, technologies and implications for education. JISC Technology and Standards Watch. Bristol, UK: JISC. Retrieved December 28, 2010 from http:// www.jisc.ac.uk/media/documents/techwatch/ tsw0701b.pdf Hassan, M. S. (2007). Critical success factors for e-learning acceptance: Confirmatory factor models . Computers & Education, 49(2). Mahmud, K., & Gope, K. (2009). Challenges of implementing e-learning for higher education in least developed countries: A case study on Bangladesh. International Conference on Information and Multimedia Technology (ICMT 2009). Morgan, G. (2003). Faculty use of course management systems. Research study from the EDUCAUSE center for applied research. Retrieved from http://www.educause.edu/ECAR/FacultyUseofCourseManagementSy/158560

Instructional Design: The practice of maximizing the effectiveness, efficiency and appeal of instruction and other learning experiences. Roughly speaking, it consists of determining the skills and needs of the learner, the end goal of instruction, and designing activities to assist in the transition. Learning Management System: Commonly abbreviated as LMS, are software applications for the administration, documentation, delivery, and reporting of training programs. Online Surveys: a series of questions proposed through a computer for the purpose of gathering information from respondents. Web Enhanced Learning: Combines traditional face-to-face courses with pedagogically sound uses of Web technology into the design and delivery of the course.

ENDNOTES 1



2



3



4



Woods, R., Baker, J. D., & Hopper, D. (2004). Hybrid structures: Faculty use and perception of Web-based courseware as a supplement to faceto-face instruction. The Internet and Higher Education, 7(4). doi:10.1016/j.iheduc.2004.09.002

KEY TERMS AND DEFINITIONS Blended Learning: Combines traditional face-to-face courses with computer-mediated education. Learners and teachers work together within and outside the class to improve the quality of learning and teaching. Collaborative Activities: Two or more people learning or attempting to learn something together using methodologies and environments in which learners actively engage in a common task. eLearning: Policy: courses of action, regulatory measures, laws, and funding priorities concerning ICT support to educational processes.

In September the courses for the next academic year are created, and students who have passed the exams during the summer start to withdraw from the corresponding courses. Thus, August is the best moment for data collection on one year. Moreover, in August there is no educational activity going on, so that it is also the best choice for technical updates. In the Italian system each exam is given, and may be taken by the same student, many times per year, so that the organizers need to know how many students are to be expected in order to choose appropriate classrooms/ laboratories. The former are mostly used to collect essays and projects autonomously produced by (groups of) students, while the latter to register for in-presence exams. Forum news is omitted from the analysis, since each Moodle course has one by default.

291

292

Chapter 14

Effective Use of E-Learning for Improving Students’ Skills Lorenzo Salas-Morera University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández University of Córdoba, Escuela Politécnica Superior, Spain

ABSTRACT The educational system promoted by the European higher education area advocates the introduction of new teaching methodologies in order to improve students’ skills as well as their knowledge in the subject areas they are studying. In response to this, new teaching strategies were implemented in Industrial Engineering and Software Engineering degree courses. The main goal of the project was to improve students’ skills in areas including problem-solving, information management, group working and the acquisition of writing and speaking skills, by means of e-learning tools. In addition to implementing the new strategies, a set of assessments including surveys, forum activity analyses and group tutorial evaluations were also carried out. The combined use of these techniques proved a very useful way of improving the students’ general skills and knowledge, especially in terms of design methods and organisation and planning ability and in general academical performance.

DOI: 10.4018/978-1-60960-884-2.ch014

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Effective Use of E-Learning for Improving Students’ Skills

INTRODUCTION Over the last few years, major regulatory changes have been introduced at European level, leading to an overhaul not only of the structure of university qualifications but also of teaching methods, with the incorporation of more active methodologies in which the role played by students in the learning process has expanded significantly. Teaching methods involving fluid and effective interaction between teacher and student, and amongst the students themselves, aimed at facilitating the exchange of opinions and general information, streamlining the tutorial system and encouraging group collaboration, have thus acquired particular importance. The results presented here are drawn from a six-year experiment in Industrial Engineering and Software Engineering courses. Using a range of teaching techniques, the aim of these courses was to enhance the active and responsible participation of students in the learning process, and to enhance the acquisition not only of subject-related knowledge but also of certain general skills appropriate to the study of engineering. To this end, modified teaching methods were incorporated into four modules of the Engineering Projects section, in the last year of the course, replacing traditional teaching methods based on teacher-centred classes and examinations with an alternative based on various techniques such as asynchronous discussion forums, group tutoring, collaborative learning, online quizzes and peer assessment. To attain this objective, the following secondary considerations had to be addressed (Salas-Morera, Berral-Yerón, Serrano-Gómez & Martínez-Jiménez, 2009; Lan & Yang, 2009): A. Students need to be motivated to work regularly and follow the correct sequence of activities set out in the syllabus. B. Overall student working time needs to be appropriate to the credits assigned for the subject, adopting a realistic approach to all

the activities required: attendance at theoretical and practical classes, private study, completion of tests, preparation of reports, tracking and participating in the forum. C. The teacher needs to be able to attend – appropriately and at a suitable pace – to all the students in the time available. D. The students need to receive prompt and reliable feedback on the results of the class assessments. E. The members of the group need to interact proactively between themselves and with their teacher, raising questions and generating productive contributions to the discussion forums, constructing a knowledge base that is critical and robust. The results of this experiment were generally highly positive. Specifically, students rated very highly the exchange of information through the forum, while the role of the teacher as motivator and moderator was regarded as crucial. Similarly, online quizzes were very positively viewed by students, who – in exchange for investing a moderate but timely amount of effort – were able to reap considerable academic rewards. Finally, it was seen as essential that the general scheduling of the students’ work be well coordinated in terms of the course as a whole, and realistic in terms of the amount of effort required, in order to avoid the kind of backlog that would hinder the assigned tasks being properly completed.

BACKGROUND E-learning tools are being widely incorporated into the teaching-learning process as a backup and complement to conventional university classes, as well as in distance-learning institutions (Yau, Lam & Cheung, 2009). These tools, among which Blackboard, WebCT and Moodle figure prominently, help teachers to make learning more attractive, dynamic and participatory by

293

Effective Use of E-Learning for Improving Students’ Skills

virtue of forums, virtual tutorials, blogs, social networks, wikis, surveys, online quizzes etc. (Kozaris, 2010). Thus, it is no longer enough to simply expose students to course content; there needs to be interaction between students and teacher, among the students themselves and with the learning community and its learning resources (Banks, 2003). For their part, the students receive information immediately, both in terms of content and activities, as well as the results of assessments, making this type of tool highly efficient (Lahwal, Amaimin, & Al-Ajlan, 2009). Such tools enable teachers to maintain a tighter control over the pace of the group, adapting the degree of difficulty and the introduction of content to the general progress of the group, and allowing the students to organise their own work both individually and as a team (De Laat, Lally, Lipponen, & Simons, 2007). However, using e-learning tools brings its own difficulties: first it requires a high level of commitment on the part of the academic authorities, who need to devote sufficient resources in terms of hardware and technical staff to guarantee that such tools are available with enough flexibility and efficiency (McPherson & Nunes, 2008); secondly, in order to get the most out of these technologies, teachers as well as students need to exhibit certain skills that they may not necessarily possess at the outset (De Laat & Lally, 2003; Kirkwood, 2009). In this way, one of the most important issues using e-learning tools is usability in terms of whether a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of user (according with the international standard ISO 9241-11). Kakasevski, Mihajlov, Arsenovski & Chungurski (2008), analysed standard modules of Moodle giving useful information for administrators, teachers and students on how to improve effective usage of this system. Kirner, Custódio & Kirner (2008) analysed intuitiveness, operationability, efficiency of use, learnability, attractiveness, and user satisfaction, concluding that Moodle has a satisfactory usability level.

294

Nevgi, Virtanen & Niemi (2006) note that learning through the Web is a social process involving not only individual but also social and group skills in the collaborative creation of knowledge. They therefore recommend a package consisting of assessment or self-assessment tests, asynchronous discussions, tutoring sets and learning diaries which provide the user group with the tools required for real, profound collaborative learning. This context prompts a major change in the teacher’s role: compared to the traditional educational system in which the teacher was merely required to transmit information, his/her role has now become much more complex – and more demanding – in that the teacher is expected to act as advisor, content facilitator, resource provider, technician and social animator (Zhao, & Jiang, 2008). In the same way, there are additional difficulties in managing students’ groups and interactions when on-line collaborative activities are programmed (Tuparova & Tuparov, 2010; Vivek & Ansari, 2010). Asynchronous discussion forums are a very useful tool for encouraging the critical dimension of learning; students interactions, both among themselves and with the instructor, yield a synergy in the approach to preparing, sharing and understanding information, which fosters a fuller understanding of the material to be studied (Erlin, Yusof & Rahman, 2009). Yet the use of activities of this sort has certain drawbacks, due mainly to a lack of active student participation in the e-learning platform, a lack of quality in the contributions, and the use of the forum as a means of exchanging information unrelated to the subject (Ertmer, Richardson, Belland, Camin, Connolly, Coulthard, Lei & Mong, 2007; Lan & Yang, 2009; So, 2009). Mazzollini & Maddison (2003), examining the role of the teacher as forum director, report that instructor intervention tends to generate more student satisfaction than peer participation; increased instructor participation also helps to shorten discussions, without necessarily impairing their final quality. It is therefore essential

Effective Use of E-Learning for Improving Students’ Skills

to decide beforehand what goals the forum seeks to achieve – e.g. active student participation, problem-solving, deeper conceptual understanding of the subject under study, exchange of information – since these goals should govern the level and style of teacher participation. Similarly, Andresen (2009) emphasises that online discussion forums do not constitute a way of eradicating or substituting the teacher, but on the contrary demand a higher level of attention than traditional classes, something that needs to be acknowledged by the academic authorities. However, despite the fact that the forums have been criticised in the past for their lack of quality and low levels of student participation, Andresen notes that these problems have diminished in recent years. According to Rovai (2007), it is essential to bear in mind a number of factors if an effective strategy is to be devised that will allow online discussion forums to be exploited to the full: motivating students to take part in discussions, for example by awarding them a mark; leaving students in no doubt from the outset of the course that their participation is expected; providing opportunities for socioemotional discussions, which give rise to a strong sense of community; and ensuring that discussions remain content-oriented and task-oriented. At the same time, teachers need to exhibit an attitude that fosters the social dimension, avoiding becoming the centre of attention themselves and encouraging equality in terms of the culture, gender and status of their students. Taking a similar approach, Hew & Cheung (2008) also note the lack of participation on the part of the students and the poor quality of their contributions, and propose various strategies through which the teacher can attract student participation: e.g. by giving his own opinions and experience; by starting discussion threads, asking questions about specific aspects of the subject; by appraising student input; by suggesting new directions for the discussion; by thanking students for their posts; and by clarifying specific issues. Given that participation in online forums needs to be evaluated, other major considerations to

be borne in mind when using forums as teaching tools include the evaluation method selected and the ratio of the teacher effort involved to the effectiveness of the assessment in terms of learning objectives. Dennen (2008) highlights the importance of evaluating student participation in forums, but questions the efficacy of the evaluation systems used, arguing that while one may readily measure the length and the number of messages posted by each student, these measures do no necessarily reflect the learning achieved. Whilst online discussion is valuable in that it encourages a sense of social connection, students cannot be expected to have sufficient rhetorical, analytic and argumentation skills to fully engage in certain types of activities. Dennen suggests no fewer than four different methods of assessment: A. Participation measures: this cannot strictly be considered a method of assessing learning, since a relationship between the quantity of messages and learning cannot be assumed. B. Message content and quality measures: much more reliable, but more difficult to measure and requiring more effort on the part of the teacher (something that must also be borne in mind). A simple way of doing it is to classify the messages as on-topic or off-topic, but this would prevent the possibility of awarding a nuanced score, while exposing students to the temptation of submitting mini-essays, thereby interfering with the interaction. C. Holistic measurements: simultaneously taking into account both quality and quantity engenders a positive feedback loop encouraging learning more than mere participation. D. Asking for short reports on the students’ experiences in the discussion processes. Another tool widely used as a complement to e-learning strategy is e-assessment, whose chief advantages are a considerable reduction in marking time for the teacher and the opportunity for students to schedule their individual efforts,

295

Effective Use of E-Learning for Improving Students’ Skills

completing the tests at a variety of times and locations. There are doubts, however, about the true efficacy of e-assessments. The following questions have been raised: do e-assessments place an additional burden of stress on the students? Do students possess adequate ICT skills to tackle these assessments successfully? Does this type of examination conform with the students’ learning style and with the type of technology most commonly used in the workplace? (Whitelock, 2009). Angus & Watson (2009) argue that compliance with three basic criteria is essential in order for online assessments to be genuinely effective and assist in improving learning: a) online assessment methods must be an effective proxy for traditional methods; b) students must be at least indifferent to whether online or traditional methods are used; and c) online methods should provide better outcomes than traditional methods. They conclude that exposure to regular low-mark online testing significantly improves student learning as measured by a final proctored exam. In a similar vein, Kibble (2007) and Dobson (2008) investigate whether student participation in low-mark online quizzes enhances final results, concluding that students either agree or strongly agree that online tests constitute a suitable assessment method and that they have a positive effect on the final mark. Kibble (2007) also stresses that the best assessment strategy is to use different methods simultaneously, such as online tests, group assignments and final examinations, with the aim of encouraging different ways of reflecting on content. Savander-Ranne, Lundén & Kolari (2008) designed a course in which students were exposed to a learning cycle comprising taught classes, pre-lecture assignments, student seminar presentations and concept tests aimed at encouraging student participation, with the consequence that the students displayed a substantial improvement in their communication and problem solving skills, as well as an improvement in general learning outcomes.

296

Furthermore, it is very important that professors pay attention, not only on students’academical performance and knowledge acquisition, but on a complete curriculum design which must take into consideration the practical skills needed for the students’ professional development (Zhou, 2004; Barrella, Simmons & Buffinton, 2006). Thus, considerable efforts have been made over recent decades, to define the skills needed by engineering students in order to find the right job and ensure career development, favouring those curricula that focus on the acquisition not only of knowledge but also of skills and competences. Rehman, Said & Al-assaf (2009) stress the need for coherent development of students’ skills over the whole curriculum. In the case of engineering and technology, as well as the relevant local and national regulations, the Criteria for Accrediting Engineering Programs (ABET, 2009) must also be taken into account. Among such skills the following stand out: A. An ability to apply knowledge of mathematics, science, and engineering B. An ability to design and conduct experiments, as well as to analyse and interpret data C. An ability to design a system, component, or process to meet desired needs within realistic constraints prompted by economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability considerations. D. An ability to function in multidisciplinary teams E. An ability to identify, formulate, and solve engineering problems F. An understanding of professional and ethical responsibility G. An ability to communicate effectively H. The broad education necessary to understand the impact of engineering solutions in a global, economic, environmental and societal context

Effective Use of E-Learning for Improving Students’ Skills

I.

A recognition of the need for, and an ability to engage in, life-long learning J. A knowledge of contemporary issues K. An ability to use the techniques, skills and modern engineering tools necessary for engineering practice.

METHODS The aim was to adapt the teaching methodology used in the modules of the University of Córdoba’s Engineering Projects section to the model set out by the European Higher Education Area. The modules in question are worth 4.5 ECTS credits (European Commission, 2009) and the contents relate to Project Management. According to the syllabus, the following student skills should be developed: A. B. C. D. E. F. G. H. I. J. K. L. M.

Adaptation to new situations Analysis of client requirements Ability to analyse and synthesise Ability to apply knowledge in practice Ability to manage information Ability to organise and plan Oral and written communication skills Estimating and programming work Design methods New technologies (ICT) Strategic organisation and planning Problem solving Decision taking

In order to acquire these abilities, a combined strategy was devised based on a variety of teaching tools, both face-to-face and non face-to-face using Moodle, such that in combination they obliged the students to study the theoretical aspect of the subject, while also requiring them to exercise the targeted skills (Salas-Morera, Berral-Yerón, Serrano-Gómez & Martínez-Jiménez, 2009). Specifically, activities comprised: online quizzes,

asynchronous discussion forums, groups tutorials and practical sessions, as well as theory classes. Table 1 sets out the knowledge and skills involved in the subject, their connection to the activities carried out, and the importance of these activities for the final mark. The last column shows the number of activities intended to foster a particular skill, and the last row shows the number of skills needed for a particular activity. According to the European Credit Transfer System (ECTS) and the Spanish regulation (Ministerio de Educación Cultura y Deporte, 2003), the average student should be able to pass the set subjects for a given academic year with a total workload (in Spain) of between 1500 and 1600 hours, which has to include all the activities to be undertaken by the students: classes, practical sessions, study, homework, tutorials, seminars etc. An academic year of 60 credits involves a workload per credit of between 25 and 30 hours. When drawing up the teaching programmes for their subjects, teachers have to take a range of factors into consideration such as: a) the activities to be undertaken by the students should be directed not only towards the acquisition of knowledge, but also towards the development of the appropriate skills, b) the student workload should be realistic, practicable in the time available in the semester and consistent with the credits awarded for the material and c) it is essential that everyone knows from the outset what tasks they have to do, when they have to do them and their bearing on the final mark, which is why a weekly task schedule was drawn up. The first challenge when scheduling activities was therefore to ensure that students were objectively able to complete the tasks within the total number of hours allotted to the subject. For that purpose, during the initial ECTS implementation phase, students were asked to complete weekly surveys in order to ascertain whether the number of working hours assigned to each activity was realistic, and whether it was consistent with the requirements of the course as

297

Effective Use of E-Learning for Improving Students’ Skills

Table 1. Activities undertaken in relation to skills and percentage weighting for the final mark FORUM (10%)

GROUP TUTORIALS (12.5%)

Theoretical contents

X

X

Adaptation to new situations

X

X

Analysis of client requirements

X

X

Ability to analyse and synthesise

X

X

ONLINE QUIZZES (25%) X

X

Estimating and programming work

3 X

3

X

2 2

X

X

4

X

X

4

X

X

X

4

X

3

X

X X

5 3

X X

X

X

X

X

Ability to organise and plan Oral and written communication skills

X

FINAL EXAMINATION (40%)

X

Ability to apply knowledge in practice Ability to manage information

PRACTICAL SESSIONS (12.5%)

Design methods

X

X

New technologies (ICT)

X

X

X

X

Strategic organisation and planning

X

X

X

X

X

5

X

X

2

X

X

4

10

10

Problem solving Decision making

X

X

10

10

a whole. Initially, 4.5 ECTS credits were assigned to each subject, so that the activities scheduled should require a total student workload of between 112 and 135 hours. The activities and sequence scheduled for the course were as follows: theoretical classes in the lecture theatre for the whole group (75-90 students, depending on the year); practical classes in the laboratory or computer room in groups of up to 25 students; an online forum devoted to the topics studied in the module with the free and simultaneous participation of all students; fortnightly feedback on forum activity through group tutorials of up to 25 students; periodical completion of online quizzes; and feedback on quiz outcomes in whole-group tutorials. Figure 1 shows the general organisation of the scheduled activities and their interconnections: the subject was divided into three theoretical blocks (T.B.) in which the teachers tackled both theory

298

6

4

and problems with the whole group. Interspersed between these sessions were five practical sessions (P.S.), conducted in small groups, whilst students were simultaneously encouraged to take part in the asynchronous discussion forum (A.D.F.), starting threads on any aspect of the subject matter that took their interest, with a view to fostering cooperation both amongst themselves and with the teacher. Six online quizzes (O.Q.) were set, at uniform intervals throughout the semester, testing familiarity with the material covered. After each set of two quizzes, a tutorial (T) was scheduled for the whole student group. Finally, forum activity was tracked through five group tutorials (G.T.), in which the students were asked to prepare for their classmates a summary of forum postings, together with further information and a critical opinion on the postings, which in turn led to new discussion. Given the importance of a realistic workload, at the start of the course a calendar

Effective Use of E-Learning for Improving Students’ Skills

of activities was published on Moodle and was updated and adjusted on a weekly basis. shows the breakdown of anticipated hours of work for each of the scheduled activities, in accordance with the number of credits assigned to the subject matter (112-135). The details of the activities were as follows:

Online Quizzes The main aim of the Moodle online quizzes is to encourage students to study on a regular basis, at a rhythm that will allow them to keep up with

the course as it progresses. The students have to complete six quizzes, two per theoretical block, in such a way that the first quiz covers the first half of each block while the second covers all the material studied up until that point, not just in that particular block but from the start of the semester. Whenever a quiz comes round, the students have four days in which they are free to complete it at any time, on condition that once they open it they have to finish it within 40 minutes. The total time allotted for preparing for the quiz, revising the necessary concepts and completing it was two hours. Quizzes comprised ten to fifteen questions

Figure 1. General flowchart of scheduled activities and links between them

299

Effective Use of E-Learning for Improving Students’ Skills

Table 2. Distribution of students’ workload Assigned time (hours)

Face-to-face activities Whole group activities

Assigned time (hours)

Individual and virtual activities Online quizzes and preparation (O.Q.)

12

Theoretical Blocks & Problem Solving (T.B.)

33

Asynchronous Discussion Forums (A.D.F.)

10

Tutorial Sessions (T.)

6

Studying

30

Homework

10

Groups of up to 25 students Practical Sessions (P.S.)

12

Personal Tutoring

1

Group Tutorial Sessions (G.T.)

9

Total individual and virtual activities estimated time

63

Total face-to-face time assigned

60

Final Examination

5

Students’ total working time (60+63+5)

which might involve multiple choice, matching and numerical answers – if the latter, a certain margin of error was allowed. Each question was awarded a maximum score of one point, with a penalty for every incorrect option chosen. Each quiz comprised several blocks of questions, set jointly by all the teachers; questions varied in difficulty, and there were sufficient questions to ensure that when randomly assigned to students, each student would have a different quiz. Questions were also programmed to appear in random order, and choices for each question were also presented randomly. The quizzes were not particularly difficult, and their content related to concepts that were basic to the subject; the students therefore needed to study and revise these concepts at least with the same frequency that the teacher scheduled the quizzes, with great benefits for the development of other activities. Once the students had completed the two quizzes for each block, the teacher obtained from Moodle a file containing a statistical analysis of student answers, which he then analysed in order to identify the main weaknesses; afterwards, a 2-hour tutorial was held with the whole group, in which the tested material was discussed, paying special attention to those areas in which the poorest results were obtained.

300

128

Asynchronous Discussion Forum and Group Tutorials The asynchronous discussion forum proved to be among the most productive of the tools tested in these courses, in relation to the effort required. All group members were required to register on the forum, and messages were automatically tracked by e-mail, so that all users were immediately and almost effortlessly informed of new developments as they appeared. The main aim of the forum was to maintain contact and keep the group’s attention focused on the themes raised by the subject matter. The contact is extremely fluid and effective, and helps communication extend much further than in the face-to-face sessions, enabling students to raise doubts and make constructive contributions. For their part, teachers were able to take stock of the way knowledge is developing within the group, allowing them to direct their students’ work from a distance in a way that was virtually unnoticeable. The teacher’s role in regard to student forum postings was twofold: when a student asked a question or put forward a subject for debate, the teachers kept to the sidelines in the hope that the students would come up with an answer between themselves, as freely and spontaneously as possible, intervening to make minor suggestions or encourage others to join in. Secondly, if the teacher noticed any drop-off in forum postings

Effective Use of E-Learning for Improving Students’ Skills

over a certain period, he suggested new subjects for discussion, thus encouraging students to seek information on additional aspects of the subject. Subjects suggested by teachers included: ACM/ IEEE Computing Curricula, ethical regulations in the engineering professions, and professional activities in general. It was considered important not to base evaluations of forum activity on the number of contributions from each student, as this would encourage a flood of low-quality, irrelevant contributions; the students were informed of this fact. Thus the evaluation of forum content, although it required more work on the part of the teaching staff, was holistic in nature, awarding a higher score to the most relevant contributions and lower scores to the least relevant. Although Moodle gives the possibility of evaluating forum contributions individually, an external spreadsheet was used, so each student was evaluated for this aspect globally at the end of the semester. Forum activity did not take place in isolation from the rest of the activities in the course, but rather as a reinforcement of all the other activities, in its role as the default communication tool. This role was further enhanced by what are known as group tutorials (G.T.). The aim of these tutorials was to enable students to work together on the subjects discussed in the forum over the previous weeks. To do this, students needed to make use of their analytical and synthetical skills, their ability to manage information, their ability to express themselves clearly in speaking and writing, and their planning and organisational skills. As in the collective tutorials for the quizzes, the pooling of views on the subjects discussed in the forum helped to highlight the most important concepts and, often, to address mistakes which might otherwise have been impossible to detect. For the purposes of this activity, the main group was divided into four groups of up to 25 students, and in turn each of these was sub-divided into four subgroups of four to five students. Group and subgroup divisions were made on the basis of affinity between

students at the beginning of the year. There were five discussion sessions, organised as follows: At the beginning of the year there was a onehour session with each group of up to 25 students. In this session students were asked to introduce themselves to other members of the group, explaining their reasons for choosing their degree subject and outlining their academic career to date. There was also an initial survey in which students were asked about the extent to which they believed they have already acquired the skills targeted by the module, as well as the importance they attached to these skills for their future careers. Four two-hour sessions were held with each of the groups of up to 25 students, distributed uniformly throughout the semester. Prior to each session, one of the four subgroups was asked to monitor the forum and prepare a handout with an outline of the most important themes, which they discussed with the teaching staff. Once the teacher had approved the summary as an adequate and properly organised reflection of the topics emerging in the forum, the subgroup prepared a Microsoft Office or OpenOffice presentation, which was also sent to the teacher for approval. Finally, in the group tutorial session, they gave the presentation, which lasted no more than 40 minutes, to the rest of the group; the remainder of the 2-hour session was devoted to a group discussion of the subjects addressed in the presentation. Assessment of this session was carried out jointly by the teacher (50%) and the other students in the group (50%). The following points were assessed: presentation quality; appropriateness of material (joint ratings for all subgroup members); oral expression during the presentation; and oral expression during the discussion (the last two points were rated individually).

Other Assessable Activities The remaining assessable activities comprised six two-hour practical sessions in the laboratory or computer room in small groups. Prior to each

301

Effective Use of E-Learning for Improving Students’ Skills

session, a handout for the session was published on Moodle, which dealt with problem-solving using specific software (for example Microsoft Project or OpenProject). In the course of the week following each activity, the students uploaded a short report onto Moodle regarding the results of the practical session. The assessment was completed with a final examination.

RESULTS AND DISCUSSION To evaluate the experiment the following tools were used: weekly surveys of student workloads; analysis of quiz results; analysis of forum activity; academic result of group tutorials; surveys on perceived competences before and after the course; survey of students’ opinions on all subjectrelated activities; comparison of overall academic performance.

Student Workload Surveys Over the last four years, weekly surveys were carried out through Moodle in order to determine the actual student workload in each of the activities. Every week, students were asked to indicate: 1) time spent revising and doing the online quiz; 2) time spent on the discussion forum; 3) time devoted to private study; 4) time spent drafting reports; and 5) time spent attending personal tutorials. Participation in these surveys was completely voluntary, and had no impact on the overall mark. Table 3 shows the average results for the weekly surveys carried out during the academic years 2006-2007 to 2009-2010. With regard to revising for and doing quizzes, the fact that dates were scheduled beforehand meant that time spent on revising for and actually doing the quizzes was relatively uniform throughout the course, although at times the student workload for this activity increased considerably. In the initial subject schedule, each quiz was intended to take a total of 2 hours; by contrast, survey results showed that

302

the average student spent over 18 hours, rather than a total of 12 hours, doing the 6 quizzes. In future, therefore, the time to be spent on each quiz may need to be revised from two to three hours. By contrast, the time students claimed to spend on the forum approached the ten hours anticipated, i.e. some 40 minutes per week. Moreover, the average weekly time remained extremely uniform over the 15 weeks of the course. Weekly study time maintained a steady pattern of around three hours in the first weeks, subsequently rising to a maximum of seven hours as the final examination approached; similar findings were observed for the time spent writing reports, which increased in the weeks when reports were to be handed in. Students spent strikingly little time on personal tutorials. There may have been several reasons for this: timetables for classes and other activities were so demanding that students had no time to make individual appointments with the teacher; moreover, the growing use of virtual tutorials via the forum contributed to a decline in individual tutorials, which were only requested by students with specific problems that they did not wish to share with the rest of the class via the discussion forum. The difference between the times anticipated in the initial schedule and those actually reported by the students was very striking. The 104 hours devoted to non face-to-face activities, plus the 60 devoted to face-to-face activities, gave an actual total of 164, compared to a total time of 128 hours. This difference between the real and actual times explains why the students constantly complained about the workload they had to sustain, not only in this module but also in all the others making up the subject, and may account, at least in part, for the high failure rate observed in these subjects.

Analysis of Quiz Results Table 4 shows the results of the quizzes from 2006-2007 to 2009-2010. The most remarkable finding was the difference between the number

104.29

0.24 0.02

6.95 10.51

0.00 0.00

11.03 9.96

0.01 0.00

10.32 7.04

0.05 0.01

6.56 6.80 7.17

0.02

0.01

0.06

5.21

2.11 1.77

3.94

0.92

1.17

3.69

1.58 1.63

3.96

3.39

3.69

0.99

2.01 0.71

0.58 0.58 0.48

0.72 1.46

0.65 0.54

1.03

1.16

6.21

17.53

58.63 3.91

1.17 1.67

7.43 6.78

2.34 1.57

5.43

9.30

18.58 1.24

0.62 0.48

0.93 1.26

0.66 0.64

2.32

13 12 11 10 9 8 7

0.00

5.45 6.53

0.06 0.01

6.05 6.02

0.01

1.07 TOTAL

3.56

0.00 Personal tutorial

0.02

3.02

0.58 0.82

3.45 3.06

0.88 0.29

3.20

0.08 Homework

0.12

0.42 Studying

1.96

0.74 0.62

1.58 1.36

0.75 0.55

1.98

0.52

0.92 0.04

Forum

0.54

1.11

6 5 4 3

Online quizzes and preparation

2 1 WEEK Nº

Table 3. Average weekly time devoted to each activity in the academic years 2006-2007 to 2009-2010

14

15

AVG.

TOTAL

Effective Use of E-Learning for Improving Students’ Skills

of matriculated students (between 90 and 64) and the number of students actually completing the quizzes. Every year, some students failed to complete a single quiz, something which certainly merits detailed study in its own right. The number of “efficient” students, i.e. students that completed at least four of the six quizzes was 77, 57, 64 and 54 respectively. It was noticeable that the average mark in each year (out of a maximum score of ten) was relatively high, as was the number of students passing the activity, particularly in comparison with the number of students participating, suggesting that this activity makes a significant contribution to the subject pass rate. The activity’s contribution to overall learning outcomes will be examined later. With regard to the existence of significant differences among years results, ANOVA analysis have been made both, for the overall quizzes scores (O.Q.S.) and for every individual quiz; and significant differences were only found in quizzes nº4 and nº6 of the forth year with respect to the others. Nevertheless, there is not apparent reason for this differences, more when O.Q.S. are similar.

Analysis of Forum Activity Changes in forum activity between academic years 2004-2005 and 2009-2010 are charted in Table 5, which shows that student participation in the forum increased from 2007-2008, the first year that this activity was included in the overall mark; a significant increase was noticed not only in the number of threads started by the students but also in the number of responses, indicating that the awarding of a mark greatly encouraged participation. Comparing the number of threads started with the number of participating students of the previous section gave averages of 0.47, 1, 1.55 and 1.33 threads started per student, which tended to reinforce the claim that the forum generated a growing interest among students over the years.

303

Effective Use of E-Learning for Improving Students’ Skills

Table 4. Quiz results for the four study years YEAR 06/07 90 STUDENTS

YEAR 07/08 64 STUDENTS

1

2

3

4

5

6

O.Q.S.

1

2

3

4

5

6

O.Q.S.

N

83

84

80

77

76

71

90

56

62

50

53

55

59

64

MAX

10,00

8,50

9,60

8,80

9,30

10,00

8,83

8,30

10,00

9,60

9,30

9,30

9,00

7,97

MIN

1,00

2,10

3,00

2,80

2,30

4,00

0,00

2,20

2,60

1,70

3,30

3,80

2,00

0,00

MEDIAN

7,50

5,65

7,55

6,00

8,60

8,00

6,37

6,30

6,05

7,20

6,10

8,60

7,00

6,38

MEAN

7,34

5,56

7,25

6,06

7,55

7,79

6,02

6,08

5,88

6,77

6,09

7,85

7,08

5,77

PASSES

71

52

76

56

61

70

73

45

45

42

41

48

56

47

S.D.

2,01

1,40

1,44

1,52

2,17

1,17

2,07

1,60

1,63

1,70

1,43

1,77

1,59

1,86

1

2

3

4

5

6

O.Q.S.

1

2

3

4

5

6

O.Q.S.

N

62

60

59

63

63

63

78

65

64

59

54

56

55

70

MAX

8,40

9,30

9,70

9,30

9,70

9,00

8,38

8,40

9,60

8,80

9,80

9,60

10,00

7,90

MIN

1,00

2,40

2,80

1,50

3,90

3,00

0,00

2,20

0,80

2,70

0,10

4,00

4,00

0,00

MEDIAN

5,70

6,30

8,00

8,00

7,60

7,00

6,00

5,70

5,90

6,50

4,74

8,20

8,00

5,55

MEAN

5,58

6,02

7,50

7,03

7,38

7,03

5,34

5,54

5,69

6,50

4,80

7,80

7,82

5,32

PASSES

46

41

52

47

58

61

54

39

41

51

21

50

54

45

S.D.

1,55

1,80

1,71

2,26

1,38

1,22

2,71

1,42

2,09

1,33

2,40

1,48

1,17

1,67

YEAR 08/09 78 STUDENTS

YEAR 09/10 70 STUDENTS

Similarly, comparison of the number of answers with the number of participating students in each year yielded figures of 1.18, 3.23, 7.50 and 7.57, further supporting this hypothesis. The quality of contributions also improved over time, with the students themselves realising that off-topic or repetitive responses were unlikely to receive good marks. Thus the forum proved a useful tool in focusing students’ attention on the topics

targeted by the module, leading to contributions that gradually become more and more relevant.

Academic Results for Group Tutorials The group tutorial activity brought together other activities in the module through the questions emerging on the forum. It also helped to foster the most important skills in the area, such as ability

Table 5. Discussion forum activity, 2004-2005 to 2009-2010 YEAR

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

Subjects started by the teacher (A)

25

54

26

35

46

55

Total number of replies to subjects started by the teacher (B)

48

136

33

52

193

210

Replies per subject started by the teacher (A/B)

1.92

2.52

1.27

1.49

4.20

3.82

Subjects started by students (C)

56

80

36

57

99

72

Total number of replies to subjects started by students (D)

140

198

91

184

480

409

Replies per subject started by students (C/D)

2.50

2.48

2.53

3.23

4.85

5.68

304

Effective Use of E-Learning for Improving Students’ Skills

to analyse and synthesise, to manage information, and to organise and plan work, as well as speaking and writing skills. Understandably, therefore, this activity was given core status when organising tuition. Moreover, all students were given shared responsibility in the evaluation of tutorials. The following were assessed: presentation quality; appropriateness of material (joint ratings for all subgroup members); oral expression during the presentation; and oral expression during the discussion (the last two points are rated individually). Both the teacher and the other students in the group assessed all members of the subgroup on each of the aspects mentioned above, and the overall mark was derived (50%-50%) from the teacher’s score and the students’ score. As Figure 2 shows, scores awarded by the teacher covered a wider range, whilst student scores tended to be concentrated in a narrower range between six and nine points. The first question that tends to arise when peer assessment is used is whether students are more lenient or more demanding with their colleagues than the teacher is, and whether there is a significant difference between scores obtained in this way compared to scoring performed solely by the teacher. An independent-samples t-test was conducted in order to know whether there are statistically significant differences between scores assigned by teacher and scores assigned by students in 2008-2009 and 2009-2010. In the first year a significant difference was found between scores assigned by teacher (M1=7.21; SD1=1.67) and by students (M2=6.73; SD2=0,90); t(98)=2.03, p=0.045, so students were more demanding with their classmates than the teacher was. In 2009-2010, however, the hypothesis of equality of means was accepted (M1=7.08; SD1=1.29; M2=7.30; SD2=0.55; t(83)=-1.25, p=0.21), so the student score not being significantly higher than the teacher-awarded score. Thus no reliable conclusions can be drawn from these findings, suggesting that a further sample needs to be taken in order to reach firmer conclusions.

Similarly, the teacher’s score was compared to the mark finally awarded to the students. Here there was no statistically significant difference in the two years being investigated (M1=7.21; SD1=1.67; M2=6,97; SD2=1.19; t(116)=0.95, p=0.35); (M 1 =7.08; SD 1 =1.29; M 2 =7.38; SD2=0.76; t(99)=-1.56, p=0.12), indicating that the students’ participation did not have a significant influence on the mark.

Self-Assessed Skills Surveys at the Beginning and End of the Course In order to ascertain the extent to which the experiment improved students’ skills, and determine the importance the students attached to these skills for their professional careers, surveys were carried out in 2008-2009 and 2009-2010, at the beginning and end of the year, using a scale of 1-5. The students were asked to rate the importance they attached to these skills for their future professional careers and the extent to which they thought they had already acquired them (Table 6). The most striking finding was that in both two years there was an across-the-board increase in students’ perceptions of their own skills, indicating that working method achieved its goals to a considerable degree, at least in terms of students’ perceptions. At the outset, the competence considered by students as being least important for their professional career was Design Methods; the student grading of this competence by the end of the year had increased more than for any other competence; in the second year, it was the competence recording the second-greatest increase in student grading. This could be due to the fact that across the rest of the curriculum for this degree a good deal of emphasis was given to technical aspects of engineering and to related problem-solving, but hitherto there had been no overview of design in engineering and related projects. Meanwhile, the skill showing least improvement in both years was New Technologies (ICT), possibly because the students considered that they had already developed considerable abili-

305

Effective Use of E-Learning for Improving Students’ Skills

Figure 2. Group tutorial scores in 2008-2009 and 2009-2010

ties in this area over the course of their studies and that it was now difficult for them to show further improvement. Finally, in both years the subject considered a priori most important by students was Decision-Making, thus reflecting the general approach of the degree, which is strongly oriented towards project performance and management. When the means at the start and at the end of each year were statistically compared by means of t-test, significant differences were found in that cases marked with bold characters and shading in Table 6, what clearly remarks the success of the experience in this aspect.

306

Students’ Opinions Regarding the Value of the Experiment From 2004-2005 onwards, students were asked at the end of the semester to grade the statements listed in Table 7. After ANOVA tests, significant differences between years were found in cases marked with bold characters and shading in Table 7. The most striking finding was the effect of including participation in the discussion forum (items 5 and 6) as part of the final mark in the last three years. The significant increase in the student’s interest in making interesting and constructive contributions to discussion can only be due to this factor. Over the same period, there was a similar increase in the students’ perception of teacher availability (9), which again may be largely due to forum activity being evaluated. The

Effective Use of E-Learning for Improving Students’ Skills

Table 6. Student perception of skills at the beginning and end of each year ACADEMIC YEAR 2008-2009 IMPORTANCE SD

START OF YEAR mean

SD

ACADEMIC YEAR 2009-2010

END OF YEAR

IMPORTANCE

mean

mean

mean

Adaptation to new situations

4.23

1.03

3.54

0.86

3.63

0.73

4.47

0.65

3.05

0.67

3.93

0.54

Analysis of client requirements

4.23

0.82

3.46

0.81

3.79

0.72

4.47

0.68

2.62

0.87

3.78

0.68

Ability to analyse and synthesise

4.12

0.82

3.54

0.71

3.68

0.73

4.32

0.68

2.93

0.66

3.93

0.60

Ability to apply knowledge in practice

4.04

0.96

3.42

0.86

3.97

0.80

4.52

0.62

3.18

0.75

4.00

0.77

Ability to manage information

4.08

0.80

3.24

0.78

3.74

0.68

4.27

0.71

3.12

0.69

3.85

0.65

Ability to organise and plan

4.27

0.87

3.73

0.92

4.08

0.89

4.57

0.59

2.90

0.84

4.26

0.70

Oral and written communication skills

4.23

0.99

3.38

0.98

3.82

0.98

4.60

0.72

2.83

0.83

3.63

0.82

Estimating and programming work

4.08

0.80

3.62

0.85

3.82

0.78

4.38

0.74

2.87

0.87

4.07

0.81

Design methods

3.58

0.70

3.04

0.72

3.66

0.70

4.02

0.72

2.47

0.68

3.70

0.85

New technologies (ICT)

4.23

0.99

3.50

1.10

3.56

0.84

4.45

0.70

3.12

0.85

3.52

0.88

Strategic organisation and planning

4.27

0.92

3.42

0.76

3.74

0.84

4.32

0.65

2.70

0.70

3.89

0.83

Problem solving

4.04

1.15

3.54

0.86

3.87

0.72

4.60

0.56

3.18

0.77

3.81

0.67

Decision making

4.27

1.15

3.73

1.00

4.00

0.91

4.65

0.58

3.17

0.85

4.04

0.81

Academic Performance Final student marks for the last ten academic years are shown in Figure 3 under three headings: passed, failed and missed. In the first period, 2000-2001 to 2003-2004, traditional teaching methods were used, with theoretical-practical classes and final examinations as the only method of evaluation.

SD

mean

SD

END OF YEAR

COMPETENCE

other eye-catching features were the high values assigned by students to the questionnaires carried out periodically (8) and the utility of learning resources available in Moodle (7). The remainder of the topics in the survey attracted high scores, although there were no significant differences between academic years.

SD

START OF YEAR

mean

SD

During this period there was a marked drop in the number of students passing the study subjects. This led to an overhaul of teaching methods in 2004-2005, taking advantage of the implementation of the ECTS Pilot Plan. For the first time, all the activities to be carried out by students (rather than just attendance at lectures) were scheduled in detail; the programme was fine-tuned by weekly feedback as the course progressed. The aim was to ensure that all students were kept constantly informed of the activities scheduled for the following weeks, thus enabling them to plan their work in good time. At the same time, a discussion forum was included on Moodle, although this forum did not initially contribute to the final mark; finally, online quizzes were scheduled.

307

Effective Use of E-Learning for Improving Students’ Skills

Table 7. Student views on scheduled activities, 2004-2005 to 2009-2010 Statement

04-05

05-06

06-07

07-08

08-09

09-10

1.- Information on subject teaching and assessment methods was sufficient and proved useful

3.98

3.92

3.75

3.92

4.11

4.29

2.- I think that what I learned in this subject will be useful to me in my professional life

3.89

3.88

4.11

3.96

4.18

4.19

3.- The problems studied helped to imagine real-life practical situations

3.98

3.94

4.09

3.98

3.95

4.16

4.- The problems studied helped to reinforce theoretical concepts

4.27

4.47

4.36

4.35

4.47

4.33

5.- The forum was useful for sharing information and expressing views on subject content

3.00

3.65

2.84

4.15

4.30

4.13

6.- The forum was useful for making concepts clearer

3.94

3.63

3.18

4.05

4.00

4.00

7.- The learning resources available on Moodle were useful

4.25

4.47

4.09

4.29

4.23

4.23

8.- The quizzes were useful as a way of keeping up with the subject

4.04

4.04

4.16

4.08

4.20

4.06

9.- The teacher was available for queries (albeit online/by e-mail)

3.96

4.50

4.02

4.15

4.39

4.58

10.- The teacher encouraged class participation

3.79

4.16

4.09

4.00

4.07

4.13

In the first year of implementation of the ECTS credits system, there was a notable increase in pass rate as a positive consequence of the detailed scheduling but mainly as a consequence of teachers analysing their students’ workloads, something that up until then had not been taken into account in time-tabling. However, over the next two years (2004-2005 and 2005-2006) the number of failures again rose, perhaps because more teachers changed their teaching methods, leading students to complain about the large number of tasks they were expected to perform simultaneously, and about the lack of coordination among subjects. Given this situation, and with the aim of encouraging even greater participation in the forum, an activity that had proved extremely useful in the judgement of both the teachers and the students themselves, from 2007-2008 10% of the overall mark was allotted to assessment of participation in the forum. The forum came to be one of the most useful and valued activities, both for the students, who appreciated its immediacy and ease of use, and for the teachers, who valued

308

it as a way of answering questions and clarifying grey areas, since a single reply served to disseminate information and explanations among all students, instantly. Prior to that, participation in the forum had been considered supplementary and voluntary, when in reality it had gradually become indispensable. As soon as forum activity was included in the final mark, all the activities required of the students were incorporated into the assessment, allowing teachers to calculate the students’ workload more accurately, and enabling students to adapt their efforts to reflect the percentage of the final mark represented by each activity. The result was a substantial increase in forum participation, as well as an enhancement of its value, thanks to an improvement in the quality of student postings. There was again an increase in the number of passes in that academic year, confirming that the changes had been appropriate. Finally, in 2008-2009 and 2009-2010 the group tutorial activity was added, in which groups of up to 25 students discussed topics raised in the forum in preceding weeks. The aim of this activity was

Effective Use of E-Learning for Improving Students’ Skills

to foster the basic skills required by graduates, as well as linking with other course activities; the introduction of this activity prompted a new increase in pass levels.

CONCLUSION In recent years a combination of teaching strategies including asynchronous discussion forums, online quizzes and group tutorials, has been implemented in Industrial Engineering and Software Engineering degree courses with the aim of improving some students’ skills needed in Engineering, such as ability to organise and plan, oral and written communication skills, estimating and programming work, design methods, strategic organisation and planning, problem solving and decision making, among others. These strategies have prompted greater skills acquisition, as well as enabling students to regulate their workload. Surveys carried out at the end of each year showed that students considered the forum a very useful means of exchanging information in general and clarifying concepts included in the course content. Participation in the forum, and also the quality of postings, rose substantially from the moment that assessment of this activity started to contribute to the overall mark for the course. Perhaps as a result of greater use of the forum, there was also an improvement in the way students

perceived the accessibility of their teachers, even if such accessibility was only online. Teacher participation in the forum is important and needs to fulfil a dual role: first it should motivate students to participate and exchange information, expressing appreciation for contributions and encouraging others to take part; secondly it should be used to put forward new discussion topics when forum activity abates. Students are seen to be more diffident about participating when it is the teacher who starts a discussion; it is therefore advisable for the teacher to keep to the sidelines, intervening only to correct mistakes and encourage general participation rather than actually leading discussions. Weekly monitoring of student workloads has shown that the average effort expended has exceeded that anticipated by the teachers in their scheduling in every one of the years under investigation. This finding lends support to arguments in favour of making future schedules more realistic, something that should also have a positive influence on overall results. Weekly monitoring of scheduling has also proved very useful in motivating students to keep up with the activities in a comprehensive fashion, completing them in time, as planned by their teacher. However, while the activities as a whole are designed to help the students gradually acquire the knowledge and skills needed for the subject, it was found – every year – that the number of hours of study increased substantially in the final weeks before the examination. It may

Figure 3. Final student marks as a function of the introduction of new teaching and evaluation methods

309

Effective Use of E-Learning for Improving Students’ Skills

therefore be necessary to pay more attention to this aspect, in order to enable students to work more effectively and spread their workload more uniformly over the time available. In all study years, students reacted positively towards completing the online quizzes, indicating that quizzes helped them to keep up to date in the subject. Teachers displayed considerable regard for this tool in terms of the ratio between effort expended and results obtained; a few hours of correctly-focussed work at critical junctures was found to help consolidate subject content as well as contributing to the progress of both theoretical and practical classes. Moreover, because quizzes are scheduled far enough in advance and can be completed within a flexible time frame, they neither interfere with the normal development of the teaching material nor take up students’ attention beyond what is absolutely essential. Group tutorials are a core activity in the new teaching methodology. They enable the work of the group to focus on the topics discussed in the forum, and they actively contribute to the development of skills such as the ability to manage information, to analyse and synthesise, as well as enhancing oral and written expression and other competences. Peer assessment of this activity consolidates the responsibility students feel towards themselves as well as towards their colleagues, even though they tended to award marks within a narrow band between 6 and 9 points. However, in the two study years, peer scores was not decisive for the final marks obtained, given that there were no significant differences between the final marks and those awarded by the teachers alone, which indicates that the assessment system needs to be revised in the future. According to students, skills improved substantially in each study year. The skills showing most improvement were Design Methods and Organisation and Planning Ability; this is consistent with the general philosophy of the content of the Projects module and the way the activities were planned.

310

Future prospects of research deal with improving activity planning, making it more realistic with the students’ workload, analysing the real effect of co-evaluation, not only on the final marks but also on the students’ attitude in facing the academic activities, and finally, searching for new ways of evaluating skill acquisition.

REFERENCES ABET. (2009). Criteria for accrediting engineering programs. Accreditation Board for Engineering and Technology, 29. Retrieved November 22, 2010, from http://www.abet.org/Linked%20 Documents-UPDATE/Criteria%20and%20PP/ E001%2010-11%20EAC%20Criteria%201-2710.pdf Andresen, M. A. (2009). Asynchronous discussion forums: Success factors, outcomes, assessments, and limitations. Journal of Educational Technology & Society, 12(1), 249–257. Angus, D. S., & Watson, J. (2009). Does regular online testing enhance student learning in the numerical sciences? Robust evidence from a large data set. British Journal of Educational Technology, 40(2), 255–272. doi:10.1111/j.14678535.2008.00916.x Banks, S., Goodyear, P., Hogson, V., & McConnell, D. (2003). Introduction to the special issue on advances in research on networked learning. Instructional Science, 31, 1–6. doi:10.1023/A:1022583918064 Barrella, E., Simmons, J., & Buffinton, K. (2006). Professional engineering education best practice study for 1st-year, multi-disciplinary courses. ASEE Annual Conference and Exposition, Conference Proceedings (10). De Laat, M., & Lally, V. (2003). Complexity, theory and praxis: Researching collaborative learning and tutoring processes in a networked learning community. Instructional Science, 31, 7–39. doi:10.1023/A:1022596100142

Effective Use of E-Learning for Improving Students’ Skills

De Laat, M., Lally, V., Lipponen, L., & Simons, R. J. (2007). Online teaching in networked learning communities: A multi-method approach to studying the role of the teacher. Instructional Science, 35, 257–286. doi:10.1007/s11251-006-9007-0 Dennen, V. P. (2008). Looking for evidence of learning: Assessment and analysis methods for online discourse. Computers in Human Behavior, 24, 205–219. doi:10.1016/j.chb.2007.01.010 Dobson, J. L. (2008). The use of formative online quizzes to enhance class preparation and scores on summative exams. Advances in Physiology Education, 32, 297–302. doi:10.1152/advan.90162.2008 Erlin, Y. N., & Rhaman, A.A. (2009). Students’ Interactions in Online Asynchronous Discussion Forum: A Social Network Analysis. International Conference on Education Technology and Computer, 25-29. Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., & Mong, C. (2007). Using peer feedback to enhance the quality of student online postings: An exploratory sStudy. Journal of Computer-Mediated Communication, 12, 412–433. doi:10.1111/j.10836101.2007.00331.x European Commission. (2009). ECTS users’ guide. Luxembourg: Office for official publications of the European communities, p. 60. Retrieved July 19, 2010, from http://ec.europa. eu/education/lifelong-learning-policy/doc/ects/ guide_en.pdf Hew, K. F., & Cheung, W. S. (2008). Attracting student participation in asynchronous online discussions: A case study of peer facilitation. Computers & Education, 51, 1111–1124. doi:10.1016/j. compedu.2007.11.002 Kakasevski, G., Mihajlov, M., Arsenovski, S., & Chungurski, S. (2008). Evaluating usability in learning management system Moodle. Proceedings of the ITI 2008 30th Int. Conf. On Information Technology Interfaces (pp. 613-618).

Kibble, J. (2007). Use of unsupervised online quizzes as formative assessment in a medical physiology course: Effects of incentives on student participation and performance. Advances in Physiology Education, 31, 253–260. doi:10.1152/ advan.00027.2007 Kirkwood, A. (2009). E-learning: You don’t always get what you hope for. Technology, Pedagogy and Education, 18(2), 107–121. doi:10.1080/14759390902992576 Kirner, T. G., Custódio, C. A., & Kirner, C. (2008). Usability evaluation of the Moodle system from the teachers’ perspective. IADIS International Conference e-Learning (pp. 371-378). Kozaris, I. A. (2010). Platforms for e-learning. Analytical and Bioanalytical Chemistry, 397(3), 893–898. doi:10.1007/s00216-010-3587-x Lahwal, F., Amaimin, M., & Al-Ajlan, A. (2009). Perception cultural impacts: Principles for trainers’ skills for e-learning. NCM 2009 - 5th International Joint Conference on INC, IMS, and IDC (Article number 5331619, pp. 986-993). Lan, Y. F., & Yan, C. L. (2009). A practical approach to encourage students’ participation in asynchronous online discussions based on expectancy theory. International Conference on Virtual Environments, Human-Computer Interfaces and Measurements Systems (Article number 5068907, pp. 271-276). Mazzolini, M., & Maddison, S. (2003). Sage, guide or ghost? The effect of instructor intervention on student participation in online discussion forums. Computers & Education, 40, 237–253. doi:10.1016/S0360-1315(02)00129-X McPherson, M. C., & Nunes, J. M. (2008). Critical issues for e-learning delivery: What may seem obvious is not always put into practice. Journal of Computer Assisted Learning, 24(5), 433–445. doi:10.1111/j.1365-2729.2008.00281.x

311

Effective Use of E-Learning for Improving Students’ Skills

Ministerio de Educación Cultura y Deporte. (2003). REAL DECRETO 1125/2003, de 5 de septiembre, por el que se establece el sistema europeo de créditos y el sistema de calificaciones en las titulaciones universitarias de carácter oficial y validez en todo el territorio nacional. Boletín Oficial del Estado. Nº 224, 34355-34356. Retrieved November 6, 2010, from http://www.boe.es/boe/ dias/2003/09/18/pdfs/A34355-34356.pdf Nevgi, A., Virtanen, P., & Niemi, H. (2006). Supporting students to develop collaborative learning skills in technology-based environments. British Journal of Educational Technology, 37(6), 937–947. doi:10.1111/j.1467-8535.2006.00671.x Rehman, H. U., Said, R. A., & Al-assaf, Y. (2009). An integrated approach for strategic development of engineering curricula: Focus on students’ design skills. IEEE Transactions on Education, 52(4), 470–481. doi:10.1109/TE.2008.930508 Rovai, A. P. (2008). Facilitating online discussions effectively. The Internet and Higher Education, 10, 77–88. doi:10.1016/j.iheduc.2006.10.001 Salas-Morera, L., Berral- Yerón, J., SerranoGómez, I., & Martínez-Jiménez, P. (2009). An assessment of the ECTS in software engineering: A teaching experience. IEEE Transactions on Education, 52(1), 177–184. doi:10.1109/ TE.2008.921801 Savander-Ranne, C., Lundén, O. P., & Kolari, S. (2008). An alternative teaching method for electrical engineering courses. IEEE Transactions on Education, 51(4), 423–431. doi:10.1109/ TE.2007.912500 So, H. J. (2009). When groups decide to use asynchronous online discussions: Collaborative learning and social presence under a voluntary participation structure. Journal of Computer Assisted Learning, 25, 143–160. doi:10.1111/j.13652729.2008.00293.x

312

Tuparova, D., & Tuparov, G. (2010). Management of students’ participation in e-learning collaborative activities. Procedia Social and Behavioral Sciences, 2, 4757–4762. doi:10.1016/j. sbspro.2010.03.764 Vivek, S., & Ansari, Y. (2010). Collaborative learning issues in synchronous online interactions. International Conference on Technology for Education (T4E), 5-8. Whitelock, D. (2009). Editorial: E-assessment: Developing new dialogues for the digital age. British Journal of Educational Technology, 40(2), 199–202. doi:10.1111/j.1467-8535.2008.00932.x Yau, J., Lam, J., & Cheung, K. S. (2009). A review of e-learning platforms in the age of e-learning 2.0. 2nd International Conference on Hybrid Learning and Education (ICHL 2009), LNCS 5685 (pp. 208-217). Zhao, J., & Yiang, Y. (2008). A case study on collaborative e-tutoring. Proceedings of 2008 IEEE International Symposium on IT in Medicine and Education (Article number 4743903, pp. 439-444). Zhou, Z. (2004). From classrooms to the real engineering world: The training program in the microelectronics research center at Georgia Tech. IEEE Transactions on Education, 47, 188–120. doi:10.1109/TE.2003.822262

ADDITIONAL READING An, H., Shim, S., & Lim, K. (2009). The effects of different instructor facilitation approaches on students’interactions during asynchronous online discussions. Computers & Education, 53, 749–760. doi:10.1016/j.compedu.2009.04.015 Armellini, A., & Aiyegbayo, O. (2010). Learning design and assessment with e-tivities. British Journal of Educational Technology, 41(6), 922–935. doi:10.1111/j.1467-8535.2009.01013.x

Effective Use of E-Learning for Improving Students’ Skills

Badcock, P. B., Pattison, P. E., & Harris, K. L. (2010). Developing generic skills through university study: A study of arts, science and engineering in Australia. Higher Education, 60, 441–458. doi:10.1007/s10734-010-9308-8

Hew, K. F., & Cheung, W. S. (2010). Higher-level knowledge construction in asynchronous online discussions: An analysis of group size, duration of online discussion, and student facilitation techniques. Instructional Science, 17.

Ben Ammar, M., Neji, M., Alimi, A. M., & Gouardères, G. (2010). The affective tutoring system. Expert Systems with Applications, 37, 3013–3023. doi:10.1016/j.eswa.2009.09.031

Ileh, M. T., & Saleh, I. (2009). Case based reasoning approach of tutoring in e-Ieaming platform. International Conference on Multimedia Computing and Systems -Proceedings (Art. no. 5256711, pp. 150-157).

Benson, A., Lawler, C., & Whitworth, A. (2008). Rules, roles and tools: Activity theory and the comparative study of e-learning. British Journal of Educational Technology, 39(3), 456–467. doi:10.1111/j.1467-8535.2008.00838.x Bye, L., Smith, S., & Monghan Rallis, H. (2009). Reflection using an online discussion forum: Impact on student learning and satisfaction. Social Work Education, 28(8), 841–855. doi:10.1080/02615470802641322 Calvani, A., Fini, A., Molino, M., & Ranieri, M. (2010). Visualizing and monitoring effective interactions in online collaborative groups. British Journal of Educational Technology, 41(2), 213–226. doi:10.1111/j.1467-8535.2008.00911.x Choi, H., & Kang, M. (2010). Applying an activity system to online collaborative group work analysis. British Journal of Educational Technology, 41(5), 776–795. doi:10.1111/j.14678535.2009.00978.x Dermo, J. (2009). E-assessment and the student learning experience: A survey of student perceptions of e-assessment. British Journal of Educational Technology, 40(2), 203–214. doi:10.1111/j.1467-8535.2008.00915.x Dringus, L. P., & Ellis, T. (2010). Temporal transitions in participation flow in an asynchronous discussion forum. Computers & Education, 54, 340–349. doi:10.1016/j.compedu.2009.08.011

Jassó, J., Milani, A., & Pallottelli, S. (2008). Blended e-learning: Survey of on-line student assessment. 19th International Conference on Database and Expert Systems Application (pp. 626-630). Khalid, Sh. U., Basharat, A., Shahid, A. A., & Hassan, S. (2009). An adaptive e-learning framework to supporting new ways of teaching and learning. International Conference on Information and Communication Technologies (Art. Nº 5267175, pp. 300-306). Lan, W., Jiacheng, X., & Xueli, Q. (2009). Exploration of Moodle-based collaborative learning for the deaf (pp. 145–147). International Forum on Computer Science-Technology and Applications. Manouselis, N., Vuorikari, R., & Van Assche, F. (2010). Collaborative recommendation of elearning resources: An experimental investigation. Journal of Computer Assisted Learning, 26(4), 227–242. doi:10.1111/j.1365-2729.2010.00362.x Michinov, N., Brunot, S., Le Bohec, O., Juhel, J., & Delaval, M. (2011). Procrastination, participation, and performance in online learning environments. Computers & Education, 56, 243–252. doi:10.1016/j.compedu.2010.07.025 Moreno, L., Gonzalez, C., Castilla, I., Gonzalez, E., & Sigut, J. (2007). Applying a constructivist and collaborative methodological approach in engineering education. Computers & Education, 49, 891–915. doi:10.1016/j.compedu.2005.12.004

313

Effective Use of E-Learning for Improving Students’ Skills

Nicol, D. (2007). Laying a foundation for lifelong learning: Case studies of e-assessment in large 1st-year classes. British Journal of Educational Technology, 38(4), 668–678. doi:10.1111/j.14678535.2006.00657.x Qi, C. X., Qi, Z. F., Sang, J. S., & Sun, Y.-J. (2010). Web-based collaborative learning system. International Conference on Networking and Digital Society (pp. 557-560). Rubio, R., Martín, S., & Morán, S. (2010). Collaborative Web learning tools: Wikis and blogs. Computer Applications in Engineering Education, 8(3), 502–511. doi:10.1002/cae.20218 Summers, M., & Volet, S. (2010). Group work does not necessarily equal collaborative learning: Evidence from observations and self-reports. European Journal of Psychology of Education, 25(4), 473–492. doi:10.1007/s10212-010-0026-5 Vighnarajah, Luan, W. S., & Bakar, K. A. (2009). Qualitative findings of students’ perception on practice of self-regulated strategies in online community discussion. Computers & Education, 53, 94–103. doi:10.1016/j.compedu.2008.12.021 Wang, Q. (2010). Using online shared workspaces to support group collaborative learning. Computers & Education, 55, 1270–1276. doi:10.1016/j. compedu.2010.05.023 Wang, Q., & Woo, H. L. (2010). Facilitating coordination in the collaborative learning process. 2nd International Conference on Education Technology and Computer (Art. nº. 5529271, V1-181-V1-185).

KEY TERMS AND DEFINITIONS Blended Learning: Learning and teaching environment in which e-learning and e-tutoring, among others, are mixed with face-to-face activities. This combination of different models

314

of teaching and learning is based on transparent communication amongst teachers and students involved with a course with the aim of facilitating the general learning process. Co-Evaluation: Evaluation in which each student participating in an academical activity, is evaluated not only by the professor, but by each one of his/her group partners too. The aim of this technique is reinforcing the responsibility of the students in their own knowledge process by means of emitting critical judgments about the others’ work. Collaborative Learning: Learning and teaching technique based on creating knowledge in a group of students by means of sharing activities and materials. These activities can include collaborative writing, group projects, problem solving, and debates, among others. E-Learning: Transfer of skills and knowledge my means of computers and Internet applications. It involves Web-based learning, computer-based learning, virtual classroom and digital collaboration. Educational Technology: Practice of facilitating students learning by using appropriate technological processes and resources including software, hardware and Internet applications and activities. Engineering Education: Activity of teaching basic and specialized knowledges needed for doing professional practice of engineering. Group Tutoring: In the context of this chapter group tutoring is an activity in groups up to 25 students used to discuss about the concepts of asynchronous discussion forum. It involves students doubts, further readings and general group knowledge construction, and it’s useful not only for making more comprehensive analysis of subject’s concepts, but to reinforce speaking, writing, analyzing and synthesizing students’ skills.

315

Chapter 15

Strategies of LMS Implementation at German Universities Carola Kruse Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan Technische Universität Braunschweig, Germany Arne Koesling Leibniz Universität Hannover, Germany Marc Krüger Leibniz Universität Hannover, Germany

ABSTRACT In Germany, a learning management system (LMS) has become an everyday online tool for the academic staff and students at almost every university. Implementing an LMS, however, can be very different depending on the university. We introduce some general aspects on the strategies at German universities on how to implement an LMS. These aspects are mainly influenced by two main approaches, the top-down and bottom-up approach, which determine the decisions and actions on different levels at the university. In order to show how the strategies are carried out, we are presenting three case studies from universities based in the German federal state of Lower Saxony. We are going to reveal that both approaches play a part in each strategy, however differently weighted. It becomes clear that networking and collaboration plays a crucial role, not only concerning the technical development of the LMS software but also in organisational and educational terms.

DOI: 10.4018/978-1-60960-884-2.ch015

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Strategies of LMS Implementation at German Universities

INTRODUCTION Using a central learning management system (LMS) in academic terms is subject to some strategies that are following the universities’ goals on several levels such as for educational or administrative reasons. In this article, we are focussing on the different strategies at German universities on how to implement an LMS into everyday university life. The originating motive is that an LMS that is intensely used by the academic staff and students can also set a path to improve teaching and learning at university, either in regard of organisation or quality. We understand under the intensity of LMS usage both the quantity (How many people are using the LMS?) as well as the quality (At what extend and how diverse is the LMS used?). On the basis of three case studies based upon implementation processes at three universities in Germany, we show different ways to anchor LMS usage within an educational institution. The article is divided into two main parts. First, we introduce general aspects of LMS implementation at German universities starting off with the description and discussion of two approaches concerning the implementation process (top-down and bottom-up approach). Then, we are going in to detail by showing how these approaches affect the decision making on one LMS, the setting up of services, the process of change management as well as the development of networking and collaboration. In the second part, we are focussing on our cases by presenting the common ground concerning strategy goals and LMS choice. Subsequently, we are going into detail by recounting how three universities in Hannover, Osnabrück and Braunschweig have gone through the process of implementation. Finally, we are comparing these experiences to the general information of the first part and are making suggestions for forthcoming processes.

316

As academic structures are very diverse from nation to nation and bring along different terms for affected people and institutions, we find it necessary to clarify the terms and concepts we are using here regarding German universities. Please, refer to the section “Key Terms & Definitions” at the end of this article for more information.

TWO APPROACHES OF LMS IMPLEMENTATION IN GERMAN EDUCATIONAL INSTITUTIONS The use of new media, including LMS, was at least until 2004 rather subject to time-limited projects at many universities in Germany (Kleimann & Wannemacher, 2004). Although there already existed initiatives to fold up technology enhanced learning (TEL) activities within universities some years before (e.g. “NewMediaNet”1 at the University of Freiburg), besides projects, the continuous use of new media was narrowed to single departments. This becomes apparent in the succeeding nationwide calls for project proposals issued by the Federal Ministry of Education and Research (in short BMBF) “Neue Medien in der Bildung” and “Neue Medien in der Bildung 2”. As the first call demanded project proposals for single eLearning-units (Bundesministerium für Bildung und Forschung, 2000), the BMBF recognized that it did not go any further than that: no processes were initiated to implement eLearning structures into the universities. For that reason the second call demanded project proposals to integrate eLearning into the universities sustainably (DLR, 2009). TEL related systems were operated due to the interests and the engagement of single persons and in some cases attracted larger user groups. But those bottom-up approaches were limited since they were not reflecting the needs of the university as a whole. However, particularly around that time, some universities already identified the potential of new media and launched activities to provide

Strategies of LMS Implementation at German Universities

centralized services for the use of TEL systems. Often an LMS is implemented as a starting point for TEL at university. While the selection of an appropriate LMS can be done by extensive evaluations and strategy considerations of technical aspects like costs, performance, adaptability to corporate identity and usability aspects within this top-down strategy, a broad acceptance of the chosen solution regarding all concerned user groups (teachers, students) cannot be predicted safely in advance. Kerres (2005) distinguishes between two approaches of organisation development for integrating new media in universities. While the minimal change on the one hand tries to do only small changes in structures and processes to ensure acceptance of the teachers, the active approach on the to other hand makes use of a change management to induce extensive modifications to the organisational structure. Depending on the attitude of the key administration as well as its engagement towards the use of a common LMS, it evolves in a more or less profound change of the university landscape. Either it is at one period of time undecided whether to implement an LMS at all or which LMS to choose to maintain university administration and courses. In this case, it is possible that a faculty or even a singular department feels the need for setting up an LMS for itself during that period of uncertainty. Often other faculties or departments notice how helpful an LMS can be for daily work and start to participate in using the LMS so the number of users rises gradually. This is what we call bottom-up approach. The other direction is, when the key administration pushes the integration of LMS in the university so the faculties have to use the LMS for some processes – often administrative processes – and hopefully recognize the benefits an LMS can provide. We call this top-down approach.

Pros and Cons of the Two Approaches From our experiences and observations in the German university landscape, there are diversified advantages and disadvantages for both of these approaches. The top-down approach involves a risk that the focus on LMS is predominantly set on administrational processes than the didactical possibilities. For example, the academic staff is eased by organisational procedures through the LMS but not supported in questions about how to integrate an LMS within their course. Furthermore, it is more likely that some people are very much displeased with the compulsory LMS usage – probably another one than they have been using before. Also, it is possible that an LMS does not fulfil the demands of the different teaching and learning cultures at the faculties. The sequential nature of engineering courses demands other features than the holistic nature of the humanities. For example, in mechanical engineering content is based on facts, laws and rules that often cannot be questioned. In TEL, this content can easily be transformed into a Web Based Training or learning quizzes. In comparison, the students of a philosophy course build and discuss theses that can be viewed and argumented from different perspectives. In this case, regarding TEL, it would be necessary to supply communication means such as discussion boards or chats. So, if the LMS does not offer the needed features, it is most likely that one or the other faculty is dissatisfied with its use. However, the top-down approach also comprises advantages. It ensures a high frequency of usage which potentially maintains even the use of didactical possibilities of the LMS, because the academic staff might become interested in didactical applications after they have got in touch through administrative use and widespread provision. It brings along centrally organized services to professionally support a large number of users with a wide range of services (e.g. the universities of Trier (2010), Frankfurt/Main (2010) or

317

Strategies of LMS Implementation at German Universities

Duisburg-Essen (2010) have one central service unit for all LMS users). As the LMS implementation is headed by the key administration, the users can expect a sustainable operation of the system and support. For the academic staff as well as the students, it is a relief that they can depend on one LMS only and do not have to change from one to the other system depending on the courses they give or attend. Singular institutions that would have operated an LMS on their own otherwise do not have to put up with technical and support questions. The bottom-up approach takes the risk of having miscellaneous LMS each with only a subset of users at one university. While each LMS demands technical operation and supporting services it is obvious that several LMS produce redundant workload. Should there later on be a decision to use a single LMS the change management will be difficult to merge different LMS users and data to one. Considering the practical use, for academic staff and students several LMS

mean to adapt to several LMS structures with different interfaces and to overlook several data sources and probably hold several user logins and passwords, depending on the IT infrastructure of the university. For example, at the Technische Universität Braunschweig, we have evaluated that several students had to use up to six different LMS in their courses (Fleck, 2010). No matter how many LMS are operated at the university, the bottom-up approach also means that the LMS is not intended to be linked to other IT systems such as the university management information system or library system. Of course, this approach also has its advantages. Since the LMS was chosen based on specific demands, it ensures users to have a concrete will to use the LMS in their university courses. From our observations, the effect is that if not the number but the intensity of usage is relatively high as well as the rate of self-organisation. Users do not have to be convinced and are more likely to be satisfied with the system. Therefore, they often recommend

Table 1. Overview of the two approaches Approach

Bottom-Up

Top-Down

Proceeding

One faculty or one department starts using an LMS, afterwards other faculties/departments start participating

The key administration pushes the integration of an LMS into the university

Advantages

LMS is chosen based on user demands. Users have concrete will and request to use the LMS. High intensity of usage in courses. High satisfaction with the chosen system refinement participation by the users is higher

High frequency of usage. Academic staff might get interested in the didactical possibilities during administrative usage. Other IT systems used at the university can be connected to the LMS support and authentication can be organised centrally from the beginning. Students can use one single LMS and not different ones in their subjects.

Disadvantages

Low number of users. Coexisting of several LMS in different faculties or departments. Redundant workload for supporting the several LMS. If one LMS is chosen to be the general one at that university: difficulties to merge the different users and data to one LMS. Several login data for academic staff and students. Users might have to handle with different LMS interfaces. Other IT systems are most likely not to be connected to the several LMS

Focus is possibly on administrational processes. Didactical possibilities might play a minor role. People might be displeased by the reason that one LMS is forced when they have used another one before. The LMS mirght not fulfil the special needs of some academic staff or departments. Lower intensity of usage in courses

Rate of usage

low usage/few users in the beginning, can increase fastly

high/many users from the beginning

Kind of usage

Probably more didactical

Probably more administrative

318

Strategies of LMS Implementation at German Universities

the usage to others so that there is not as much need for marketing activities. The participation in evaluation and refinement processes of the system and organisation is also usually high. Considering all these pros and cons that only display a part of arguments and examples, it is obvious that one approach is not superior to the other. In the contrary, experiences in Germany show that the universities tend have gone through a mixture of both during changing phases. The extend of the two approaches varies from university to university, as mentioned above, depending on the attitude of the key administration towards a common LMS and also depending on the successfulness of singular institutions to apply a specialized LMS to their individual needs. We can state that the decision or at least the approval of one central LMS by the key administration is required to establish a sustainable university-wide solution. Experiences have proofed that one central LMS is necessary for several reasons: The academic staff and students can rely on one central system and do not have to switch from one to the other. The operation and support of one LMS is more efficient and avoids redundant workload. In most cases, it is possible to link the LMS with other central IT systems so that it can be used more effectively in adminstrative terms as well as in terms of the diversity of teaching and learning. The university can distinguish itself from others by creating a disctinctive image with the LMS. In the following, we are going to discuss the processes of choosing one central LMS.

•

Choosing a Central LMS for a University

SERVICES AND CHANGE MANAGEMENT

Baumgartner et al. (2002) consider the following parameters to be the critical ones for the decision on LMS usage:

When introducing new structures and processes in the university to integrate new media, there are some basic services that need to be provided. Generally, we distinguish between central and local services. In the following, when talking about services and change management, we mean central services and change management which have a

•

communication possibilities (like internal messages, chat, discussion boards)

•

•

didactical advantages (like online exercises, tutorials, learning diaries) administrative possibilities (like participant lists, dividing participants into groups, configuring the whole system based on one’s needs) technical possibilities (like stable operation, adaptability to individual means, interoperability with other IT systems, data protection)

These parameters are assessed variously at different institutions or rather universities. Probably the parameters are weighted differently depending on whether the top-down approach or the bottomup approach occurs at the certain university. Considering the top-down approach, the technical and the administrative possibilities are given more attention due to the fact that the key administration follows the goal to unify the administrative processes at the university. In the case of a bottom-up approach, the didactical und communicational advantages may be more relevant due to the fact that the students’ and academic staffs’ needs take centre stage and both groups are more interested in didactics and communication. According to the weighting of the approaches, a decision is based upon the different possibilities of the LMS. After this decision is made, there are subsequent decisions implied concerning the tangible implementation of the LMS and the services that can be provided.

319

Strategies of LMS Implementation at German Universities

closer connection to the top-down-approach than to the bottom-up one. The services are intended to ease LMS usage for the students, the academic staff and the university administration and give them the skills to embrace the use of this media within their activities. Kubicek et al. (2004) identified eight fields of activities to consider when setting up TEL in an educational institution. It is the services business to make sure that these tasks are performed well in order to support all persons concerned in an optimal way. • • • • • • • •

• • • •

Students Academic Staff Institutions (Departments, Institutes, Schools etc.) Key administration and administrational services

The activity field tasks are related to these stakeholders in a dissimilar way. In the following table, the fields and stakeholders are matched. The fields’ relevance for the particular stakeholder is marked with the symbols +, o and -. “+” means high or very high relevance “o” means middle-rate relevance (neither high nor low) “-“ means low or no relevance It is important to keep the demands and aims of these groups in mind when taking a look at the activity fields. Some of these them are the same for different stakeholders such as the need for security, privacy or availability, but many of them are particular for one of the stakeholders. For example, Students want to get necessary information easily. Within a university there are many sources of information and it can be difficult to find out where to get which information. In the course of the Bologna process (European Commission, 2010), the universities are restructuring from diploma to

Integration to university administration Integration into curricula Qualification and pedagogic support, consulting Development of content Operation of systems, technical service and support Marketing Quality Management Strategy Development

One can identify four types of stakeholders in a university context. The acceptance of these affected groups is most essential to the work of integrating new media.

Table 2. Overview of fields of activity and the different stakeholders Students

Academic Staff

Institutions

Key administration and services

Integration to university administration

-

-

o

+

Intgration into curricula

o

o

o

+

Qualification and pedagogic support, consulting

o

+

o

-

Development of content

o

+

-

-

Operation of systems, technical service and support

+

+

+

+

Marketing

+

+

o

+

Quality Management

+

+

+

+

Strategy Development

-

-

o

+

320

Strategies of LMS Implementation at German Universities

bachelor and master degree programmes. This makes it complicated for the students’ orientation due to their tight timetables. An LMS can provide a central information source and planning tool for their study life. Also, they need possibilities to communicate with the academic staff and fellow students. New media offer many communication channels, so the students expect to be able to use them within the university as well. In addition students have to use every possibility to learn and to exercise. Having to go through many exams and to render many course achievements, students need some space to learn alone or collaboratively. These spaces can also be provided virtually in addition to the courses they attend and the things they do at the library or at home. Furthermore, some students are interested in mobile learning as well as the use of new technologies. They expect the university to provide state-of-the-art digital media, and they are entitled to do so due to the fact that they are paying high admission fees for their studies. Academic staff has to use the time for teaching effectively. As having several functions within a university – doing research as well as preparing and holding several courses – academic staff has to organize its time well. Since university teaching becomes gradually more important (Stifterverband für die Deutsche Wissenschaft, 2010), an increasing number of academic staff wants to improve their courses. At the moment, there is a rising number of initiatives initiated by the Federal Ministry of Education and Research to improve teaching and learning at universities in Germany (Stifterverband für die Deutsche Wissenschaft, Bundesministerium für Bildung und Forschung, Winde, 2010).To achieve that goal, the academic staff needs to organize a lot of material for the courses, including several subjects. The material has to be sorted and to be filed well. Even though more and more academic staff members want to refine the material for the courses and to integrate online units, they do not want to have

their courses replaced by pure online learning (Kaltenbaek, 2009). Institutions want to stand out within the university as there is often a competition between the several departments offering the best courses and being most innovative. A basic need of all departments is to organize their person and course data most efficiently and effectively. Due to the fact that there is a lot of these data, it has to be clear what data can be found in which place and is transferred to which other systems at what time. In the same way key administration and administrational services need to have all IT systems integrated into a network within the university and well-defined data sources (Igel, 2009). Although the leadership is aware of the institutions’ competition, it wants all institutions to cooperate in order to have a functioning unit. Since university teaching gradually becomes more important it wants an improvement of teaching as well. Altogether, the goal is to satisfy the academic and administrational staff and to ensure a high quality standard for the whole university. Besides the competition amongst academic staff as well as institutions, there is also a competition between universities in the country or internationally to have the best reputation. It is obvious that Kubicek’s et al. activity-fields cover the needs and aims of the different groups unequally. Some of them concern all groups, for example the operation of systems or qualification, others are more important for a particular group, for example the development of content for the teachers. Regarding the top-down and bottom-up approach, it is clear that the needs and aims as well as the activity fields which are related to the affected groups become more important in every special case. For example, considering the bottomup approach, students and teachers are most likely the groups that are more affected because they are the first persons that act in that case. So at first, their needs and aims take centre stage and so do the co-related activity fields. Otherwise, considering the top-down approach, the needs and aims of

321

Strategies of LMS Implementation at German Universities

key administration and administrational services as well as the institutions are the groups that have to be brought into the focus and therefore other activity fields play a decisive role. Just as every university needs the services, it also has to apply a change management when implementing new media. It covers several decision processes as well as the coordination of different fields. It starts with an evaluation of the state of the art, a strategy on convincing other LMS users to change the system, and goes on to planning the linking of other important IT systems as well as establishing an operation and support unit with a qualification concept for several thousand people. Other fields of change management concern the involvement of stakeholders as well as the strategy of further development (Seufert, 2008). As many universities are undergoing similar processes, an information exchange between them is necessary for an effective progress. This complex process shows that there is some networking and collaboration on multiple levels.

Network and Collaboration First of all, it is necessary to differentiate between networking and collaboration. Networking stands for the cooperation of different parties whereas each party has its own set of tasks and decision power. A network is established to make use of the different expertises and to exchange that know-how in the perspective of a special issue of common interest. Collaboration of different parties on the other side serves to share their tasks and decisions in order to reach a common goal. Both networking and collaboration should not be viewed seperately. One can be the grounds for the other at certain stages. Both have the characteristic to set free synergetic effects. Concerning the LMS implementation there is the need for networking and collaboration on an internal and external level. We understand internal as the level within a university and external as the connection between two or more universities and/

322

or other external institutions. From a technical point of view, there is on the one hand networking and collaboration focussing on the LMS itself and its environment and on the other hand the networking and collaboration concerning the interrelationship between the LMS and other IT systems and their environments. In the following, a number of parties are listed that possibly network and/or collaborate in one way or the other at a university and beyond: •

•

• •

•

•

•

•

Central service units within the university, e.g. the university library with an online library system, the computer centre, the public information office responsible for the university’s website), and others The university administration as most of the university administrations have an administrating system for matriculation data or staff directories The faculties as institutions within a university The teachers and the students within the university teaching different subjects and also from different universities teaching the same subject The key administration, especially the vice president responsible for teaching and learning Other universities using this special LMS or having experiences with implementing LMS Other universities having didactical experience in content production and/or the development of TEL courses Concerning LMS with open source code: developers from other universities, companies or with private interest

How networking and collaboration works shall be demonstrated with the following two examples.

Strategies of LMS Implementation at German Universities

The Task Force on Technology Enhanced Teaching and Learning (Network) Since an LMS is implemented for the use in teaching and learning, it is obvious against the background of a diversity of teaching and learning cultures that it holds a large variety of application scenarios. In order to convince professors, academic staff and students of the advantages, the service units have to be aware of this wide range. This is why in Northern Germany, about ten universities from different federal states have built a task force to develop and exchange special issues on Technology Enhanced Teaching and Learning. Normally, the service units do not have identical profiles so that they often vary in expertise and services. So, it is of a great profit when one university displays its experiences in onlinelecture production for example. Other universities can give critical feedback and have a practical example about the pros and cons of this special service. Furthermore, the universities can decide to collaborate concerning this special service in order to make quicker advancements (Phan Tan & Prey, 2010; Phan Tan & Krüger, 2009).

The Developer Community of an Open Source lMS (Collaboration) In the case of an LMS like Stud.IP2, those developers are employed by the universities using Stud. IP having the task to adapt Stud.IP to the needs of that specific university. A core group of developers steers the developments of all programmers to some extend resulting in benefits for every participating university. However since those developers primarily represent the needs of a single university, this redirection of effort to maintain one single monolith software product is a non-trivial task. On one hand, dearly needed maintenance tasks for the software is not covered properly, since the focus always is on the addition of new functions. On the other hand, the pace of implementing the

university-specific functions in generic ways to let the community benefit absorbs a great deal of time because of discussions, compromises and finally reprocessing the software-code. Nevertheless, by the use of plugins with a specialised functionality spectrum, the Stud.IP community experienced an innovative way of developing and sharing additional functions needed for university-specific requirements. Plugins add functionality to the base system, keeping the addition of this functionality optional. In the field of didactics and change management, the universities are able to profit by the experiences of the other universities, especially the TEL interested staff and later the central institutions that are responsible for the LMS.

Strategy Goals at Lower Saxon Universities After we have discussed the two approaches and their impact on different aspects such as LMS choice, service and change management as well as networking and collaboration, we will now make the shift to our case studies. At our universities (Universities of Braunschweig, Hannover and Osnabrück, all are situated in the German federal state Lower Saxony) that we are representing in this article, we mainly follow four strategic goals: The first goal is to achieve a high penetration of LMS usage by attracting the faculty staff. Secondly, the main argument for using an LMS instead of traditional means of course administration and moderation is in the main point the efficient and effective use of LMS tools. It is absolutely plausible for a teacher to use the LMS instead of a paper/pencil list when it comes to maintain attendance lists or administer learning groups within a course. Further administrative advantages such as the distribution of digital documents and bibliographies are as convincing as communicative advantages when using discussion boards, chats and wikis to collect, facilitate and summarize the students’ contributions.

323

Strategies of LMS Implementation at German Universities

Thirdly, it is intended to take the high penetration of LMS usage as a basis for a larger diversity of teaching and learning through media as well as an optimised way of traditional teaching and learning supported by an LMS. Examples of diversity are for instance online and blended learning formats, student oriented collaborative learning, subject/university comprehensive courses or international courses. Last but not least, the use of an LMS is also a contribution to the Bologna process. Especially, comprehensive and international courses draw the universities together and are a great opportunity for exchange. There are different ways to reach these strategy goals. To exemplify this fact we regard three universities in Lower Saxony, Germany, that implemented Stud.IP as LMS and also take part in the Stud.IP network but with different foci.

The Decision on “Stud.IP” at Lower Saxon Universities The possibility and decision to install and offer the LMS Stud.IP was often based on the fact that Stud.IP is Open Source software and beyond the effort for installation without further costs in testing phases. For both approaches, this given fact was ideal for a start. Furthermore, the universities in Lower Saxony started networking early in 2002 (the so-called ELAN (E-Learning Academic Network Niedersachsen (2010)) projects, funded by the Ministry of Science and Culture3) and noticed the associated universities’ experienced benefits by using Stud.IP. This was a crucial argument for many universities to decide on this LMS, too. Again, it counts for both approaches, for it was possible to exchange experiences on different levels. The significant differences to other LMS became evident: The structure of Stud.IP was more suited to represent the organisational structure of German educational institutions and to tackle their needs. For example, while other LMS followed

324

a strict course structure, Stud.IP offered courses that were connected to teachers, professional groups and faculties. The teachers were enabled to create online course rooms with a few clicks and therefore it required little experience for the system itself. Primarily, the functionalities were limited to file upload and download, participants lists, discussion boards and literature lists and therefore focussed on the support of attendance tuition. In other words, Stud.IP is not yet natively capable of creating WBT or playing SCORM modules while other LMS were more eligible to support distance learning and commercial further education. It is closer to the traditional nature of teaching and learning at German universities than some other LMS and therefore a low-threshold technical application for teachers and students. Moreover, Stud.IP was targeted to the concerns of students right from the start: Each course that a student is subscribed to, is entered into a calendar, so are changes in dates for lectures. Besides the file upload and download functionality and the simple to use communication with teachers, this function is most popular among students and eases their study life. Altogher, eleven universities are using Stud.IP in Lower Saxony. Only very few made a clear decision against it. In the following, we describe what is necessary for implementing an LMS into the university and why it is obvious to collaborate especially in this process.

CASE-STUDIES Technische Universität Braunschweig (TU BS) At the Technische Universität Braunschweig, Stud.IP was introduced in 2004 during the ELAN projects, when Braunschweig was amongst other tasks responsible for the didactical support of the manyfold TEL projects in the network. The system was tentatively installed by the universities’ central service unit Kompetenz-

Strategies of LMS Implementation at German Universities

zentrum Hochschuldidaktik für Niedersachsen (KHN), the Centre of Excellence for Lower Saxon Higher Education, as a consequence of its media didactical consulting activities. The faculty of construction engineering started the use of the system and funded the support done at KHN after the successful introduction of Stud.IP. This bottom up constellation (support at KHN, financial funding from a university department) was continued until 2009. Within this period, word-of-mouth recommendation took place and the user community grew up to over 10,000 users, although there was no proactive advertisement for the system. Since 2006, a cooperation with the university IT centre was started, because the increased payload of the system required an enhanced technical service and more suitable localization. Furthermore, the KHN developed several didactical concepts for using this LMS. Those concepts were the results from didactical workshops where the participants – professors and academics – prepared their own technology enhanced teaching. With the help of Stud.IP, the teaching and learning programme was extended by online- and blended-learning components. Therefore, the participating university teachers were able to actively experience the didactical integrated use of Stud.IP in the learning process. In autumn 2009, after a comparative evaluation with several other LMS, there was a decision for a university-wide usage of Stud.IP at the TU Braunschweig and its interlink to administrative systems (university management information system, library, content management system, and portals). Since that point of time, human resources and adequate technical hardware has been provided from central university funding to facilitate support, services and operation of the system. The KHN is still responsible for supervision, therefore this is a clear commitment to a didactical focal point in the utilization of the LMS. The students demand the usage of Stud.IP in their courses in the meantime due to the fact they got used to the advantages for their learning process.

The Technische Universität Braunschweig adds authority to the students suggestions and requests with regard of their quality assurance (Faßbender, Miosge 2010). The usage of Stud.IP is seen as one of the activities making a contribution to quality improvement at the university. Considering networking and collaboration within the university, it was a major advantage that the 5-year-period of bottom-up development lasted this long, as the networking activities with the faculty of construction engineering and the IT centre were built on solid grounds of common interest. The top-down desicion for a university-wide system was another push for the other administrative systems to collaborate. For example, special plugins are being programmed in comprehensive teamwork to enhance library functions and provide a single sign on for the IT systems. A task force with representatives from all IT systems is exchanging experiences, problems and demands on regular terms in order to further develop technical and organisational questions that relate to all IT systems. On external terms, the TU Braunschweig is both member of the network on technology enhanced teaching and learning as well as the developer core group mentioned above. As the KHN is offering its teaching and learning programme to all academic staff in Lower Saxony, too, it raises the comprehensive awareness for didactical questions on TEL. Braunschweig covers about 13,500 students, 240 professors and almost 3,000 additional teachers. Currently, there are 12,000 user registered in the system (11,000 students, 1,000 teachers, administrators, and tutors) of which more than 8,300 users (or 7,500 students) have been active during the last semester in the summer of 2010 with nearly 2,000 active courses and 4,700 documents.

Leibniz Universität Hannover (LUH) The first Stud.IP installation at the Leibniz Universität Hannover was set up in 2004 at L3S Research Center, a central institution at the university

325

Strategies of LMS Implementation at German Universities

concerned with digital learning and knowledge. After the Stud.IP system was provided by L3S, the Leibniz Universität Hannover experienced a surprisingly high demand. The information that the Stud.IP system was available spread very quickly within the university. Therefore, without requiring further advertisement for Stud.IP, several teachers used it within their lectures. So, the decision to use Stud.IP as LMS at the LUH resulted from a high demand by the teachers, not from a systematic comparative evaluation of LMS. It is believed so far that this development is a good basis of decision making. Not IT experts, administrative officers, executive comittees or TEL experts did take decisions in advance, but the teachers themselves. Hannover is sure that they provided a system giving the most benefits in daily use for teachers and students this way. Since the midyear 2005, the widespread use of Stud.IP at the LUH has been systematically fostered, however generously funded by the Federal Ministry of Education and Research. Therefore, several measurements, information events, trainings, interlinks of Stud.IP to HIS-LSF (HochschulInformationsSystem, 2010) (the university management information system in which course data is administrated) and ILIAS (another LMS, but with a stress on content production) (ILIAS, 2010), systematic further development within the developer community, didactical orientation of trainings were performed.Within an extensive TEL strategy, Stud.IP was established as the official LMS of LUH in summer 2008. In the meantime, Stud.IP is used for administrative matters as well as for supporting university courses. Stud.IP can be seen as well-established now. Considering networking and collaboration within the university, it is obvious that after the induction phase at L3S it was mainly due to the funded project that led to university-wide collaboration. So, it was possible to set up a profound and elaborate central team that had the task to spread and gather all information and processes within the university. After the funding phase,

326

the financial support of this so-called e-learning service department (“elsa”) has been taken over by the university itself. “elsa” is also member of the two networks mentioned above. Against the background that the LUH has 20,600 students, 308 professors and over 2,200 additional teachers the following figures can be given: During the last semester in the summer of 2010, there were 14,700 student users online at least one time within the last 30 days. In addition, 1,100 professors and teachers as well as 600 tutors are using Stud.IP. Up to 150 users are concurrently online at the same time and 1.7 million documents have been downloaded in the winter semester 2009/2010.

Universität Osnabrück (UOS) The Stud.IP System has been operated at Universität Osnabrück since 2003. The decision of the university leadership to use Stud.IP as a mandatory LMS for the management of all university courses and other purposes was early and expected. Stud.IP has therefore been used university-wide since 2004. For example, the course catalogue and staff directory of the complete university is generated from Stud.IP and provided as PDF on the website. Stud.IP is also used for the course room management of the university as well as all electronic application procedures. The university established a department called “Zentrum für Informationsmanagament und virtuelle Lehre der Universität Osnabrück (virtUOS)” responsible for the implementation and maintenance of TEL systems in an organisational and technical manner. Osnabrück was right from the start a location at which the use of an LMS was supported strongly by a top-down approach, so the human resources for support are mostly provided by central university funds. Services with a didactical focus like trainings and consulting are also done, but they are not a focal point of the Osnabrück range of services regarding Stud.IP. This seems to be due to the fact that the

Strategies of LMS Implementation at German Universities

executive board demands that Stud.IP supports the university administration on principle terms. “virtUOS” is a central service unit considering most administrative systems at the university, therefore it is not as much necessary to collaborate with other institutions as the other two universities. Due to this matter of fact, it is possible to make shortcut decisions and carry them out almost instantly. This makes it possible to collaborate on more elaborate terms, such as developing TEL based introductions to the library coorporately with the library. Like the universities in the other two case studies presented, Osnabrück is highly integrated in a network. The focal point of Universität Osnabrückis to play a leading role in the Stud.IP development within the Stud.IP developer community. The introduction of Stud.IP as a central unversity-wide LMS is also illustrated by the amount of users. The Universität Osnabrück covers about 9,300 students, 214 professors and over 1,100 additional teachers. During the last semester in the summer of 2010, almost 8,400 students and about 3,400 other university members (professors and academic staff) are actively using the system.

However, even though Stud.IP was introduced top-down it is not only used as a course catalogue only, when an adequate service is provided. This is indicated by the following figures: There are 34,427 lectures with 247,852 documents in the system, 3,507 literature lists, 445,848 lecture-dates and 8,705 news.

THE CASE STUDY UNIVERSITIES: AN OVERVIEW Table 3 shows an extract of the case studies, focussing on the key features with respect to the two approaches.

CONCLUSION Each case study in this article was a blend of the two approaches. While at the Leibniz Universität Hannover the top-down and the bottom-up approach came about equal, at the Universität Osnabrück the top-down approach was realised more than the bottom-up. Quite contrary to Hannover, the

Table 3. Overview of key features on the LMS implementation at three universities TU Braunschweig (13,500 students)

Leibniz Universität Hannover (20,600 students)

Universität Osnabrück (9,300 students)

Strategy/Approach

More bottom-up than top-down

Equally top-down and bottomup

More top-down than bottomup

Implementation

first 2004 university-wide 2009 (implementation process in progress)

first 2004 university-wide 2008 (implementation process completed)

first 2003 university-wide 2004

Focus of use

didactical

hybrid

administrative

Funding

institutional funding from 2004 to 2009 central university funding since 2009

institutional funding from 2004 to 2005      third-party project funding from 2005 to 2008      central university funding since 2008

third-party project funding from 2003 to 2004      central university funding since 2004

Institution responsible for service

within the university, teaching and learning centre

within the university, elearning centre

within the university, elearning centre

Number of student users

7,500 (57%)

14,700 (71%)

8,400 (90%)

327

Strategies of LMS Implementation at German Universities

TU Braunschweig was determined more by the bottom-up approach than by the top-down approach, because the university leadership came to the decision to have Stud.IP as the central LMS for the whole university at a rather late point of time. As one can see in the table above, the three universities implemented Stud.IP almost even though not quite at the same time. So the network pictured earlier in this article was founded along these developments offering a platform of exchange at a very early stage of growth, whereas one university was able to make the step from a project to an established system earlier than the others. Simultaneously, the network encouraged not only the described but also other Lower Saxony universities to use Stud.IP as an LMS in a manner of a domino effect. Altogether, the networking got essential for the further development and establishment of Stud.IP at most universities. Nevertheless or even therefore, each university described in the three case studies generated a main focus of LMS use which depends on the approach that determined the implementation process. While the Universität Osnabrück has an administrative focus due to the fact the LMS was implemented mostly via top-down approach and the university leadership preferred using the LMS to support the university administration, the TU Braunschweig uses the LMS to support the faculty staff and improve the lectures that way because the bottom-up approach was the superior one for the implementation and the academic staff was able to bring in their requirements and aims very early. Since the Leibniz Universität Hannover had a top-down approach equivalent to the bottom-up approach, the usage of the LMS has a hybrid character, for the faculty staff as well as the university leadership were able to bring in their requirements and aims at an equal share. For the next step, it is appropriate to analyze if the approach by which an LMS is integrated into university is related to the long-term kind of usage of the system. As being deducted from the fact that each case study includes both of the two approaches,

328

the resources for keeping the LMS running at all three universities are paid by the central university funding in the meantime. In addition, every university has institutionalised the offer for didactical, technical and other kinds to provide services either by instructing an existing institution within the university that was already concerned with supporting the LMS and its users before without having the official instruction – or by founding a new specialized institution. At all universities, we experienced the necessity to have such a central institute to bring the affected persons and institutes together and to make commonly gainful decisions. We assume that the approaches have been successful, since all three universities have an above average intensity of usage. Nevertheless, a continuous service offer is mandatory to guarantee the proceeding of the use of the LMS. At the moment there is a basic support for user requests and mentoring and a course programme with lessons in getting the basic information about the LMS across to the teachers, students and the administrative staff.

OUTLOOK For the further etablishment of Stud.IP, we have pegged the following tasks as the logical steps that have to be done. There are mainly didactical, but also technical tasks. Enlarging training offerings and the didactical mentoring to supply specialised needs like realising several didactical settings including online work sheets etc. Currently, the trainings just give an introduction to the LMS usage by teaching the participants to use the basic LMS features like uploading files or managing the list of their course participants. Another important task is appointing TEL coaches to support individual ideas and teachers in realising their individual teaching ideas. For these means, the universities are collaborating by exchanging trainers for special topics or by offering comprehensive courses.

Strategies of LMS Implementation at German Universities

The next important technical step is to interlink Stud.IP with more of the other systems. As many of the universities in Germany use a university management information system, Hannover and Braunschweig as well as three other universities are working on a middleware in the so-called “ProLiza” project. The goal is to develop an interlink between Stud.IP and HIS-LSF (see the Hannover case above) in order to pass through the basic data about courses and academic staff so that it is not necessary to fill in the data at different sources and produce redundant workload. The middleware is programmed collaboratively and the universities are exchanging experiences in the organisation of the implementation process. The finished product shall be applicable to other universities using the two systems as well. ProLiza is another example for the effectiveness of a good working network. In the end, it seems that networking as a very important strategy to foster LMS usage. By the exchange of experiences and evaluation results, the universities are able to adapt the good experiences and validated results and to avoid mistakes and extrawork. As the universities are not developing straightforwardly but more in terms of circles, the past has shown that each university can profit form the other, no matter if it has started off first as an early adopter or if it has jumped on the bandwagon lateron.

REFERENCES Baumgartner, P., Häfele, H., & Maier-Häfele, K. (2002). E-learning praxishandbuch: Auswahl von lernplattformen; Marktübersicht - funktionen - fachbegriffe. Innsbruck, Austria: Studienverl. Bundesministerium für Bildung und Forschung. (2000). Exzellenzinitiative. Retrieved on December 20, 2010, from http://www.bmbf.de/de/1321. php

Bundesministerium für Bildung und Forschung. (2000). Förderprogramm Neue Medien in der Bildung. Lehr- und Lernsoftware. Retrieved on December 20, 2010, from http://www.bmbf.de/ pub/neue_medien_bildung.pdf Bundesministerium für Bildung und Forschung. (2000). Qualitätspakt Lehre. Retrieved on December 20, 2010, from http://www.bmbf.de/ de/15375.php Deutsches Zentrum für Luft- und Raumfahrt (DLR). (2009). Projektträger im DLR – Neue medien in der bildung, Retrieved on December 20, 2010, from http://www.dlr.de/pt/desktopdefault. aspx/tabid-5882/9540_read-18559/ Elan (n. d.). E-Learning Academic Network Niedersachsen. E-Learning Academic Network Niedersachsen. Retrieved on August 17, 2010, from http://www.elan-niedersachsen.de European Commission. (2010). The bologna process - Towards the european higher education area. Retrieved on August 30, 2010, from http://ec.europa.eu/education/higher-education/ doc1290_en.htm Faßbender, H., & Miosge, C. (2010). Technische Universität Braunschweig. Hervorragende Qualität und viel Raum für eigene Ideen. In Winde, M. (Ed.), Von der Qualitätsmessung zum Qualitätsmanagement. Praxisbeispiele an Hochschulen (pp. 48–57). Essen, Germany: Edition Stifterverband. Fleck, W. (2010, August 24). Evaluationsergebnisse von studierendeninterviews zu Stud.IP, Sommer 2010. Unpublished manuscript. Goethe Universität Frankfurt am Main. E-Learning Center am HRZ. Retrieved August 30, 2010, from http://www.rz.uni-frankfurt.de/internet/elearning/ index.html HochschulInformationsSystem. (n. d.). Lehre, Studium, Forschung. Retrieved August 30, 2010, from http://www.his.de/abt1/ab10

329

Strategies of LMS Implementation at German Universities

Igel, C. (2009). Learning-management an hochschulen. In Issing, L. J., & Klimsa, P. (Eds.), Online-Lernen. Handbuch für Wissenschaft und Praxis (pp. 207–215). München, Germany: Oldenbourg. ILIAS. (n. d.). ILIAS Information Center. Retrieved August 30, 2010, from http://www.ilias.de Kaltenbaek, J. (2009). Hochschule online - Online lehren und lernen in der Hochschule. In Issing, L. J., & Klimsa, P. (Eds.), Online-Lernen. Handbuch für Wissenschaft und Praxis (pp. 367–388). München, Germany: Oldenbourg. Kerres, M. (2005). Strategieentwicklung für die nachhaltige implementation neuer medien in der hochschule. In T. Pfeffer (Ed.), Medien in der Wissenschaft, Vol. 32. Handbuch Organisationsentwicklung. Neue Medien in der Lehre: Dimensionen, Instrumente, Positionen (pp. 147–162). Münster, Germany: Waxmann. Kleimann, B., & Wannemacher, K. (2004). Elearning an deutschen hochschulen: Von der projektentwicklung zur nachhaltigen Implementierung. Hannover, Germany: HIS. Kubicek, H., Breiter, A., Fischer, A., & Wiedewald, C. (2004). Organisatorische einbettung von e-learning an deutschen hochschulen. Retrieved on August 30, 2010, from http://www. ifib.de/publikationsdateien/MMKH_Endbericht_2004-05-26.pdf Phan Tan, T.-T., & Krüger, M. (2009). Hochschulübergreifende weiterbildung zum e-learning in niedersachsen. In Apostolopoulos, N. (Ed.), Grundfragen multimedialen lehrens und lernens. Bildungsimpulse und bildungsnetzwerke: Tagungsband GML 2 2009, 12. - 13. März 2009 (pp. 69–80). Berlin, Germany: Univ.-Verl. der TU Univ.-Bibliothek.

330

Phan Tan, T.-T., & Prey, G. (2010). Programm zur didaktischen e-kompetenzentwicklung in Niedersachsen. Zeitschrift für hochschulentwicklung, 5(1). Retrieved from http://www.zfhe.at Seufert, S. (2008). Innovationsorientiertes bildungsmanagement: Hochschulentwicklung durch sicherung der nachhaltigkeit von e-learning. Wiesbaden, Germany: VS Verlag für Sozialwissenschaften / GWV Fachverlage GmbH Wiesbaden. Stifterverband für die Deutsche Wissenschaft. (n. d.). Exzellenz in der Lehre. Aktuelle Programme und Initiativen. Rertieved December 20, 2010, from http://www.stifterverband.de/ Stifterverband für die Deutsche Wissenschaft. (2010). Lehre neu denken. Die Zukunft des akademischen Lehren und Lernens. Retrieved December 20, 2010, from http://www.stifterverband.info/publikationen_und_podcasts/positionen_dokumentationen/lehre_neu_denken/ lehre_neu_denken.pdf Universität Duisburg-Essen (u. d.). Zentrum für Hochschul- und Qualitätsentwicklung: Kompetenzentwicklung in Studium & Lehre. Retrieved August 30, 2010, from http://zfh.uni-duisburgessen.de/elearning Universität Trier (u. d.). Koordinationsstelle ELearning. Retrieved August 30, 2010, from http:// www.uni-trier.de/index.php?id=33362 Winde, M. (Ed.). (2010). Von der Qualitätsmessung zum Qualitätsmanagement. Praxisbeispiele an Hochschulen. Essen, Germany: Edition Stifterverband.

Strategies of LMS Implementation at German Universities

ADDITIONAL READING Albrecht, R. (2001). Lehren und lernen mit neuen medien: Plattformen, Modelle, Werkzeuge. Münster, Germany: Waxmann. Anderson, T. (2004). Theory and practice of online learning. Athabasca, Canada: Athabasca University. Apostolopoulos, N. (Ed.). (2009). Grundfragen multimedialen lehrens und lernens: Bildungsimpulse und bildungsnetzwerke: Tagungsband GML 2 2009, 12. - 13. März 2009. Berlin, Germany: Univ.-Verl. der TU Univ.-Bibliothek. Appelrath, H.-J., Boles, D., Kleinefeld, N., Marcos, I., Reil, D., Runge, M., & Willer, S. (2006). Einsatz des open-source-lernmanagementsystems Stud.IP zur Unterstützung der Präsenzlehre der Universität Oldenburg. In C. Hochberger (Ed.), GI-Edition Proceedings: Vol. 94. Informatik 2006: Informatik für Menschen; Beiträge der 36. Jahrestagung der Gesellschaft für Informatik e.V. (GI); 2. bis 6. Oktober 2006 in Dresden (pp. 53–58). Bonn, Germany: Ges. für Informatik.

Bertelsmann-Stiftung. (2001). Studium online: Hochschulentwicklung durch neue Medien (2. Aufl.). Gütersloh, Germany: Verl. BertelsmannStiftung. Bremer, C., & Kohl, K. Eleonora. (2004). E-learning-strategien und e-learning-kompetenzen an hochschulen. Bielefeld, Germany: Bertelsmann. Bruns, B., & Gajewski, P. (2000). Multimediales lernen im Netz: Leitfaden für entscheider und planer (2. Aufl.). Berlin, Germany: Springer. Dick, E. (2000). Multimediale Lernprogramme und telematische Lernarrangements: Einführung in die didaktische Gestaltung. Nürnberg, Germany: BW Bildung und Wissen Verl. und Software. Eileen O’ Donnell. (2010). E-learning can improve learning: Preparing students for work. Saarbrücken, Germany: LAP Lambert Academic Publishing. El-Ghareeb, H. (2010). Evaluation of service oriented architecture in e-learning: Integrating Bboth LMS and UMIS. Saarbrücken, Germany: LAP Lambert Academic Publishing.

Appelrath, H.-J., & Schulze, L. (Eds.). (2009). Auf dem Weg zu exzellentem e-learning: Vernetzung und Kooperation der Hochschullehre in Niedersachsen. Münster, Germany: Waxmann.

ELAN. (2005). ELAN: Bericht zur Förderphase ELAN I (1.10.2002 - 31.12.2004). Oldenburg, Germany: Bibliotheks- und Informationssystem der Univ.

Ballin, D., & Brater, M. (1996). Handlungsorientiert lernen mit multimedia: Lernarrangements planen, entwickeln und einsetzen. Nürnberg, Germany: BW Bildung und Wiss. Verl. und Software.

Ellis, R. A., & Goodyear, P. (2010). Students’ experiences of e-learning in higher education: The ecology of sustainable innovation. New York, NY: Routledge.

Baumgartner, P., & Payr, S. (2001). Studieren und Forschen mit dem Internet. Innsbruck, Austria: Studienverl.

Freiburg, U. (n. d.). Servicestelle e-learning. Retrieved August 17, 2010, from http://www. rz.uni-freiburg.de/services/elearning

Behrens, U. (2001). Teleteaching is easy!?: Pädagogisch-psychologische Qualitätskriterien und Methoden der Qualitätskontrolle für Teleteaching-Projekte (2. Aufl.). Landau, Germany: Verl. Empirische Pädagogik (Univ., Diss.--Trier, 1999.).

Fry, H. (2008). A handbook for teaching and learning in higher education: Enhancing academic practice (3rd ed.). New York, NY: Routledge.

331

Strategies of LMS Implementation at German Universities

Grace, S., & Gravestock, P. (2009). Inclusion and diversity: Meeting the needs of all students. New York, NY: Routledge. Häfele, H., & Maier-Häfele, K. (2005). 101 e-learning seminarmethoden: Methoden und strategien für die online- und blended learning seminarpraxis (2. Aufl.). Bonn, Germany: ManagerSeminare Verl.-GmbH. Hamm, I. (1998). Hochschulentwicklung durch neue Medien: Erfahrungen - Projekte - Perspektiven: mit einer bestandsaufnahme über multimedia-projekte an deutschen hochschulen (2. Aufl.). Gütersloh, Germany: Verl. Bertelsmann-Stiftung. Hochberger, C. (Ed.). (2006). Informatik 2006: Informatik für menschen: Beiträge der 36. Jahrestagung der gesellschaft für informatik e.V. (GI):2. bis 6. Oktober 2006 in Dresden. Bonn, Germany: Ges. für Informatik. Holten, R., & Nittel, D. (2010). E-learning in hochschule und weiterbildung: Einsatzchancen und erfahrungen. Bielefeld, Germany: Bertelsmann. Issing, L. J., & Klimsa, P. (Eds.). (2009). Onlinelernen: Handbuch für wissenschaft und praxis. München, Germany: Oldenbourg. Kerres, M. (2001). Multimediale und telemediale lernumgebungen: Konzeption und entwicklung (2., vollst. überarb. Aufl.). München, Germany: Oldenbourg. doi:10.1524/9783486593815 Kleimann, B., Özkilic, M., & Göcks, M. (n. d.). Studieren im Web 2.0: Studienbezogene Web‐ und e‐learning‐dienste (HISBUS‐Kurzinformation Nr. 21). Krüger, M., & von Holdt, U. (2007). Neue medien in vorlesungen, seminaren & projekten an der leibniz Universität Hannover: Tagungsband zur e-Teaching- und e-Science-Tagung 2007. Aachen, Germany: Shaker.

332

Lee, M. J. W., & McLoughlin, C. (2010). Web 2.0-based e-learning: Applying social informatics for tertiary teaching. Hershey, PA: Information Science Reference. doi:10.4018/978-1-60566294-7 Pfeffer, T. (Ed.). (2005). Handbuch organisationsentwicklung: Neue medien in der lehre: Dimensionen, instrumente, positionen. Münster, Germany: Waxmann. Rebensburg, K. (Ed.). (2009). Grundfragen multimedialen lehrens und lernens: Bildungsimpulse und bildungsnetzwerke. Tagungsband. Berlin, Germany: Universitätsverl. der TU. Salmon, G. (2004). E-tivities - der schlüssel zu aktivem online-lernen. Zürich, Switzerland: Orell Füssli. Schenkel, P. (2000). Qualitätsbeurteilung multimedialer lern- und informationssysteme: Evaluationsmethoden auf dem prüfstand. Nürnberg, Germany: BW Bildung und Wiss. Verl. und Software. Schulmeister, R. (1997). Grundlagen hypermedialer lernsysteme: [Theorie - Didaktik - Design] (2., aktualisierte Aufl.). München, Germany: Oldenbourg. Schulmeister, R. (2006). E-learning: Einsichten und aussichten. München, Germany: Oldenbourg. doi:10.1524/9783486595062 Schulmeister, R., & Wessner, M. (2001). Virtuelle universität, virtuelles Lernen. München, Germany: Oldenbourg. doi:10.1524/9783486598926 Stud.IP e.V. (n. d.). Stud.IP - Die plattform für lehre, lernen & verwaltung. Retrieved August 30, 2010, from http://www.studip.de

Strategies of LMS Implementation at German Universities

Watkins, R. (2009). E-learning companion: A student’s guide to online success (3rd ed.). Boston, MA: Cengage Wadsworth. Yang, H. Hao, & Yuen, S. Chi-Yin. (2010). Handbook of research on practices and outcomes in e-learning: Issues and trends. Hershey, Pa.: Information Science Reference.

ENDNOTES 1



2



3



Zwingenberger, A. (2009). Wirksamkeit multimedialer lernmaterialien: Kritische bestandsaufnahme und metaanalyse empirischer evaluationsstudien. Münster, Germany: Waxmann (Techn. Hochsch., Diss.--Aachen, 2008.).

“NewMediaNet” is now called “Servicestelle E-Learning” (Universität Freiburg, 2010). Stud.IP is an open source LMS that has been developed by a group of students in Germany. Stud.IP is a short form for “Studienbegleitender Internetsupport von Präsenzlehre” (students accompanying internet support in face-to-face teaching and learning), http:// www.studip.de (Stud.IP e.V., 2010),. Since its initial start in 2000, it has been established at twenty-something universities in Germany during the past ten years. (Appelrath et al., 2006) ELAN stands for e-learning academic network in Lower Saxony (ELAN, 2010).

333

Strategies of LMS Implementation at German Universities

EXPLANATION OF KEY TERMS In this article, we use the term “university” as a comprehensive term for all kinds of universities that exist in Germany, ranging from general over technical to applied sciences and teaching schools. In general, they are structured hierarchically: The head of universities is a group of key administrators (president, vice presidents, provost). Next to several central service units (library, computer centre, centre for teaching and learning) there are a number of faculties (of engineering, life sciences, economics, humanities, etc.). Each faculty is divided up into single units called departments, institutes or schools. They can be of different sizes and levels and can be summed up as institutions. Concerning the affected people, each institution holds a professorship. In this article, we regard associate and assistant professors also as professors. Then there is the academic staff comprised of senior lecturers, lecturers and instructors. There is a small number of academic staff members who are not involved in teaching, e.g. researchers from third-party funds projects or scholarship holders. However, for an easier handling in this article, we use the term academic staff as an overall term for the teaching staff including the professors. Last but not least, there is the administrational staff working in non-academic positions. Considering the term technology enhanced learning (TEL), we use it as the application of information, communication and learning technologies in teaching and learning. It covers the whole range of formats from the addition to traditional teaching over hybrid teaching and learning up to virtual learning environments.

334

335

Compilation of References

ABET. (2009). Criteria for accrediting engineering programs. Accreditation Board for Engineering and Technology, 29. Retrieved November 22, 2010, from http://www. abet.org/Linked%20Documents-UPDATE/Criteria%20 and%20PP/E001%2010-11%20EAC%20Criteria%20 1-27-10.pdf

Alexander, S., & Hedberg, J. (1994). Evaluating technology-based learning: Which model? In Beattie, K., McNaught, C., & Wills, S. (Eds.), Multimedia in higher education: Designing for change in teaching and learning (pp. 233–244). Amsterdam, Netherlands: Elsevier.

Abou-Zahra, S. (2008). Web accessibility evaluation. Human-computer interaction series, 2008, Part II, 79-106.

Ali, A. J. (1990). Management theory in a transitional society: The Arab’s experience. International Studies of Management and Organization, 20(3), 7–35.

Aczel, J. C., Peake, S. R., & Hardy, P. (2008). Designing capacity-building in e-learning expertise: Challenges and Strategies. Computers & Education, 50, 499–510. doi:10.1016/j.compedu.2007.07.005

Alsop, G., & Tompsett, C. (2002). Grounded theory as an approach to studying students’ uses of learning management systems. ALT-J Research in Learning Technology, 10(2), 63–76. doi:10.1080/0968776020100207

Agrawal, R., Imieliński, T., & Swami, A. (1993). Mining association rules between sets of items in large databases, Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington, DC. (pp.207-216).

Anastakis, D. J., Cohen, R., & Reznick, R. K. (1991). The structured oral examination as a method for assessing surgical residents. American Journal of Surgery, 162, 67–70. doi:10.1016/0002-9610(91)90205-R

Albirini, A. (2006). Teachers’ attitudes toward information and communication technologies: The case of Syrian EFL teachers. Computers & Education, 47, 373–398. doi:10.1016/j.compedu.2004.10.013 Al-Busaidi, K. A. (2009). Evaluating the use of e-learning system as a supplementary tool at an academic institution. International Journal of Internet Education, 4, 153. Alexander, P. A., Schallert, D. L., & Hare, V. C. (1991). Coming to terms: how researchers in learning and literacy talk about knowledge. Review of Educational Research, 61(3), 315–343.

Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2). Andresen, M. A. (2009). Asynchronous discussion forums: Success factors, outcomes, assessments, and limitations. Journal of Educational Technology & Society, 12(1), 249–257. Andronico, A., Carbonaro, A., Colazzo, L., Molinari, A., Ronchetti, M., & Trifonova, A. (2004). Designing models and services for learning management systems in mobile settings. In Mobile and Ubiquitious Information Access (Vol. 2954, pp. 524–532). Berlin-Heidelberg, Germany: Springer-Verlag. doi:10.1007/978-3-540-24641-1_8

Compilation of References

ANGEL Learning (2008). The ANGEL learning management suite. Angus, D. S., & Watson, J. (2009). Does regular online testing enhance student learning in the numerical sciences? Robust evidence from a large data set. British Journal of Educational Technology, 40(2), 255–272. doi:10.1111/j.1467-8535.2008.00916.x Ardichvili, A., & Yoon, S. (2009). Designing integrative knowledge management systems: Theoretical considerations and practical applications. Advances in Developing Human Resources, 11(3), 307–319. doi:10.1177/1523422309337593 Argyle, M., & Dean, J. (1965). Eye-contact, distance and affiliation. Sociometry, 28, 289–304. doi:10.2307/2786027 Ashford-Rowe, K., & Malfroy, J. (2009). E-learning benchmark report: Learning management system (LMS) usage. Avgeriou, P., Papasalouros, A., Retalis, S., & Skordalakis, M. (2003). Towards a pattern language for learning management systems. Journal of Educational Technology & Society, 6(2), 11–24. Babo, R., & Azevedo, A. (2009). Learning management systems usage on higher education institutions. In Proceedings of 13th IBIMA Conference - Knowledge Management and Innovation in Advancing Economies: Analyses & Solutions (pp. 883 – 889). Babo, R., Lopes, C., Rodrigues, A., Pinto, M., Queirós, R., & Oliveira, P. (2010a). Gender differences in Internet usage habits - A case study in higher education. 14th International Business Information Management Association, Istanbul, Turkey. Babo, R., Lopes, C., Rodrigues, A., Pinto, M., Queirós, R., & Oliveira, P. (2010 b). Comparison of Internet usage habits in two generations of higher education students A case study. 2th International Conference on Computer Supported Education, Valencia, Spain. Baggaley, J. (2008). Where did didtance education go wrong? Distance Education, 29(1), 39–51. doi:10.1080/01587910802004837 Bagui, S. (2006). An approach to mining crime patterns. International Journal of Data Warehousing and Mining, 2(1), 50–80. doi:10.4018/jdwm.2006010103

336

Bailey, J., & Pearson, S. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29(5), 530–545. doi:10.1287/ mnsc.29.5.530 Ball, D., & Levy, Y. (2008). Emerging educational technology: Assessing the factors that influence instructors’ acceptance in information systems and other classrooms. Journal of Information Systems Education, 19(4), 431–443. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioural change. Psychological Review, 84(2), 191–215. doi:10.1037/0033-295X.84.2.191 Banks, S., Goodyear, P., Hogson, V., & McConnell, D. (2003). Introduction to the special issue on advances in research on networked learning. Instructional Science, 31, 1–6. doi:10.1023/A:1022583918064 Barclay, D., Higgins, C., & Thompson, R. (1995). The partial least squares (PLS) approach to casual modelling: Personal computer adoption and use as an illustration (with commentaries). Technology Studies, 2(2), 285–309. Barrella, E., Simmons, J., & Buffinton, K. (2006). Professional engineering education best practice study for 1st-year, multi-disciplinary courses. ASEE Annual Conference and Exposition, Conference Proceedings (10). Barret, H. C. (2008). Categories of ePortfolio tools, technical report. Retrieved on December 13, 2010, from http://electronicportfolios.com/categories.html. Baturay, M. (2008). Analysis of a learning management system model. E-Journal of New World Sciences Academy, 3(3), 464–477. Baumgartner, P., Häfele, H., & Maier-Häfele, K. (2002). Elearning praxishandbuch: Auswahl von lernplattformen; Marktübersicht - funktionen - fachbegriffe. Innsbruck, Austria: Studienverl. Beatty, B., & Ulasewicz, C. (2006). Faculty perspectives on moving from blackboard to the moodle learning management system. TechTrends, 50(4), 36–45. doi:10.1007/ s11528-006-0036-y

Compilation of References

Beatty, B., & Ulasewicz, C. (2006). Online teaching and learning transition: Faculty perspectives on moving from blackboard to the Moodle learning management system. TechTrends, 50(4), 36–45. doi:10.1007/s11528006-0036-y Beetham, H. (2009). E-portfolios in post-16 learning in the UK: Developments, issues and opportunities. Retrieved October 2, 2009, from http://www.jisc.ac.uk/ uploaded_documents/eportfolio_ped.doc Bennett, R. E., Rock, D. A., & Wang, M. (1991). Equivalence of free-response and multiple-choice items. Journal of Educational Measurement, 28, 77–92. doi:10.1111/j.1745-3984.1991.tb00345.x Bereby-Meyer, Y., Meyer, J., & Budescu, D. V. (2003). Decision making under internal uncertainty: The case of multiple-choice tests with different scoring rules. Acta Psychologica, 112, 207–220. doi:10.1016/S00016918(02)00085-9 Bereby-Meyer, Y., Meyer, J., & Flascher, O. M. (2002). Prospect theory analysis of guessing in multiple choice tests. Journal of Behavioral Decision Making, 15, 313–327. doi:10.1002/bdm.417 Berlanga, A. J., Sloep, P. B., Brouns, F., Bitter-Rijpkema, M. E., & Koper, R. (2008). Towards a TENCompetence e-portfolio. [iJET]. International Journal of Emerging Technologies in Learning, 3, 24–28. Bersin, J., Howard, C., O’Leonard, K., & Mallon, D. (2009). Learning management systems. Bersin & Associates. Beyer, H., & Holtzblatt, K. (1998). Contextual design, defining vustomer-centred systems. MA, USA: Morgan Kaufmann Publishers, Inc. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11, 65–70. doi:10.1016/j.iheduc.2008.03.002 Blackboard (2010). Blackboardlearn+ ANGEL edition. Retrieved on April 19, 2010, from http://www. blackboard.com/Teaching-Learning/Learn-Resources/ ANGEL-Edition.aspx

Blackboard. (2009). Engaging learners for engaging learning. Retrieved on September 7th, 2009, from http://www.blackboard.com/Teaching-Learning/LearnPlatform.aspx. Blackboard.com. (2005). Blackboard & WebCT Announce Agreement to Merge. Retrieved on December 13, 2010, from http://investor.blackboard.com/phoenix. zhtml?c=177018&p=irol-newsArticle&ID=767025. Bonk, C. J., & Graham, C. R. (2006). The handbook of blended learning. San Francisco, CA: Pfeiffer. Botturi, L., Cantoni, L., & Tardini, S. (2006). Introducing a Moodle LMS in higher education: The e-courses experience in Ticino. Retrieved on June 21, 2010, from http:// www.elearninglab.org/wp-content/uploads/2010/04/ jelks_ecourses_paper.pdf Bouhnik, D., & Marcus, T. (2006). Interaction in distancelearning courses. Journal of the American Society for Information Science and Technology, 57(3), 299–305. doi:10.1002/asi.20277 Boulakfouf, S., & Zampunieris, D. (2008). Integrating Web 2.0 features into a learning management system. Conference on e-Learning 2008. Retrieved December 29, 2010, from http://publications.uni.lu/record/1695/ files/DZ_Pages%20from%20ecel08-cd1.pdf Bredo, E. (1994). Cognitivism, situated cognition and Deweyian pragmatism. Philosophy of Education. Retrieved June, 2010, Online at http://www.ed.uiuc.edu/ EPS/PES-Yearbook/94_docs/BREDO.HTM] Bridgeman, B. (1991). Essays and multiple-choice tests as predictors of college freshman GPA. Research in Higher Education, 32, 319–332. doi:10.1007/BF00992895 Britain, S., & Liber, O. (1998). A framework for pedagogical evaluation of virtual learning environments. Retrieved on December 13, 2010, from http://www.leeds.ac.uk/ educol/documents/00001237.htm. Brkovic, M., Milosevic, D., & Krneta, R. (2006). SCos for adaptive e-learning. 28th International Conference on Information Technology Interfaces (pp. 29-34). Cavtat/ Dubrovnik.

337

Compilation of References

Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177–192. doi:10.1080/10494820600852795

Bush, M. (1999). Alternative marking schemes for online multiple choice tests. Paper presented at the 7th Annual Conference on the Teaching of Computing, Belfast. Retrieved August 14, 2010, from http://www.caacentre. ac.uk/resources/bibliography/biblio2.shtml.

Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L. (1999). Design and delivery of integrated learning systems: Their impact on student achievement and attitudes. Journal of Educational Computing Research, 21(4), 475–486.

Buzzi, C., Buzzi, M., & Leporini, B. (2009). Accessing e-learning systems via screen reader: An Example. In Jacko, M. J. A. (Ed.), Human-Computer Interaction, Part IV, HCII 2009, LNCS 5613 (pp. 21–30).

Buchanan, T. (2002). Online assessment: Desirable or dangerous? Professional Psychology, Research and Practice, 33, 148–154. doi:10.1037/0735-7028.33.2.148

Byam, N. (1995). The emergence of community in computer-mediated communication. In Jones, S. G. (Ed.), Cybersociety. Newbury Park, CA: Sage.

Bull, J., Conole, G., Davis, H. C., White, S., & Sclater, N. (2002). Rethinking assessment through learning technologies. In A. Williamson, C. Gunn, A. Young, & T. Clear (Eds.), Proceedings of the 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE2002) (Vol. 1, pp. 8-11). Auckland, New Zealand: UNITEC Institute of Technology.

Carvalho, M. K. F. (2009). Confidence judgments in real classroom settings: Monitoring performance in different types of tests. International Journal of Psychology, 44, 93–108. doi:10.1080/00207590701436744

Bundesministerium für Bildung und Forschung. (2000). Exzellenzinitiative. Retrieved on December 20, 2010, from http://www.bmbf.de/de/1321.php Bundesministerium für Bildung und Forschung. (2000). Förderprogramm Neue Medien in der Bildung. Lehr- und Lernsoftware. Retrieved on December 20, 2010, from http://www.bmbf.de/pub/neue_medien_bildung.pdf Bundesministerium für Bildung und Forschung. (2000). Qualitätspakt Lehre. Retrieved on December 20, 2010, from http://www.bmbf.de/de/15375.php Burlea, A. (2008). The complexity of an e-learning system: A paradigm for the human factor, The Inter-Networked World: ISD theory, practice and education (Vol. 2, pp. 267–278). New York, NY: Springer-Verlag. Burnett, R., & Marshall, D. P. (2003). Web theory: An introduction. New York, NY: Routledge Press. Burniske, R. W., & Monke, L. (2001). Breaking down the digital walls. Albany, NY: State University of New York Press. Bush, M. E. (2006). Quality assurance of multiple-choice tests. Quality Assurance in Education, 14, 398–404. doi:10.1108/09684880610703974

338

Catherall, P. (2004). Delivering e-learning information services in higher education. Oxford, UK: Chandos Publishing. Cavus, N., & Momani, A. M. (2009). Computer aided evaluation of learning management systems. Procedia Social and Behavioral Sciences, 1, 426–430. doi:10.1016/j. sbspro.2009.01.076 Ceraulo, S. (2005). Benefits of upgrading to an LMS. Distance Education Report, 9, 6–7. Cerioli, M., & Ribaudo. M. (2009). Great is the enemy of good: Is perfecting specific courses harmful to global curricula performances? 4th European Conference on Technology Enhanced Learning (EC-TEL 2009) (LNCS 5794). Berlin-Heidelberg, Germany: Springer. Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, J. H. (2008). A virtual learning environment for parttime MASW students: An evaluation of the WebCT. Journal of Teaching in Social Work, 28, 87–100. doi:10.1080/08841230802179027

Compilation of References

Chapman, B. (2010). LMS KnowledgeBase 2010: Indepth profiles of 90+ learning management systems, with custom comparison across 200+ features. Brandon Hall Research, Retrieved on November 9, 2010, from http:// www.brandon-hall.com/publications/lmskb/ lmskb.shtml Commonwealth of Learning (2004) LMS Evaluation Tool User Guide, 1.2, 3waynet Inc. and the Commonwealth of Learning. Chau, Y., Patrick, K., & Hu, J. (2001). Information Technology acceptance by individual professionals: A model comparison approach. Decision Sciences, 32(4), 699–718. doi:10.1111/j.1540-5915.2001.tb00978.x Cheung, B., Stewart, B., & McGreal, R. (2006) Going mobile with MOODLE: First steps. IADIS International Conference on Mobile Learning 2006, Dublin, Ireland. Chin, W. (2001). PLS Graph user’s guide, version 3.0. Houston, TX: Bauer College of Business, University of Houston. Chin, W., & Newsted, P. R. (1999). Structural equation modeling analysis with small samples using partial least squares. In Hoyle, R. H. (Ed.), Statistical strategies for small sample research (pp. 307–341). Thousand Oaks, CA: Sage Publications. Chin, W. (1998). The partial least square approach to structural equation modelling. In Marcoulides, G. A. (Ed.), Modern Methods for Business Research (pp. 295–336). London, UK: Lawrence Erlbaum Associates. Chu, L. F., Young, C. A., Ngai, L. K., Cun, T., Pearl, R. G., & Macario, A. (2010). Learning management systems and lecture capture in the medical academic environment. International Anesthesiology Clinics, 48, 27–51. doi:10.1097/AIA.0b013e3181e5c1d5 City & Guilds. (2009). How are centres using ePortfolios. Retrieved November 12, 2009, from http://www.cityand guilds.com/44912.html Claroline Consortium. (2010). Claroline documentation. Retrieved December 28, 2010, from http://doc.claroline. net/en/index.php/Main_Page.

Colace, F., & De Santo, M. (2003). Evaluating online learning platforms: A case study. Proceedings of the 36th Hawaii International Conference on System Sciences. Retrieved December 29, 2010, from http://www.hicss. hawaii.edu/HICSS36/HICSSpapers/ETWBE03.pdf Cole, J., & Foster, H. (2007). Using Moodle - Teaching with the popular open source course management system. Sebastopol, CA: O’Reilly Community Press. Common Cartridge Specification, I. M. S. (2010). Version 1.0 Final Specification. IMS Global Learning Consortium Inc. Retrieved November 12, 2010, from http://www. imsglobal.org/cc/index.html. Compeau, D., & Higgins, C. (1995). Computer selfefficacy: Development of a measure and initial test. Management Information Systems Quarterly, 19, 189–211. doi:10.2307/249688 Connolly, T. M., & Stansfield, M. H. (2007b). From elearning to online games-based e-learning: Implication and challenges for higher education and training. In Li, F. (Ed.), Social implications and challenges of e-business. Hershey, PA: Idea-Group Publishing. Connolly, T. M. & Stansfield, M. H. (2007a). From e-learning to games-based e-learning. International Journal of Information Technology and Management, 26(2/3/4), 188-208. Conole, G., de Laat, M., Dillon, T., & Darby, J. (2006). JISC LXP: Student experiences of technologies: Final Report: Joint Information Systems Committee. Retrieved October 7, 2010, from http://www.jisc.ac.uk/. Coombs, C., Milholland, J., & Womer, F. (1956). The assessment of partial knowledge. Educational and Psychological Measurement, 16, 13–37. Corbeil, J. R., & Valdes-Corbeil, M. E. (2007). Are you ready for mobile learning? EDUCAUSE Quarterly, 30(2). Cradler, J. (2008). Research on e-learning. Learning and Leading with Technology, 3(5), 54–57. Crano, W., & Brewer, M. (2002). Principles and methods of social research. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

339

Compilation of References

Curyer, S., Leeson, J., Mason, J., & Williams, A. (2007). Developing e-portfolios for VET: Policy issues and interoperability. Australian flexible learning framework. Retrieved October 16, 2009, from http://e-standards. flexiblelearning.net.au/docs/vet-eportfolio-report-v1-0. pdf Curzon, L. B. (1997). Teaching in further education: An outline of principles and practice (5th ed.). London, UK: Cassell. Dabbagh, N. (2005). Pedagogical models for e-learning: A theory-based design framework. International Journal of Technology in Teaching and Learning, 1(1), 25–44. Dagger, D., O’Connor, A., Lawless, S., Walsh, E., & Wade, V. (2007). Service oriented e-learning platforms: From monolithic systems to flexible services. IEEE Internet Computing Special Issue on Distance Learning. Dalsgaard, C. (2006). Social software: E-learning beyond learning management systems. European Journal of Open, Distance and E-learning. Daniels, P. (2009). Course management systems and implications for practice. International Journal of Emerging Technologies & Society, 7(2), 97–108. Davenport, T. H., & Prusak, L. (1998). Working knowledge: How organizations manage what they know. Boston, MA: Harvard Business Review Press. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13, 319–339. doi:10.2307/249008 Davis, B., Carmean, C., & Wagner, E. D. (2009). The evolution of the LMS: From management to learning Deep analysis of trends shaping the future of e-learning. Sage Road Solutions. LLC. De Laat, M., & Lally, V. (2003). Complexity, theory and praxis: Researching collaborative learning and tutoring processes in a networked learning community. Instructional Science, 31, 7–39. doi:10.1023/A:1022596100142 De Laat, M., Lally, V., Lipponen, L., & Simons, R. J. (2007). Online teaching in networked learning communities: A multi-method approach to studying the role of the teacher. Instructional Science, 35, 257–286. doi:10.1007/ s11251-006-9007-0

340

DeBord, K. A., Aruguete, M. S., & Muhlig, J. (2004). Are computer-assisted teaching methods effective? Teaching of Psychology, 31, 65–68. doi:10.1207/ s15328023top3101_13 DeLone, W., & McLean, E. (2003). The Delone and Mclean model of Information Systems success: A 10-year update. Journal of Management Information Systems, 19(4), 9–30. DeLone, W. H., & McLean, E. R. (1992). Information Systems success: The quest for the dependent variable. Information Systems Research, 3, 60–95. doi:10.1287/ isre.3.1.60 Dennen, V. P. (2008). Looking for evidence of learning: Assessment and analysis methods for online discourse. Computers in Human Behavior, 24, 205–219. doi:10.1016/j.chb.2007.01.010 DeRouin, R. E., Fritzsche, B. A., & Salas, E. (2005). Elearning in Organizations. Journal of Management, 31(6), 920–940. doi:10.1177/0149206305279815 Deshpande, M., & Karypis, G. (2002). Using conjunction of attribute values for classification. Proceedings of the 11th International Conference on Information and Knowledge Management, McLean, VA (pp. 356-364). Deutsches Zentrum für Luft- und Raumfahrt (DLR). (2009). Projektträger im DLR – Neue medien in der bildung, Retrieved on December 20, 2010, from http:// www.dlr.de/pt/desktopdefault.aspx/tabid-5882/9540_ read-18559/ Dey, A. K., & Abowd, G. D. (1999). Towards a better understanding of context and context-awareness. GVU Technical Report GIT-GVU-99-22. College of Computing, Georgia Institute of Technology, Atlanta, Georgia. Dillenbourg, P. (2008). Integrating technologies into educational ecosystems. Distance Education, 29(2), 127–140. doi:10.1080/01587910802154939 Dillon, P., Wang, R., & Tearle, P. (2007). Cultural disconnection in virtual education. Pedagogy, Culture & Society, 15, 153–174. doi:10.1080/14681360701403565 Dobson, J. L. (2008). The use of formative online quizzes to enhance class preparation and scores on summative exams. Advances in Physiology Education, 32, 297–302. doi:10.1152/advan.90162.2008

Compilation of References

Doll & Torkzadeh, G. (1988). The measurement of end user computing satisfaction, IS Quarterly, 12(2), 259-274.

Ellis, R. K. (2009). Field guide to learning management systems, ASTD learning circuits.

Dorninger, C., & Schrack, C. (2007). E-portfolio’s in education-learning tools or means of assessment? In Proceedings of the International Computer-Aided Learning Conference (ICL2007), Villach, Austria.

Ellis, R. K. (2009a). Field guide to learning management systems. Learning circuits, American society for training and development. Retrieved December 4, 2010, from http://www.astd.org/NR/rdonlyres/12ECDB993B91-403E-9B15-7E597444645D/23395/LMS_fieldguide_20091.pdf

Downes, S. (2005). E-learning 2.0, eLearn, 2005 (10). Drigas, A., Koukianakis, L., & Papagerasimou, Y. (2006). An e-learning environment for non-traditional students with sight disabilities. 36th Annual Frontiers in Education Conference, San Diego, CA (pp. 23-27). Dror, I. E. (2008). Technology enhanced learning: The good, the bad, and the ugly. Pragmatics & Cognition, 16(2). Amsterdam, Netherlands: John Benjamins Publishing Company. Dżega, D., & Pietruszkiewicz, W. (2010). The support of e-learning platform management by the extraction of activity features and clustering based observation of users. In S. Konstantopoulos, S. Perantonis, V. Karkaletsis, C. Spyropoulos, & G. Vouros (Eds.), Artificial Intelligence: Theories, Model and Applications-SETN 2010 (pp. 315-320). LNAI 6040. Heidelberg, Berlin, Germany: Springer-Verlag. Ebner, M., Kickmeier-Rust, M., & Holzinger, A. (2008). Utilizing Wiki-Systems in higher education classes: A chance for universal access? Springer Universal Access in Information Society International Journal, 7(4). EDUCAUSE. (2005).Retrieved on February 11th, 2011, from E-Portfolios Web page, URL: http://www.educause. edu/ELI/Archives/EPortfolios/5524 El Tartoussi, I. (2009). Networked readiness in the United Arab Emirates, The 2nd Annual Forum on E-learning Excellence in the Middle East 2009, Dubai. UAE. Elan (n. d.). E-Learning Academic Network Niedersachsen. E-Learning Academic Network Niedersachsen. Retrieved on August 17, 2010, from http://www.elanniedersachsen.de Elias, T. (2010). Universal instructional design principles for Moodle, Canada International Review of Research in Open and Distance Learning, 11(2). May – 2010.

Ellis, R. K. (2009b). Learning management systems 2009. Learning circuits and e-learning news, American society for training and development. Retrieved December 4, 2010, from http://www.astd.org/LC/2009/0509_LMS2009.htm. Ellis, R. K. (2010). Learning Management Systems 2010. Learning Circuits and E-Learning News, American Society for Training and Development. Retrieved December 4, 2010, from http://www.astd.org/LC/2010/0910_LMSsurvey.htm. Erlin, Y. N., & Rhaman, A.A. (2009). Students’ Interactions in Online Asynchronous Discussion Forum: A Social Network Analysis. International Conference on Education Technology and Computer, 25-29. Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., & Mong, C. (2007). Using peer feedback to enhance the quality of student online postings: An exploratory sStudy. Journal of ComputerMediated Communication, 12, 412–433. doi:10.1111/ j.1083-6101.2007.00331.x Eteokleous-Grigoriou, N. (2009). Instilling a new learning, work and communication culture through systemically integrated technology in education. Systems Research and Behavioral Science, 26, 707–716. doi:10.1002/sres.983 European Commission. (2009). ECTS users’ guide. Luxembourg: Office for official publications of the European communities, p. 60. Retrieved July 19, 2010, from http:// ec.europa.eu/education/lifelong-learning-policy/doc/ects/ guide_en.pdf European Commission. (2010). The bologna process Towards the european higher education area. Retrieved on August 30, 2010, from http://ec.europa.eu/education/ higher-education/doc1290_en.htm

341

Compilation of References

Evans, L., Ingersoll, R. W., & Smith, E. J. (1966). The reliability, validity and taxonomic structure of the oral examination. Journal of Medical Education, 41, 651–657. Falvo, D. A., & Johnson, B. F. (2007). The use of learning management systems in the United States. TechTrends, 51(2), 40–45. doi:10.1007/s11528-007-0025-9 Faßbender, H., & Miosge, C. (2010). Technische Universität Braunschweig. Hervorragende Qualität und viel Raum für eigene Ideen. In Winde, M. (Ed.), Von der Qualitätsmessung zum Qualitätsmanagement. Praxisbeispiele an Hochschulen (pp. 48–57). Essen, Germany: Edition Stifterverband. Faux, F., McFarlane, A. E., Roche, N., & Facer, K. (2006). Handhelds: Learning with handheld technologies. Bristol, UK: Futurelab. Feldstein, M. (2004). What’s important in a learning content management system. eLearn Magazine. Retrieved December 28, 2010, from http://www.elearnmag.org/ subpage.cfm?section=tutorials&article=4-1 Feldstein, M., & Masson, P. (2006). Unbolting the chairs: Making learning management systems more flexible. Elearn magazine. Retrieved from http://www.elearnmag. org/subpage.cfm?section=tutorials&article=22-1 on April 19, 2010. Fichten, C. S., Ferraro, V., Asuncion, J. V., Chwojka, C., Barile, M., Nguyen, M. N., & Wolforth, J. (2009). Disabilities and e-learning problems and solutions: An exploratory study. Journal of Educational Technology & Society, 12(4), 241–256. Fielding, R. (2000). Architectural styles and the design of network-based software architectures. Phd dissertation. Fleck, W. (2010, August 24). Evaluationsergebnisse von studierendeninterviews zu Stud.IP, Sommer 2010. Unpublished manuscript. Goethe Universität Frankfurt am Main. E-Learning Center am HRZ. Retrieved August 30, 2010, from http://www.rz.uni-frankfurt.de/internet/ elearning/index.html

342

Folajimi, Y. (2009). Transforming distant pedagogical learning to Web based collaborative system: An Intelligent Tutoring Systems Architecture. In T. Bastiaens et al. (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009 (pp. 1618–1622). Chesapeake, VA: AACE. Forment, M., & Guerrero, J. C. (2008, April 11-13). Moodlbile: Extending Moodle to the mobile on/offline Scenario. Proceedings of the IADIS International Conference on Mobile Learning, Algarve, Portugal. Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error. JMR, Journal of Marketing Research, 18, 39–50. doi:10.2307/3151312 Franklin, T., & Harmelen, M. (2007). Web 2.0 for content for learning and teaching in higher education. Joint Information Systems Committee (JISC) report. Retrieved August 8, 2010, from http://ie-repository.jisc.ac.uk/148/1/ web2-content-learning-and-teaching.pdf Freeman, R., & Lewis, R. (1998). Planning and Implementing Assessment. London, UK: Kogan Page. Froehlich, T. J. (1994). Relevance reconsidered - Towards an agenda for the 21st century: Introduction to special topic issue on relevance research. Journal of the American Society for Information Science American Society for Information Science, 45(3), 124–134. doi:10.1002/(SICI)1097-4571(199404)45:3<124::AIDASI2>3.0.CO;2-8 Fu, X., & Wang, L. (2005). Data dimensionality reduction with application to improving classification performance and explaining concepts of data sets. International Journal of Business Intelligence and Data mining, 1(1), 65-87. Gaeta, M., Orciuoli, F., & Titrovato, P. (2009). Advanced ontology management sytem for personalised e-learning. Knowledge-Based Systems, 22, 292–301. doi:10.1016/j. knosys.2009.01.006 Ganguli, S., Pedrosa, I., Yam, C.-S., Appignani, B., Siewert, B., & Kressel, H. Y. (2006). Part I: Preparing first-year radiology residents and assessing their readiness for on-call responsibilities. Academic Radiology, 13, 764–769. doi:10.1016/j.acra.2006.02.057

Compilation of References

García, E., Romero, C., Ventura, S., & Calders, T. (2007). Drawbacks and solutions of applying Association Rule mining in learning management systems. Proceedings of the International Workshop on Applying Data mining in e-Learning, Crete, Greece (pp. 13-22). Garrison, R. (2000). Theoretical challenges for distance education in the 21st century: A shift from structural to transactional issues. International Review of Research in Open and Distance Learning, 1(1), 1–17. Garrison, R. (1997). Computer conferencing: The postindustrial age of distance education. Open Learning, 12(2), 3–11. doi:10.1080/0268051970120202 Garrison, D. R., & Vaughan, N. D. (2007). Blended learning in higher education: Framework, principles, and guidelines. San Francisco, CA: Jossey-Bass. Gaspay, A., Dardan, S., & Legorreta, L. (2008). Distance learning through the lens of learning models: New outlets for innovation. Review of Business Research, 8(4), 53–62. Georgouli, K., Skalkidis, I., & Gerreiro, P. (2008). A framework for adopting LMS to introduce e-learning in a traditional course. Journal of Educational Technology & Society, 227–240. Gerring, J. (2007). Case study research: Principles and practices. Cambridge, UK: Cambridge University Press. Gilbert, T. (2009). Leveraging Sakai and IMS LTI to standardize integrations. In 10th Sakai Conference Pearson Education. Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K. (1991). A comprehensive study of the effects of an integrated learning system. Indiana State University. Ginsberg, R., & McCormick, V. (1998). Computer use in effective schools. Journal of Staff Development, 19(1), 22–25. Goffman, E. (1978). Behavior in public places. Notes on the social organization of gatherings (4th ed.). New York, NY: Free Press. Graf, S., & Kinshuk (2006). Considering learning styles in learning management systems: Investigating the behavior of students in an online course. Proceedings of the 1st IEEE International Workshop on Semantic Media Adaptation and Personalization, Athens, Greece (pp. 25-30).

Gras, M. (2005). Internet usage and digital divide among Spanish university students. Revista Media e Jornalismo - A Europa e os Media:14(8)(1). Grasha, T. (1990). The naturalistic approach to learning styles. College Teaching, 38(3), 106–113. Gray, K., Chang, S., & Kennedy, G. (2010). Use of social Web technologies by international and domestic undergraduate students: Implications for learning and teaching in Australian universities. Technology, Pedagogy and Education, 19(1), 31–46. doi:10.1080/14759390903579208 Grosseck, G. (2009). To use or not to use Web 2.0 in higher education? Procedia Social and Behavioral Sciences, 1, 478–482. doi:10.1016/j.sbspro.2009.01.087 Gunawardena, C. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2/3), 147–166. Gunawardena, C., & Zittle, F. (1997). Social presence as a predictor of satisfaction within a computer mediated conferencing environment. American Journal of Distance Education, 11(3), 8–26. doi:10.1080/08923649709526970 Gunawardena, C. N. (1993). The Social context of online education. In Proceedings of the Distance Education Conference, Portland, Oregon. Gunawardena, C., Carabajal, K., & Lowe, C. A. (2001). Critical analysis of models and methods used to evaluate online learning networks. Education Resources Information Center (ERIC). Retrieved December 29, 2010, from http://eric.ed.gov/PDFS/ED456159.pdf GUnet Asynchronous eLearning Group (2010). Open e-class asynchronous e-learning platform - Platform description. Greek Academic Network. Retrieved December 28, 2010, from http://eclass.gunet.gr/manuals/ OpeneClass23_en.pdf. Guri-Rosenblit. (2005). ‘Distance education’ and ‘elearning’: Not the same thing. Higher Education, 49, 467–493. doi:10.1007/s10734-004-0040-0 Hair, J., Anderson, R., Tatham, R., & Black, W. (1998). Multivariate data anlaysis. Upper Saddle River, NJ: Prentice Hall.

343

Compilation of References

Haldane, A. (1998). The convergence of e-learning and knowledge management: Fusion or confusion? People development in a knowledge economy, 2. Learning Futures Ltd., Retrieved on April 26, 2010 from http://www.learningfutures.co.uk/LFdocs/ MKelleher_06-01-03_15-29-27.pdf.

Holden, C. (2004). What we mean when we say “repositories”, user expectations of repository systems, Academic ADL Co-Lab. Retrieved on December 13, 2010, from http://www.hewlett.org/NR/rdonlyres/158FC043-A56F43C6-ABA7-EB9A62656FCB/0/RepoSurvey2004-1. pdf.

Hall, J. (2003). Assessing learning management systems. Chief learning officer. Retrieved on April 19, 2010, from http://www.clomedia.com/features/2003/January/91/ index.php.

Holmberg, B., & Ortner, G. E. (1991). Research into distance education. Frankfurt am Main, Germany: Peter Lang.

Hamuy, E., & Galaz, M. (2010). Information versus communication in course management system participation. Computers & Education, 54, 169–177. doi:10.1016/j. compedu.2009.08.001 Harasim, L. (2006). History of e-learning: Shift happened. The international handbook of virtual learning environment. Berlin-Heidelberg, Germany: Springer-Verlag. Harman, L., & Koohang, A. (2007). Learning objects: Standards, metadata, repositories, and LCMS. Information Sciense Institute, Informing Sciense. Hawkins, B. L., & Rudy, J. A. (2007). Educause Core Data Service. Fiscal Year 2006 Summary Report. CO, USA: Educause. Hayashi, A., Chen, C., Ryan, T., & Wu, J. (2004). The role of social presence and moderating role of computer self efficacy in predicting the continuance usage of e-learning systems. Journal of Information Systems Education, 15(2), 139–154. Henry, S. L. (2007). Just ask: Integrating accessibility throughout design. Madison, WI: ET\Lawton. Available at www.uiAccess.com/justask/ Hew, K. F., & Cheung, W. S. (2008). Attracting student participation in asynchronous online discussions: A case study of peer facilitation. Computers & Education, 51, 1111–1124. doi:10.1016/j.compedu.2007.11.002 HochschulInformationsSystem. (n. d.). Lehre, Studium, Forschung. Retrieved August 30, 2010, from http://www. his.de/abt1/ab10

344

Holmberg, B. (1995). Theory and practice of distance education, 2nd edition. London, UK, and New York, NY: Routledge. Holmes, B., & Gardner, J. (2006). E-learning concepts and practice. London, UK: SAGE Publications. Holtzblatt, K., & Jones, S. (1994). Contextual inquiry: A participatory design technique for system design. In Schuler, D., & Namioka, A. (Eds.), Participatory design: Principles and practice. Englewood Cliffs, NJ: Prentice-Hall. Homberg, B. (2005). The Evolution, Principles and Practices of Distance Education (Vol. 11). Hagen, Germany: FernUniversität, ZIFF. Horton, W. (2003). E-learning tools and technologies: A consumer’s guide for trainers, teachers, educators, and instructional designers. Indianapolis, IN, USA: Wiley. Horton, W. (2006). E-learning by design. San Francisco, CA: John Wiley & Sons, Inc. Houser, C., & Thornton, P. (2005). Poodle: A coursemanagement system for mobile phones. In Proc. of the 4th IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE ’05). IEEE Computer Society. Hsu, J. (2002). Data mining trends and developments: The key data mining technologies and applications for the 21st century, Proceedings of the19th Annual Conference for Information Systems Education (ISECON 2002), San Antonio, TX (Art 224b). Huang, S., & Yang, C. (2009). Designing a semantic bliki system to support different types of knowledge and adaptive learning. Computers & Education, 53, 701–712. doi:10.1016/j.compedu.2009.04.011

Compilation of References

IEEE LOM. (2002). IEEE Standard for Learning Object Metadata IEEE 1484.12.1-2002. Retrieved on November 12, 2010, from http://www.ieeeltsc.org/standards/1484-12-1-2002/. Igbaria, M. (1990). End-user computing effectiveness: A structural equation model. Omega, 18(6), 637. doi:10.1016/0305-0483(90)90055-E Igbaria, M., Guimaraes, T., & Davis, G. B. (1995). Testing the determinants of microcomputer usage via a structural equation model. Journal of Management Information Systems, 11(4), 87–114. Igel, C. (2009). Learning-management an hochschulen. In Issing, L. J., & Klimsa, P. (Eds.), Online-Lernen. Handbuch für Wissenschaft und Praxis (pp. 207–215). München, Germany: Oldenbourg. Iglesias, A., Moreno, L., Revuelta, P., & Jiménez, J. (2009). APEINTA: A Spanish educational project aiming for inclusive education in and out of the classroom. ITiCSE ‘09, Paris, France (p. 393). New York, NY: ACM, NY. ILIAS. (n. d.). ILIAS Information Center. Retrieved August 30, 2010, from http://www.ilias.de IMS CP. (2007). IMS Content Packaging v1.2 Final specification. Retrieved on November 12, 2010, from http://www.imsglobal.org/content/packaging/. IMS Global Learning Consortium. (2006a). IMS question and test interoperability overview, version 2.1 public draft (revision 2) specification. Retrieved December 5, 2010, from http://www.imsglobal.org/question/qtiv2p1pd2/ imsqti_oviewv2p1pd2.html Jaffer, S., Ng’ambi, D., & Czerniewicz, L. (2006). The role of ICTs in higher education in South Africa: One strategy for addressing teaching and learning challenges. Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa, 10-21 July. Retrieved from http://emerge2006.net. Janossy, J. (2008). Proposed model for evaluating C/LMS faculty usage in higher education institutions. Conference MBAA 2008.

Janossy, J., & Hover, T. (2008). Proposed model for evaluating C/LMS usage. In K. McFerrin et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2008 (pp. 2979-2986). Chesapeake, VA: AACE. Jaroszewicz, S., & Simovici, D. A. (2004). Interestingness of frequent itemsets using Bayesian networks as background knowledge. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (pp. 178-186). Seattle, WA. JISC. (2008). Effective practice with e-portfolios: Supporting 21st century learning. Joint information systems committee. London, UK: JISC. Jonassen, D. H. (1996). Computer-mediated communication: Connecting communities of learners. Computers in the Classroom (158–182). Edgewood Cliffs, NJ: Prentice-Hall, Inc. Jones, D. (2004). The conceptualisation of e-learning: Lessons an implications. Studies in Learning, Evaluation. Innovation and Development, 1(1), 47–55. JORUM team. (2006). E-learning repository systems research watch. Retrieved on December 13, 2010, from http://www.jorum.ac.uk/docs/pdf/Repository_Watch_final_05012006.pdf. Jovanovic, J., Devedzic, V., Gasevic, D., Hatala, M., Eap, T., Richards, G., & Brooks, C. (2007). Using Semantic Web technologies to analyze learning content. Internet computing, 11(5), 45–53. Kakasevski, G., Mihajlov, M., Arsenovski, S., & Chungurski, S. (2008). Evaluating usability in learning management system Moodle. Proceedings of the ITI 2008 30th Int. Conf. On Information Technology Interfaces (pp. 613-618). Kalnins-Cole, T., & Peters, D. (2007). Towards an accessible learning management system. International Journal of Continuing Engineering Education and Lifelong Learning, 17(1), 84–96. doi:10.1504/IJCEELL.2007.013232 Kaltenbaek, J. (2009). Hochschule online - Online lehren und lernen in der Hochschule. In Issing, L. J., & Klimsa, P. (Eds.), Online-Lernen. Handbuch für Wissenschaft und Praxis (pp. 367–388). München, Germany: Oldenbourg.

345

Compilation of References

Keegan, D. (2005). Mobile learning: The next generation of learning. Distance Education International. Keegan, D. (2002, November). The future of learning: From eLearning to mLearning. ZIFF Papiere 119. Hagen, Germany: FernUniversitat, Ziff. Kekwaletswe, R. M. (2006). Social presence and context awareness for knowledge transformation in an m-learning environment. In Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa (10-21 July). Retrieved from http://emerge2006.net. Kekwaletswe, R. M. (2007). Knowledge transformation in a mobile learning environment: An inquiry of context and social presence awareness. PhD thesis. South Africa: University of Cape Town. Kekwaletswe, R. M. (2009). Conceptualizing ubiquitous learning through context-aware wired and wireless Web services. In Proceedings of the IADIS Mobile Learning conference, Barcelona, Spain (25 – 28 February). Kekwaletswe, R. M., & Ng’ambi, D. (2006). Ubiquitous social presence: Context-awareness in a mobile learning environment. In Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing, Taichung, Taiwan (90-95). Taichung, Taiwan.

Kettinger, W. J., & Lee, C. C. (1994). Perceived service quality and user satisfaction with the Information Services function. Decision Sciences, 25(5/6), 737–765. doi:10.1111/j.1540-5915.1994.tb01868.x Kibble, J. (2007). Use of unsupervised online quizzes as formative assessment in a medical physiology course: Effects of incentives on student participation and performance. Advances in Physiology Education, 31, 253–260. doi:10.1152/advan.00027.2007 Kim, S. W., & Lee, M. G. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24(4), 284–294. doi:10.1111/j.1365-2729.2007.00260.x Kim, K., & Bonk, C. J. (2006). The future of online teaching and learning in higher education: The survey says. EDUCAUSE Quarterly, 29(4). Kirkwood, A., & Price, L. (2005). Learners and learning in the 21st century: What do we know about students’ attitudes and experiences of ICT that will help us design courses? Studies in Higher Education, 30(3), 257–274. doi:10.1080/03075070500095689 Kirkwood, A. (2009). E-learning: You don’t always get what you hope for. Technology, Pedagogy and Education, 18(2), 107–121. doi:10.1080/14759390902992576

Kelly, T., & Bauer, D. (2004). Managing intellectual capital-via e-learning-at Cisco. In Holsapple, C. (Ed.), Handbook on Knowledge Management 2: Knowledge directions (pp. 511–532). Berlin, Germany: Springer.

Kirner, T. G., Custódio, C. A., & Kirner, C. (2008). Usability evaluation of the Moodle system from the teachers’ perspective. IADIS International Conference e-Learning (pp. 371-378).

Kennedy, G., Dalgarno, B., Sturt, C., Bennett, S., Gray, K., Waycott, J., & Judd, T. …Chang, R. (2009). Educating the Net generation. A handbook of findings for practice and policy. Australia: Australian Learning and Teaching Council.

Kleimann, B., & Wannemacher, K. (2004). E-learning an deutschen hochschulen: Von der projektentwicklung zur nachhaltigen Implementierung. Hannover, Germany: HIS.

Kerres, M. (2005). Strategieentwicklung für die nachhaltige implementation neuer medien in der hochschule. In T. Pfeffer (Ed.), Medien in der Wissenschaft, Vol. 32. Handbuch Organisationsentwicklung. Neue Medien in der Lehre: Dimensionen, Instrumente, Positionen (pp. 147–162). Münster, Germany: Waxmann.

346

Klemettinen, M., Mannila, H., Ronkainen, P., Toivonen, H., & Verkamo, A. I. (1994). Finding interesting rules from large sets of discovered association rules. Proceedings of the 3rd International Conference on Information and Knowledge Management (CIKM’94) Gaithersburg, Maryland, USA (pp. 401-408). Kneght, M., & Reid, K. (2009). Modularizing Information Literacy training via the Blackboard eCommunity. Journal of Library Administration, 49, 1–9. doi:10.1080/01930820802310502

Compilation of References

Kozaris, I. A. (2010). Platforms for e-learning. Analytical and Bioanalytical Chemistry, 397(3), 893–898. doi:10.1007/s00216-010-3587-x Kozma, R. (1987). The implications of cognitive psychology for computer-based learning tools. Educational Technology, 27(11), 20–25. Kritikou, Y., Demestichas, P., Adamopoulou, E., Demestichas, K., Theologou, M., & Paradia, M. (2008). User profile modeling in the context of Web-based learning management systems. Journal of Network and Computer Applications, 31(4), 603–627. doi:10.1016/j. jnca.2007.11.006 Kubicek, H., Breiter, A., Fischer, A., & Wiedewald, C. (2004). Organisatorische einbettung von e-learning an deutschen hochschulen. Retrieved on August 30, 2010, from http://www.ifib.de/publikationsdateien/MMKH_ Endbericht_2004-05-26.pdf Kubora, K., Terashima, K., Nakahashi, Y., & Morioka, H. (2008). Analyzing learning environments of distance education in Japan, distance learning and the Internet, Tokyo, Japan. Kukulska-Hulme, & A., Traxler, J. (2005). Mobile learning – A handbook for educators and trainers. UK: Taylor & Francis. Kwak, H., Fox, R. & Zinkhan, G. (2002). What products can be successfully promoted and sold via the Internet? Journal of Advertising Research, Jan.-Feb. Lahwal, F., Amaimin, M., & Al-Ajlan, A. (2009). Perception cultural impacts: Principles for trainers’ skills for e-learning. NCM 2009 - 5th International Joint Conference on INC, IMS, and IDC (Article number 5331619, pp. 986-993). Lam, P., McNaught, C., & Cheng, K. F. (2008). Pragmatic meta-analytic studies: learning the lessons from naturalistic evaluations of multiple cases. ALT-J, 16(2), 61–79. doi:10.1080/09687760802315879

Lam, P., Keing, C., McNaught, C., & Cheng, K. F. (2006). Monitoring e-learning environments through analyzing Web logs of institution-wide e-learning platforms. In L. Markauskaite, P. Goodyear, & P. Reimann (Eds.), Who’s learning? Whose technology? (pp. 429–440). Proceedings of the 23rd Annual Australian Society for Computers in Learning in Tertiary Education 2006 conference, Sydney, Australia. Retrieved December 29, 2010, from http:// www.ascilite.org.au/conferences/sydney06/proceeding/ pdf_papers/p62.pdf Lan, Y. F., & Yan, C. L. (2009). A practical approach to encourage students’ participation in asynchronous online discussions based on expectancy theory. International Conference on Virtual Environments, Human-Computer Interfaces and Measurements Systems (Article number 5068907, pp. 271-276). Landsberger, J. (2004). Thoughts on convergence in instructional settings. TechTrends, 48, 6–8. doi:10.1007/ BF02763348 Laudon, K. C., & Laudon, J. P. (1995). Information systems: A problem-solving approach (3rd ed.). Orlando, FL: Dryden Press. Laurillard, D. (2005). E-learning in higher education. In Ashwin, P. (Ed.), Changing higher education: The development of learning and teaching. London, UK: RoutledgeFalmer. Leal, J. & Queirós, R. (2010). From e-learning systems to specialised e-learning services. Chapter of EduJudge project book - A new learning paradigm: Competition supported by technology. Leal, J. P., & Queirós, R. (2009). CrimsonHex: A service oriented repository of specialised learning objects. In ICEIS 2009, 11th International Conference on Enterprise Information Systems. Lee, Y. (2006). An empirical investigation into factors influencing the adoption of an e-learning system. Online Information Review, 30(5), 517–541. doi:10.1108/14684520610706406

347

Compilation of References

Lehsten, P., Zender, R., Lucke, U., & Tavangarian, D. (2010, March 29-April 7). A service-oriented approach towards context-aware mobile learning management systems. 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops) (pp 268-273). Levensaler, L., & Laurano, M. (2009). Talent management systems 2010. Bersin & Associates. Levin, D., & Arafeh, S. (2002). The digital disconnect: The widening gap between Internet savvy students and their schools. Pew Internet & American Life Project. Retrieved December 29, 2010, from http://www.pewinternet.org/~/ media//Files/Reports/2002/PIP_Schools_Internet_Report.pdf.pdf Lewis, B. A., MacEntee, V. M., DeLaCruz, S., Englander, C., Jeffrey, T., & Takach, E. …Jason W. (2005). Learning management systems comparison. Proceedings of the 2005 Informing Science and IT Education Joint Conference, Flagstaff, Arizona. Li, W., Han, J., & Pei, J. (2001). CMAR: Accurate and efficient classification based on multiple class-Association rules. Proceedings of the 2001 IEEE International Conference on Data mining (pp. 369-376). Liaw, S., Huang, H., & Chen, G. (2007). Surveying instructor and learner attitudes toward e-learning. Computers & Education, 49, 1066–1080. doi:10.1016/j. compedu.2006.01.001 Liu, Y. (2004). Faculty development and CMS. In Proceedings of Society for Information Technology in Teacher Education International Conference, 2004, 2409–2412. Liu, H., & Yang, M. (2005). QoL guaranteed adaptation and personalization in e-learning systems. IEEE Transactions on Education, 48(4), 676–687. doi:10.1109/ TE.2005.858398 Lonsdale, P., Baber, C., Sharples, M., & Arvanitis, T. N. (2003). A context awareness architecture for facilitating mobile learning. In Proceedings of MLEARN 2003 (LSDA), London.

348

Lopes, C. (2007). Evaluating e-learning readiness in a health sciences higher education institution. In M. Baptista Nunes & M. McPherson (Eds.), Proceedings of the IADIS International Conference e-Learning 2007 (vol. 1, pp. 59-67). Lorenzo, G., & Ittelson, J. (2005). An overview of ePortfolios, Educause learning initiative. Lukhele, R., Thissen, D., & Wainer, H. (1994). On the relative value of multiple-choice, constructed response, and examinee-selected items on two achievement tests. Journal of Educational Measurement, 31, 234–250. doi:10.1111/j.1745-3984.1994.tb00445.x Lytras, M., & Pouloudi, A. (2006). Towards the development of a novel taxonomy of knowledge management systems from a learning perspective: An integrated approach to learning and knowledge infrastructures. Journal of Knowledge Management, 10 6), 64-80. MacKeogh, K., & Fox, S. (2009). Strategies for embedding e-learning in traditional universities: Drivers and barriers. Electronic Journal of e-Learning, 7(2). MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability (pp. 281-297). Madell, D., & Muncer, S. (2004). Gender differences in the use of the Internet by English secondary school children. Social Psychology of Education, 7, 229–251. doi:10.1023/B:SPOE.0000018552.85903.4d Mahdizadeh, H., Biemans, H., & Mulder, M. (2008). Determining factors of the use of e-learning environments by university teachers. Computers & Education, 51, 142–154. doi:10.1016/j.compedu.2007.04.004 Malikowski, S. R., Thomson, M. E., & Theis, J. G. (2007). A model for research into course management systems: Bridging technology and learning theory. Journal of Educational Computing Research, 36(2). doi:10.2190/10021T50-27G2-H3V7 Manning, S., & Dix, A. (2008). Identifying students’ mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives. Computers & Education, 51, 1154–1171. doi:10.1016/j. compedu.2007.10.010

Compilation of References

Martin, L., Gutiérrez-Restrepo, E., & Barrera, C. Rodríguez-Ascaso1, A., Santos, Olga C. & Boticario, J. G. (2007). Usability and accessibility evaluations along the e-learning cycle, IWWUA - WISE 2007 Workshops, LNCS Volume 4832/2007, Nancy, Francia (pp. 453—458). Martínez, J. Á., & Navarra, P. L. (2007). Content interoperability on e-learning platforms: Standardization, digital libraries, and knowledge management, Revista da Universidad y Sociedad del. Conocimiento. Mason, R., Pegler, C., & Weller, M. (2004). E-portfolios: An assessment tool for online courses. British Journal of Educational Technology, 35(6), 717–727. doi:10.1111/ j.1467-8535.2004.00429.x Mattheos, N., Stefanovic, N., Apse, P., Attstrom, R., Buchanan, J., Brown, P., & Walmsley, A. D. (2008). Potential of Information Technology in dental education. European Journal of Dental Education, 12(Suppl. 1), 85–91. doi:10.1111/j.1600-0579.2007.00483.x Mayadas, F. (2007). Online education: Worldwide and moving quickly. In Technology- Enabled Education (pp. 57-60), Proceeding of the 2007 MIT LINC Conference, Dubai, UAE. Mayes, J. T., Morrison, D., Bullen, P., Mellar, H., & Oliver, M. (Eds.). (2009). Transforming higher education through technology-enhanced learning. York, UK: Higher Education Academy. Mazza, R., & Milani, C. (2005). Exploring usage analysis in learning systems: Gaining insights from visualizations. 12th International Conference on Artificial Intelligence in Education (AIED 2005). The Netherlands: Amsterdam. Retrieved December 29, 2010, from http://linux3. dti.supsi.ch/~mazza/Web_area/Pubblicazioni/AIED05/ aied-ws2005.pdf Mazzolini, M., & Maddison, S. (2003). Sage, guide or ghost? The effect of instructor intervention on student participation in online discussion forums. Computers & Education, 40, 237–253. doi:10.1016/S03601315(02)00129-X

McCoubrie, P., & McKnight, L. (2008). Single best answer MCQs: A new format for the FRCR part 2a exam. Clinical Radiology, 63, 506–510. doi:10.1016/j.crad.2007.08.021 McDonald, J. (2002). Is “as good as face-to-face” as good as it gets? Journal of Asynchronous Learning Networks, 6(2), 10–23. McGee, P., & Diaz, V. (2007). Wikis and podcasts and blogs! Oh, my! What is a faculty member supposed to do? EDUCAUSE Review, 42(5). McKelvie, G., Dotsika, F., & Patrick, K. (2007). Interactive business development, capturing business knowledge and practice: A case study. The Learning Organization, 14(5), 407–422. doi:10.1108/09696470710762637 McKenzie, C. R. M., Wixted, J. T., Noelle, D. C., & Gyurjyan, G. (2001). Relation between confidence in yes-no and forced-choice taks. Journal of Experimental Psychology. General, 130, 140–155. doi:10.1037/00963445.130.1.140 McLean, L. D., Dauphinee, W. D., & Rothman, A. (1988). The oral examination in internal medicine of the Royal College of Physicians and Surgeons of Canada: A reliability analysis. Annals of the Royal College of Physicians and Surgeons of Canada, 21, 510–514. McNaught, C., Lam, P., & Ho, A. (2009). The digital divide between university students and teachers in Hong Kong. Proceedings ASCILITE 2009, Auckland, New Zealand. McPherson, M. C., & Nunes, J. M. (2008). Critical issues for e-learning delivery: What may seem obvious is not always put into practice. Journal of Computer Assisted Learning, 24(5), 433–445. doi:10.1111/j.13652729.2008.00281.x Miller, T., & Hutchens, S. (2009). 21st century teaching technology: Best practicess and effectiveness in teaching psychology. International Journal of Instructional Media, 36 3), 255-262. Minhsein, H. (2005). Integrating Internet into traditional education: A practical study.

McBride, R., & Schostak, J. (1994). What is qualitative research. Retrieved May 7, 2009, from http://www.enquireylearning.net/ELU/Issues/Research/Res1Ch1.html

349

Compilation of References

Ministerio de Educación Cultura y Deporte. (2003). REAL DECRETO 1125/2003, de 5 de septiembre, por el que se establece el sistema europeo de créditos y el sistema de calificaciones en las titulaciones universitarias de carácter oficial y validez en todo el territorio nacional. Boletín Oficial del Estado. Nº 224, 34355-34356. Retrieved November 6, 2010, from http://www.boe.es/boe/ dias/2003/09/18/pdfs/A34355-34356.pdf Minović, M., Štavljanin, V., Milovanović, M., & Starčević, D. (2008). Usability issues of e-learning systems: Casestudy for Moodle learning management system. In Meersman, R., Tari, Z., & Herrero, P. (Eds.), OTM 2008 Workshops, Lecture Notes in Computer Science 5333 (pp. 561–570). Heidelberg, Germany: Springer. Misko, J., Choi, J., & Lee, I. (2004). E-learning in Australia and Korea: Learning from practice. Korea Research Institute for Vocational Education & Training. Retrieved on June 19, 2010, from http://www.ncver.edu.au/research/ core/cp0306.pdf Monsakul, J. (2007). Learning management systems in higher education: A review from faculty perspective, 4th International Conference on E-learning for KnowledgeBased Society, Bangkok, Thailand. Moodle. (2010). Current Moodle statistics, Moodle.org. Retrieved on May 27, 2010, from http://moodle.org/stats. Moodle. (2010). Moodle Statistics. Retrieved August 25, 2010, from http://moodle.org/stats/ Moore, M. G. (1993). Theory of transactional distance. In Keegan, D. (Ed.), Theoretical principles of distance education (pp. 33–38). New York, NY: Routledge. Moreno, L., Iglesias, A. M., Castro, E., & Martinez, P. (2008). Using accessible digital resources for teaching database design: towards an inclusive distance learning proposal. ITICSE 2008, Madrid, Spain (pp. 32–36). New York, NY: ACM. Moreno, L., Martínez, P., & Ruiz-Mezcua, B. (2009). A bridge to Web accessibility from the usability heuristics, 5th Annual Usability Symposium USAB 2009. Usability & HCI eInclusion, LNCS 5889 (pp. 290-300). Berlin, Germany: Springer.

350

Morgan, G. (2003). Faculty use of course learning systems. Educause center for applied research reports. Retrieved December 29, 2010, from http://net.educause.edu/ir/ library/pdf/ecar_so/ers/ers0302/ekf0302.pdf Murray, G. R., Riley, C., & Scime, A. (2007, May). A new age solution for an age-old problem: Mining data for likely voters. Paper presented at the 62nd Annual Conference of the American Association of Public Opinion Research, Anaheim, CA. Nachmias, R., Mioduser, D., & Shemla, A. (2001). Information and Communication Technologies usage by students in an Israeli high school: Equity, gender, and inside/outside school learning issues. Education and Information Technologies, 6(1), 43–53. doi:10.1023/A:1011367212148 Naidu, S. (2006). E-learning a guidebook of principles, procedures and practices, 2nd revised edition, Commonwealth Educational Media Center for Asia (CEMCA), New Delhi, India. Naismith, L., Lonsdale, P., Vavoula, G., & Sharples, M. (2005). Literature review in mobile technologies and learning. NESTA Futurelab Series. National Centre for Educational Statistics. (2003). Distance education at degree-granting postsecondary institutions: 2000-2001 U. Washington, DC: S. Department of Education. Nevgi, A., Virtanen, P., & Niemi, H. (2006). Supporting students to develop collaborative learning skills in technology-based environments. British Journal of Educational Technology, 37(6), 937–947. doi:10.1111/j.14678535.2006.00671.x Newell, A. F., & Gregor, P. (2000). User sensitive inclusive design: In search of a new paradigm. CUU 2000 First ACM Conference on Universal Usability, 2000. Ng’ambi, D. (2004). Towards a knowledge sharing framework based on student questions: The case for a dynamic FAQ environment. PhD thesis. South Africa: University of Cape Town. Ngwenyama, O. K., & Lee, A. (1997). Communication richness in electronic mail: Critical social theory and the contextuality of meaning. Management Information Systems Quarterly, 21(2), 145–167. doi:10.2307/249417

Compilation of References

Nichani, M. (2009). LCMS = LMS + CMS [RLOs] – How does this affect the learner? The instructional designer? Retrieved on December 13, 2010, from http://www.elearningpost.com /articles/archives/ lcms_LMS_cms_rlos. Nichols, M. (2003). A theory for elearning. Journal of Educational Technology & Society, 6(2), 1–10. Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14–37. doi:10.1287/orsc.5.1.14 NYU. (2010). New York University. Retrieved from http:// www.nyu.edu/ Oliveira, P., Oliveira, C., Souza, F., & Costa, N. (2006). Teaching stategies to promote active learning in higher education: Current develepments in technology-assisted education, FORMATEX 2006, (pp. 636-640). Retrieved on September7, 2010, from http://www.formatex.org/ micte2006/pdf/636-640.pdf O’Neill, K., Singh, G., & O’ Donoghue, J. (2004). Implementing e-learning programmes for higher education: A review of the literature. Journal of Information Technology Education, 3. O’Neill, M. E. (2005). Automated use of a Wiki for collaborative lecture notes. SIGCSE Bull, 37(1). New York, NY: ACM. Ong, S. S., & Hawryszkiewycz, I. (2003). Towards personalised and collaborative learning management systems. Third IEEE International Conference on Advanced Learning Technologies (ICALT’03). Retrieved December 29, 2010, from http://www.computer.org/portal/web/csdl/ doi/10.1109/ICALT.2003.1215113 O’Reilly, T. (2005). What is Web 2.0: Design patterns and business models for the next generation of software. Retrieved from http://www.oreillynet.com/pub/a/oreilly/ tim/news/2005/09/30/what-is-web-20.html (Dec. 2010). Osei-Bryson, K.-M. (2004). Evaluation of decision trees: A multicriteria approach. Computers & Operations Research, 31(11), 1933–1945. doi:10.1016/S03050548(03)00156-4

Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating learning management systems: Adoption of hexagonal e-learning assessment model in higher education. Transforming Government: People, Process and Policy, 3(2), 11–130. Padmanabhan, B., & Tuzhilin, A. (2000). Small is beautiful: Discovering the minimal set of unexpected patterns. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data mining, Boston, MA (pp. 54-63). Pan, G., & Bonk, C. (2007). The emergence of opensource software in North America. International Review of Research in Open and Distance Learning, 8(3), 1–17. Pantel, C. (1997). A framework for comparing Webbased learning environments. Master’s thesis, School of Computing Science, Simon Fraser University, Canada. Parasuraman, A., Zeithaml, V. A., & Berry, L. (1988). SERVQUAL: A multiple-item scale for measuring customer perceptions of service quality. Journal of Retailing, 64(1), 12–40. Park, H., Baek, Y. K., & Gibson, D. (2008). Design of an adaptive mobile learning management system. User interface design and evaluation for mobile technology, I, 286-301. Patriotta, G. (2003). Organizational knowledge in the making: How firms create, use, and institutionalize knowledge. Oxford, UK: Oxford University Press. Paulsen, M. (2002). Online education sytems in Scandinavian and Australian universities_A comparative study. International Review of Research in Open and Distance Learning, 3(2), 1–14. Paulsen, M. F. (2003). Online education and learning management systems. Bekkestua, Norway: NKI. Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4), 134–148. Paulsen, M. F. (2002). Online education systems: Discussion and definition of terms. Retrieved December 29, 2010, from http://www.porto.ucp.pt/open/curso/modulos/ doc/Definition%20of%20Terms.pdf

351

Compilation of References

Perry, B. (2009, June). Customized content at your fingertips. Training & Development, 29–31.

Polanyi, M. (1958). Personal Knowledge. Chicago, IL: The University of Chicago press.

Pettit, J., & Kukulska-Hulme, A. (2007). Going with the grain: Mobile devices in practice. Australasia Journal of Educational Technology, 23(1), 17–33.

Power, C., Petrie, H., Sakharov, V., & Swallow, D. (2010). K. virtual learning environments: Another barrier to blended and e-learning. Miesenberger et al. (Eds.), ICCHP 2010, Part I, LNCS 6179 (pp. 519–526).

Phan Tan, T.-T., & Krüger, M. (2009). Hochschulübergreifende weiterbildung zum e-learning in niedersachsen. In Apostolopoulos, N. (Ed.), Grundfragen multimedialen lehrens und lernens. Bildungsimpulse und bildungsnetzwerke: Tagungsband GML 2 2009, 12. - 13. März 2009 (pp. 69–80). Berlin, Germany: Univ.-Verl. der TU Univ.-Bibliothek. Phan Tan, T.-T., & Prey, G. (2010). Programm zur didaktischen e-kompetenzentwicklung in Niedersachsen. Zeitschrift für hochschulentwicklung, 5(1). Retrieved from http://www.zfhe.at Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence and performance in an online course. Journal of Asynchronous Learning Networks, 6(1). Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skill training. Management Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989 Pietruszkiewicz, W., & Dżega, D. (2009). The practical aspects of rich Internet application development and quality factors: RIA–based decision support system. Quality assessment in Web 2009 (Vol. 561). CEUR. Pietruszkiewicz, W., & Dżega, D. (2010). An application of data mining in the management of e-learning platform. In Korczak, J. (Ed.), Business informatics. Data mining and business intelligence (pp. 60–70). Wrocław, PA: Publishing House of Wrocław University of Economics. Pituch, K. A., & Lee, Y. K. (2006). The influence of system characteristics on eLearning use. Computers & Education, 47(2), 222–244. doi:10.1016/j.compedu.2004.10.007 Pituch, K., & Lee, Y. (2006). The influence of system characteristics on e-learning use. Computers & Education, 47, 222–244. doi:10.1016/j.compedu.2004.10.007 Polanyi, M. (1966). The tacit dimension. London, UK: Routledge & Kegan Paul.

352

Prensky, M. (2007). Changing paradigms – from “being taught” to “learning on your own with guidance”. Educational Technology. July-August. Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon - The Strategic Planning Resource for Education Professionals, 9(5). Pressley, M., Ghatala, E. S., Woloshyn, V., & Pirie, J. (1990). Sometimes adults miss the main ideas and do not realize it: Confidence in responses to short answer and multiple-choice comprehension questions. Reading Research Quarterly, 25, 232–249. doi:10.2307/748004 Puddu, F., Repetto, M., & Vallarino, E. (2008). The WEL initiative: A model of Web enhanced learning for faculty members. Journal of E-learning and Knowledge Society, 4(3). Queirós, R., & Leal, J. P. (2010). E-learning frameworks: A survey. In INTED2010 Proceedings (pp. 1345-1354). Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco, CA: Morgan Kaufmann. Raaij, E., & Schepers, J. (2008). The acceptance and use of a virtual learning environment in China. Computers & Education, 50, 838–852. doi:10.1016/j.compedu.2006.09.001 Rabin, J., & McCathieNevile, C. (Eds.). (2006). Mobile Web best practices 1.0. (W3C Proposed Recommendation). Retrieved on August 25, 2010, from http://www. w3.org/TR/mobile-bp/ Race, P. (2005). Making Learning Happen. London, UK: Sage. Rainer, R. K., Turban, E., & Potter, R. E. (2007). Introduction to Information Systems: Supporting and transforming business. NJ: Wiley.

Compilation of References

Rajasethupathy, K., Scime, A., Rajasethupathy, K. S., & Murray, G. R. (2009). Finding “persistent Rules”: Combining association and classification results’, Expert Systems With Applications, 36(3P2), 6019-6024. Rakotonirainy, A., Loke, S. W., & Fitzpatrick, G. (2000). Context awareness for the mobile environment. Retrieved Oct 11, 2009 from ftp://ftp.cc.gatech.edu/pub/gvu/ tr/2000/00-18r.ps.Z. Regueras, L. M., Verdú, E., Castro, J. P., Pérez, M. A., & Verdú, M. J. (2008). Design of a distributed and asynchronous system for remote evaluation of students’ submissions in competitive e-learning. In International Conference on Engineering Education (ICEE ‘08). Rehak, D., & Mason, R. (2003). Engaging with the learning object economy. In Littlejohn, A. (Ed.), Reusing online resources: A sustainable approach to e-learning (pp. 22–30). London, UK: Kogan Page. Rehman, H. U., Said, R. A., & Al-assaf, Y. (2009). An integrated approach for strategic development of engineering curricula: Focus on students’ design skills. IEEE Transactions on Education, 52(4), 470–481. doi:10.1109/ TE.2008.930508 Reiser, R. A., & Dempsey, J. V. (Eds.). (2002). Trends and issues in instructional design and technology. Upper Saddle River, NJ: Pearson Education. Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, J. (2008). Evaluating learning support systems usability an empiracle approach. Communication & Cognition, 41(1 & 2), 143–158. Ribaudo, M., & Rui, M. (2009). AulaWeb, Web-based learning as a commodity - The experience of the University of Genova. 1st International Conference on Computer Supported Education (CSEDU 2009). INSTICC Press. Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1), 68–88. Richardson, H. C., & Ward, R. (2005). Developing and implementing a methodology for reviewing e-portfolio products, The centre for recording achievement (CRA). Retrieved October 10, 2009, from http://www.jisc.ac.uk/ uploaded_documents/epfr.doc

Roca, J., Chiu, C., & Martinex, F. (2006). Understanding e-learning continuous intention: An extension of the technology acceptance model. International Journal of Human-Computer Studies, 64, 683–696. doi:10.1016/j. ijhcs.2006.01.003 Rodrigues, A., Pinto, M., & Queirós, R. (2010). Grau de Preparação e Utilização do E-learning no Ensino Superior. Estudo de Caso na ESEIG-IPPorto. Setúbal, Portugal: Conferência Investigação e Intervenção em Recursos Humanos. Rogers, L., & Newton, L. (2001). Integrated Learning Systems -- An ‘open’ approach. International Journal of Science Education, 23(4), 405–422. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Romero, C., Ventura, S., & Garcia, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51, 368–384. doi:10.1016/j.compedu.2007.05.016 Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j.compedu.2007.05.016 Roqueta, M. (2008). Learning management systems: A focus on the learner. Distance Learning, 5(4), 59–66. Rose, E. (2004). “Is there a class with this content?” WebCT and the limits of individualization. The Journal of Educational Thought, 38(1), 43–65. Rossi, P. G. (2009). Learning environment with elements of artificial intelligence. Journal of E-learning and Knowledge Society - English Version, 5(1), 193-199. Rößling, G., Joy, M., Moreno, A., Radenski, A., Malmi, L., Kerren, A., & Iturbide, J. Á. (2008). Enhancing learning management systems to better support computer science education (pp. 142-166). Paper presented at the SIGCSE 2008, 4, Nov. 2008. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2).

353

Compilation of References

Rovai, A. P. (2008). Facilitating online discussions effectively. The Internet and Higher Education, 10, 77–88. doi:10.1016/j.iheduc.2006.10.001 Ryder, M., & Wilson, B. (1997). From center to periphery: Shifting agency in a complex technical learning environments. Paper presented at the Meeting of the American Educational Research Association, Chicago, IL. Saady, A. (2005). E-learning curve, ITP Technology, Retrieved from www.ITP.net. Saba, F. (2000). Research in distance education: A status report. International Review of Research in Open and Distance Learning, 1(1), 1–9. Sabry, K., & Barker, J. (2009). Dynamic Interactive Learning Systems. Innovations in Education and Teaching International, 46(2), 185–197. doi:10.1080/14703290902843836 Salas-Morera, L., Berral- Yerón, J., Serrano-Gómez, I., & Martínez-Jiménez, P. (2009). An assessment of the ECTS in software engineering: A teaching experience. IEEE Transactions on Education, 52(1), 177–184. doi:10.1109/ TE.2008.921801

Savander-Ranne, C., Lundén, O. P., & Kolari, S. (2008). An alternative teaching method for electrical engineering courses. IEEE Transactions on Education, 51(4), 423–431. doi:10.1109/TE.2007.912500 Scharf, E. M., & Baldwin, L. P. (2007). Assessing multiple choice question (MCQ) tests - A mathematical perspective. Active Learning in Higher Education, 8, 31–47. doi:10.1177/1469787407074009 Schein, E. (1985). Organizational culture and leadership. San Francisco, CA: Jossey-Bass. Schmidt, A., Beigl, M., & Gellersen, H. W. (1999). There is more to context than location. Computers & Graphics, 23, 893–901. doi:10.1016/S0097-8493(99)00120-X Scime, A., Murray, G. R., & Hunter, L. Y. (2010). Testing terrorism theory with data mining. International Journal of Data Analysis Techniques and Strategies, 2(2), 122–139. doi:10.1504/IJDATS.2010.032453 Scime, A. (2008). Web page extension of data warehouses. In Wang, J. (Ed.), Encyclopedia of Data Warehousing and Mining (2nd ed., pp. 1211–1215). Hershey, PA: Idea Group Reference.

Samarawickrema, G., & Stacey, E. (2007). Adopting Web-based learning and teaching: A case study in higher education. Distance Education, 28(3). doi:10.1080/01587910701611344

Scime, A. & Murray, G. R. (2007). Vote prediction by iterative domain knowledge and attribute elimination. International Journal of Business Intelligence and Data mining, 2(2), 160-176.

Sampson, D., Papapaulou, P., Zervas, P., & Papanikou, C. (2008). Supporting higher education teaching through wireless and mobile devices: The case study of mobile Moodle. In Proc. of the AACE E-Learn 2008-World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education.

Sclater, N. (2008). Large-scale open source e-learning systems at open university UK. Research Bulletin EDUCAUSE Center for Applied Research, 2008(12), 1-13.

Santos, O. C., Boticario, J. G., Del Viso, A. F., De La Cámara, S. P., Sánchez, C. R., Gutiérrez & Restrepo, E. (2007). Basic skills training to disabled and adult learners through an accessible e-learning platform. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (LNCS 4556, part 3, pp. 796-805). Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students (4th ed.). Pearson Education Ltd.

354

SCORM. (2004). 4th Ed. Specification. Retrieved on November 12, 2010, from http://www.adlnet.gov/Pages/ Default.aspx. Scribner, S. (1987). Thinking in action: Some characteristics of practical thought. In Sternberg, R., & Wagner, R. (Eds.), Practical intelligence: Nature and origins of competence in everyday world. Cambridge, UK: Cambridge University Press. Seddon, P. B. (1997). A respecification and extension of the DeLone and McLean model of IS success. Information Systems Research, 8(3), 240–253. doi:10.1287/ isre.8.3.240

Compilation of References

Sen, A., Dacin, P. A., & Pattichis, C. (2006). Current trends in Web data analysis. Communications of the ACM, 49(11), 85–91. doi:10.1145/1167838.1167842 Seufert, S. (2008). Innovationsorientiertes bildungsmanagement: Hochschulentwicklung durch sicherung der nachhaltigkeit von e-learning. Wiesbaden, Germany: VS Verlag für Sozialwissenschaften / GWV Fachverlage GmbH Wiesbaden. SHAME. (2006). SHAME - Standardized Hyper Adaptable Metadata Editor. Retrieved on December 13, 2010, from http://kmr.nada.kth.se/shame. Shariq, Z. S. (1999). How does knowledge transform as it is transferred? Speculations on the possibility of a cognitive theory of knowledgescapes. Journal of Knowledge Management, 3(4), 243–251. doi:10.1108/13673279910303998 Sharples, M. (Ed.). (2006). Big issues in mobile learning. Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative. Nottingham, UK: University of Nottingham. Shea, P. J., Pickett, A. M., & Pelz, W. E. (2003). A followup investigation of “teaching presence” in the SUNY Learning Network. Journal of Asynchronous Learning Networks, 7(2), 61–80. Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. Toronto, Canada: Wiley. Simões, L. & Gouveia, L. (2009). Geração Net, Web 2.0 e Ensino Superior. Cadernos de Estudos Mediáticos, 21-32. Simonson, M., Maurer, R., Montag-Torardi, M., & Whitaker, M. (1987). Development of a standardized test of computer literacy and a computer anxiety index. Journal of Educational Computing Research, 3(2), 231–247. doi:10.2190/7CHY-5CM0-4D00-6JCG Sink, C., Sink, M., Stob, J., & Taniguchi, K. (2008). Further evidence of gender differences in high school–Level computer literacy. Chance, 21(1), 49–53. doi:10.1007/ s00144-008-0011-9 Smythe, C. (2003). IMS abstract framework - A review. IMS Global Learning Consortium, Inc.

So, H. J. (2009). When groups decide to use asynchronous online discussions: Collaborative learning and social presence under a voluntary participation structure. Journal of Computer Assisted Learning, 25, 143–160. doi:10.1111/j.1365-2729.2008.00293.x Stergiopoulos, C., Tsiakas, P., Kaitsa, M., Triantis, D., Fragoulis, I., & Ninos, C. (2006b). Methods of electronic examination applied to students of electronics. Comparison of results with the conventional (paper-andpencil) method. In J. A. Moinhos Cordeiro, V. Pedrosa, B. Encarnação, & J. Filipe (Eds.), Proceedings of the 2nd International Conference on Web Information Systems and Technologies: Internet Technology / Web Interface and Applications(WEBIST 2006) (pp. 305-311). Setubal, Portugal: INSTICC Press Stergiopoulos, C., Tsiakas, P., Triantis, D., & Kaitsa, M. (2006a). Evaluating electronic examination methods of students of electronics. Effectiveness and comparison to the paper-and-pencil Method. In Proceedinsg of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC 2006) (Vol. 2, pp. 143-151). Los Alamitos, CA: IEEE Computer Society. Stifterverband für die Deutsche Wissenschaft. (n. d.). Exzellenz in der Lehre. Aktuelle Programme und Initiativen. Rertieved December 20, 2010, from http://www. stifterverband.de/ Stifterverband für die Deutsche Wissenschaft. (2010). Lehre neu denken. Die Zukunft des akademischen Lehren und Lernens. Retrieved December 20, 2010, from http:// www.stifterverband.info/publikationen_und_podcasts/ positionen_dokumentationen/lehre_neu_denken/lehre_ neu_denken.pdf Strivens, J. (2007). A survey of e-pdp and ePortfolio practice in UK higher education. UK: Higher Education Academy. Sturgess, P., & Nouwens, F. (2004). “Evaluation of online learning management systems. Turkish Online Journal of Distance Education, 5(3). Sumner, M., & Hostetler, D. (1999). Factors influencing the adoption of technology in teaching. Journal of Computer Information Systems, 40(1), 81.

355

Compilation of References

Sun, P., Tsai, R., Finger, G., Chen, Y., & Yeh, D. (2008). What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50, 1183–1202. doi:10.1016/j.compedu.2006.11.007 Sutherland, R., Facer, K., Furlong, R., & Furlong, J. (2000). A new environment for education? The computer in the home. Computers & Education, 34(3-4), 195–212. doi:10.1016/S0360-1315(99)00045-7 Swan, K. (2002). Building communities in online courses: The importance of interaction. Education Communication and Information, 2(1), 23–49. doi:10.1080/1463631022000005016 Swan, K. (2003). Developing social presence in online discussions. In Naidu, S. (Ed.), Learning and teaching with technology: Principles and practices (pp. 147–164). London, UK: Kogan Page. Swedish National Agency for Higher Education. (2008). E-learning quality: Aspects and criteria for evaluation of e-learning in higher education. Report 2008:11 R, PA: Swedish National Agency for Higher Education, Stockholm. Retrieved on July 30, 2010, from http:// www.hsv.se/download/18.8f0e4c9119e2b4a60c800028 057/0811R.pdf Tan, P.-N., Steinbach, M., & Kumar, V. (2006). Introduction to data mining. Boston, MA: Addison Wesley. Tapscott, D. (1999). Growing up digital: the rise of the next generation. New York, NY: McGraw-Hill. Tastle, J., White, A., & Shackleton, P. (2005). E-learning in higher education: The challenge, effort, and return of investment. International Journal on E-Learning, 241–251. Taylor, J. (2001). 5th generation distance education. 20th ICDE World Conference on Open Learning and Distance Education (pp. 1-11). Dusseldorf, Germany. Teo, T. (2009). Modelling technology acceptance in education: A study of pre-service teachers. Computers & Education, 52, 302–312. doi:10.1016/j.compedu.2008.08.006 Tham, C. M., & Werner, J. M. (2005). Designing and evaluating e-learning in higher education: A review and recommendations. Journal of Leadership & Organizational Studies, 11(2), 15–26. doi:10.1177/107179190501100203

356

Thomas, K., Lam, P., & Ho, A. (2009). Knowledge diffusion in eLearning: Learner attributes and capabilities in an organization. In G. Siemens & C. Fulford, (Eds.), ED-MEDIA 2009 (pp.493–497). Proceedings of the 21st Annual World Conference on Educational Multimedia, Hypermedia and Telecommunications, Honolulu, Hawaii. (pp. 22–26). Chesapeake, VA: Association for the Advancement of Computers in Education. Thomas, P. G. (2003). Evaluation of electronic marking of examinations. In D. Finkel, (Ed.), Proceedings of the 8th ACM Annual Conference on Innovation and Technology in Computer Science Education (ItiCSE 2003) (pp. 50-54). New York, NY: ACM. Thompson, R., & Compeau, R., Deborah, & Higgins, C. (2006). Intentions to use Information Technologies: An integrative model. Journal of Organizational and End User Computing, 18(3), 25–47. doi:10.4018/joeuc.2006070102 Trentin, G. (2001). Designing Online Courses. The Web in Higher Education: Assessing the Impact and Fulfilling the Potential. The Haworth Press. Triantis, D., Anastasiadis, C., Tsiakas, P., & Stergiopoulos, C. (2007). Asyncronous teaching methods and electronic assessment applied to students of the module “Physics of semiconductor devices”. Journal of Materials Education, 29, 133–140. Triantis, D., Stavrakas, I., Tsiakas, P., Stergiopoulos, C., & Ninos, D. (2004). A pilot application of electronic examination applied to students of electronic engineering: Preliminary results. WSEAS transactions on advances in engineering education, 1, 26-30. Tsiakas, P., Stergiopoulos, C., Nafpaktitis, D., Triantis, D., & Stavrakas, I. (2007). Computer as a tool in teaching, examining and assessing electronic engineering students. In Proceedings of the International Conference of Computer as a Tool (IEEE-EUROCON 2007) (pp. 2490-2497). Tuparova, D., & Tuparov, G. (2010). Management of students’ participation in e-learning collaborative activities. Procedia Social and Behavioral Sciences, 2, 4757–4762. doi:10.1016/j.sbspro.2010.03.764 Ullrich, C., Borau, K., Luo, H., Tan, X., Shen, L., & Shen, R. (2008). Why Web 2.0 is good for learning and for research: Principles and prototypes. 17th International World Wide Web Conference (WWW 2008).

Compilation of References

Underwood, J. (1997). Integrated learning systems: Where does the management take place? Education and Information Technologies, 2, 275–286. doi:10.1023/A:1018677616969

Wagner, N., Hassanein, K., & Head, M. (2008). Who is responsible for e-learning success in higher education? A stakeholder’s analysis. Journal of Educational Technology & Society, 11(3), 26–36.

Universität Duisburg-Essen (u. d.). Zentrum für Hochschul- und Qualitätsentwicklung: Kompetenzentwicklung in Studium & Lehre. Retrieved August 30, 2010, from http://zfh.uni-duisburg-essen.de/elearning

Wainer, H., Wang, X.-B., & Thissen, D. (1994). How well can we equate test forms that are constructed by examinees? Journal of Educational Measurement, 31, 183–199. doi:10.1111/j.1745-3984.1994.tb00442.x

Universität Trier (u. d.). Koordinationsstelle E-Learning. Retrieved August 30, 2010, from http://www.uni-trier.de/ index.php?id=33362

Wainer, H., & Thissen, D. (1993). Combining multiplechoice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education, 6, 103-1 18.

Uzunboylu, H., Ozdamli, F., & Ozcinar, Z. (2006). An evaluation of open source learning management systems according to learners tools. Online Submission. Retrieved December 28, 2010, from http://www. eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ ERICServlet?accno=ED494265 Van Beveren, J. (2002). A model of knowledge acquisition that refocuses knowledge management. Journal of Knowledge Management, 6(1), 18–22. doi:10.1108/13673270210417655 Van der Linden, W. J., & Glas, C. A. W. (2000). Computerized adaptive testing: Theory and practice. Heidelberg, Germany: Springer. Venkatesh, V., & Davis, F. (2000). A theoretical extension of the technology acceptance model: Four longitudinal. Management Science, 46(2), 186–204. doi:10.1287/ mnsc.46.2.186.11926 Ventouras, E., Triantis, D., Tsiakas, P., & Stergiopoulos, C. (2010). Comparison of examination methods based on multiple-choice questions and constructed-response questions using personal computers. Computers & Education, 54, 455–461. doi:10.1016/j.compedu.2009.08.028 Vivek, S., & Ansari, Y. (2010). Collaborative learning issues in synchronous online interactions. International Conference on Technology for Education (T4E), 5-8. Volery, T., & Lord, D. (2000). Critical success factors in online education. International Journal of Educational Management, 14(5). doi:10.1108/09513540010344731

Walker, J. & Jorn, L. (2007) Net generation students at the University of Minnesota, Twin Cities Student Educational Technology Survey 2007. Walther, J. (1994). Interpersonal effects in computer mediated interaction. Communication Research, 21(4), 460–487. doi:10.1177/009365094021004002 Wan, Z., Fang, Y., & Neufeld, H. (2007). The role of Information Technology in technology-mediated learning: A review of the past for the future. Journal of Information Systems Education, 18(2), 183–192. Wang, W., & Wang, C. (2009). An empirical study of instructor adoption of Web-based learning systems. Computers & Education, 53, 761–774. doi:10.1016/j. compedu.2009.02.021 Wang, Y., & Chen, N. (2009). Criteria for evaluation synchonous learning management systems: Arguments from the distance language classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773 Wass, V., Wakeford, R., Neighbour, R., & Van der Vleuten, C. (2003). Achieving acceptable reliability in oral examinations: An analysis of the Royal College of General Practitioners membership examination’s oral component. Medical Education, 37, 126–131. doi:10.1046/j.13652923.2003.01417.x Watson, W., & Watson, S. (2007). What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y

357

Compilation of References

Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems. What are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-007-0023-y Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40(6), 1282. doi:10.2307/257034 Weller, M. (2007). Virtual learning environments: Using, choosing and developing your VLE. UK: Routledge. Wenger, E. C. (2000). Communities of practice and social learning systems. Organization, 7(2), 225–246. doi:10.1177/135050840072002 Wenger, E. C., & Snyder, W. M. (2000). Communities of practice: The organizational frontier. Harvard Business Review, 78(1), 139–145. Whisker, G. (2001). The Postgraduate Research Handbook. Basingstoke, UK: Palgrave MacMillan. Whitelock, D. (2009). Editorial: E-assessment: Developing new dialogues for the digital age. British Journal of Educational Technology, 40(2), 199–202. doi:10.1111/ j.1467-8535.2008.00932.x Wiener, M., & Mehrabian, A. (1968). Language within language: Immediacy, a channel in verbal communication. New York, NY: Appletoncentury-Croft. Williams, J., & Goldberg, M. (2005). The evolution of e-learning. [Global.]. Universitas, 21. Wilson, S., Blinco, K., & Rehak, D. (2004). An e-learning framework - Paper prepared on behalf of DEST (Australia), JISC-CETIS (UK), and Industry Canada. Winde, M. (Ed.). (2010). Von der Qualitätsmessung zum Qualitätsmanagement. Praxisbeispiele an Hochschulen. Essen, Germany: Edition Stifterverband. Winograd, T., & Flores, F. (1986). Understanding computers and cognition: A new foundation for design. Norwood, NJ: Ablex. Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques (2nd ed.). San Francisco, CA: Morgan Kaufman.

358

Wood, R. (1991). Assessment and testing: A survey of research. Cambridge, UK: Cambridge University Press. Woods, R., Baker, J., & Hopper, D. (2004). Hybrid structure: Faculty use and perception of Web-based courseware as a supplement to face-to-face instruction. The Internet and Higher Education, 7(4), 281–297. doi:10.1016/j. iheduc.2004.09.002 World Bank. (2010). Learning management system. World Bank Institute. Retrieved on April 7th, 2010, from http:// web.worldbank.org World Wide Web Consortium. (W3C) (2008). Web accessibility initiative (WAI), Web Content Accessibility Guidelines (WCAG) 2.0. Available at http://www.w3.org/ TR/WCAG20 / World Wide Web Consortium. (W3C) (2010). Web Accessibility Initiative (WAI), Authoring Tool Accessibility Guidelines (ATAG) 2.0. Available at http://www.w3.org/ TR/ATAG20/ World Wide Web Consortium. (2010). WAI-ARIA Overview. Available at http://www.w3.org/WAI/intro/aria World Wide Web Consortium. (W3C). (2010 b). Involving users in evaluating Web accessibility. Available at http:// www.w3.org/WAI/eval/users.html World Wide Web Consortium. (W3C). (2010 c). WAI, evaluating Web Sites for accessibility: Overview. Available at http://www.w3.org/WAI/eval/ Yau, J., Lam, J., & Cheung, K. S. (2009). A review of e-learning platforms in the age of e-learning 2.0. 2nd International Conference on Hybrid Learning and Education (ICHL 2009), LNCS 5685 (pp. 208-217). Yildirim, S., Temur, N., Kocaman, A., & Goktas, Y. (2004). What makes a good LMS: An analytical approach to assessment of LMS. In Proceedings of the 5th International Conference on Information Technology Based Higher Education and Training, Istanbul, Turkey (pp. 125–130). Yueh, H.-P., & Hsu, S. (2008). Designing a learning management system to support instruction. Communications of the ACM, 51(4). doi:10.1145/1330311.1330324

Compilation of References

Zhang, D., Zhao, J., Zhou, L., & Numamaker, J. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75–78. doi:10.1145/986213.986216 Zhang, H., Almeroth, K., Knight, A., Bulger, M., & Mayer, R. (2007). Moodog: Tracking students’ online learning activities. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 4415–4422). Chesapeake, VA: AACE.

Zhao, J., & Yiang, Y. (2008). A case study on collaborative e-tutoring. Proceedings of 2008 IEEE International Symposium on IT in Medicine and Education (Article number 4743903, pp. 439-444). Zhou, Z. (2004). From classrooms to the real engineering world: The training program in the microelectronics research center at Georgia Tech. IEEE Transactions on Education, 47, 188–120. doi:10.1109/TE.2003.822262

359

360

About the Contributors

Rosalina Babo is a Coordinator Professor at the Information Systems Department at School of Accounting and Administration of Porto / Polytechnic Institute of Porto (ISCAP/IPP), Portugal. Since the year 2000 she is the head of the Information Systems Department and a member of the university scientific board. She was one of the founders (2006) of CEISE/STI research center and its director until the year 2011. Her primary area of research is eBusiness as well as Usability Evaluation. Under these subjects cross country studies regarding cultural differences is a challenge for her. Also eLearning is one of her strong interests. Ana Azevedo is a lecturer in Information Systems and Technologies at the School of Accounting and Administration of Porto / Polytechnic Institute of Porto (ISCAP/IPP), Portugal. She is a PhD Student in Information Systems and Technologies at the department of information systems, University of Minho, Guimarães, Portugal. She is a researcher of Algoritmi Research Center and of CEISE/STI research center. She was one of the founders (2006) of CEISE/STI research center and its vice-director until the year 2011. Her primary areas of research interests are Decision Support Systems, Data mining, and Business Intelligence. Her research interest also includes e-Learning. She is a member of the Editorial Review Board of the International Journal of Technology and Human Interaction (IJTHI) and a member of the editorial board of the International Journal of Data Mining and Emerging Technologies. *** Kamla Ali Al-Busaidi is an assistant professor of information systems at Sultan Qaboos University in Oman. She received her PhD in management information systems from Claremont Graduate University in California in 2005, MSc in information systems management from Duquesne University in Pennsylvania in 1999, and BSc in information systems from Sultan Qaboos University in Oman in 1997. Her research interests include knowledge management systems, decision support systems, learning management systems and the deployment of information and communication technologies in Arab countries. She has published several conference proceedings and journals, including International Journal of Knowledge Management, Journal of Global Information Technology Management, International Journal of Global Management Studies and Communications of IBIMA. She also served as a reviewer for several international and local journals, and conference proceedings including AMCIS, ICIS, HICSS, IRMA, IEEE & IBIMA. She has been an organizing member for several local and international workshops and conference proceedings.

About the Contributors

Hafedh Al-Shihi is an Assistant Professor at the College of Commerce and Economics in Sultan Qaboos University. He is a member of the Association of Information Systems (AIS) and the Special Interest Group on Electronic Government (SIGeGov). He is also a Research Associate in the Center for International Corporate Governance Research at Victoria University, Australia. He has several publications on e-government / m-government, e-learning and technology adoption and dissemination and has attended several local and international conferences. He has been asked to be a keynote speaker in local and international conferences. Latest was in the “SOIE & FIESTA Conference in Tunis 2009” organized by the Tunisian Association for Artificial Intelligence. He acted in the reviewing committee in several international journals and conferences. Latest was the International Journal of Electronic Governance. Currently, he is teaching introductory and major Information Systems (IS) courses to undergraduate students and supervising the Information Systems Group of IS majored students (www.isg-zone.com). He also publishes a weekly article in Oman Newspaper related to IT. Antonio Arauzo-Azofra received MSc in Computer Science in 2001 and PhD in the field of Artificial Intelligence in 2006 from the University of Granada (Spain). He is currently Assistant Professor of Project Engineering in the Higher Technical College at the University of Córdoba (Spain). His primary areas of research are feature selection applied in classification problems for intelligent data mining in information systems and the development of intelligent user adaptive systems for automated facility layout design. Specialized in project management where he has been lecturing for 6 years, he is in active collaboration on several projects for the improvement of teaching methodologies adapted to the European Higher Education Area. He received the University of Córdoba Social Council award of Educational Innovation in 2008. Gavin Baxter is a Research Assistant in the School of Computing at the University of the West of Scotland. His research focuses on the implementation of Web 2.0 technologies in organisations for the purposes of organisational learning. His other research areas include: the applicability of blogs as communication and knowledge sharing tools in organisational project-based environments. He has also published works about organisational blogging in the journals: The Learning Organization and IJCEnt. He also researches about the application of Web 2.0 technologies in educational contexts and is presently involved on the ‘Web 2.0 European Resource Centre’ project. Rocío Calvo, I graduated in Computer Science from Carlos III University of Madrid in the 2008 and studied as a postgraduate in Computer Science and Technology at the same University in 2010. Nowadays, I am working for the Computer Science department of Carlos III University as a researcher. (MoreInformation: http://labda.inf.uc3m.es/doku.php?id=en:labda_personal:personal_rcalvo) Maura Cerioli has been an Associate Professor in Computer Science at the University of Genova since 2000. Her initial research interests were related to the study of formal methods for program specification. More recently she is working in the area of Software Engineering, and is currently undertaking studies on teaching strategies. She has taught in the years a number of subjects, including programming languages, software engineering, advanced programming techniques, data bases. She is the chair of the ICT certification program for the University of Genova and member of the committee designing the new graduate and undergraduate curricula for Computer Science in her Faculty.

361

About the Contributors

Thomas Connolly is Chair of the ICT in Education Research Group at the University of the West of Scotland and is Director of the Scottish Centre for Enabling Technologies and Director for the Centre of Excellence in Games-based Learning. His specialisms are online learning, games-based learning and database systems. He has published papers in a number of international journals as well as authoring the highly acclaimed books ‘Database Systems: A Practical Approach to Design, Implementation, and Management’, ‘Database Solutions’ and Business Database Systems, all published by Addison Wesley Longman. Professor Connolly also serves on the editorial boards of many international journals, as well as managing several large-scale externally funded research projects. Antonio J. Cubero-Atienza was born in Córdoba, Spain, in 1961. He received the BS degree in Agricultural Engineering from the University of Córdoba, Spain, in 1984, and the PhD in Agricultural Engineering from University of Córdoba, Spain, in 1988. He is a Professor of University of Córdoba, in the Department of Rural Engineering, Area of Projects Engineering. His teaching areas include Project Engineering, Safety and Health at work, between others. Currently, he is Executive Manager of Córdoba University from June 2010. His primary research areas include noise and vibration in industrial environments, environmental noise, identification models of sound, educational technology and sound quality in biofuels. Sandra Delgado Marqueta graduated top in Computer Science from the Carlos III University of Madrid in 2009 and studied a postgraduate degree in Computer Science and Technology at the same university in 2010. Currently, She is working in R+D Department in the IT Company working in accessible Web design. Dorota Dżega received her PhD degree in Computer Science from Szczecin University of Technology (currently West Pomeranian University of Technology) in Szczecin, Poland. She is Assistant Professor at Faculty of Economics and Computer Science, West Pomeranian Business School (Szczecin, Poland) and Vice Dean of e-Learning. Her research and educational activities relate to the project management (especially software projects), data analysis and computer systems in management. She has an experience in the realization of research projects and development projects financed by Polish Ministry of Science and European Union. She is a member of Association of Academic E-learning and The Polish Information Processing Society. Bob Folden has an EdD in Developmental Education, Instructional Systems and Design. Dr. Folden has been involved in various areas of distance education. Having taught over ITFS, managed a correspondence course program, worked with satellite course delivery and computer managed instruction, he has a personal knowledge of distance education. He led the development of integrated technologies for distance education which integrated satellite delivery of instruction with computerized simulation to train technicians at a distance. His courses at Texas A & M University-Commerce cover business administration and various aspects of management information systems. Most of his courses are taught at a distance using web based technologies. He also consults with a number of external entities to develop and deliver instruction at a distance.

362

About the Contributors

Laura García-Hernández was born in Spain on August 14, 1984. She received MSc in Computer Science in 2007 from the Oberta University of Catalunya (Spain). She works currently as assistant professor of Projects Engineering Area in the Department of Rural Engineering of the Higher Technical College at the University of Córdoba (Spain). Her research interests are in the fields of Facility Layout Design, Interactive Algorithms, Evolutionary Computation, Project Management and Educational Technology. Since 2007, she is in active collaboration on projects for the adaptation of teaching methodologies to the European Space for Higher Education and, in particular, to the ECTS. Carole Gould is a PhD student in the school of Computing at the University of the West of Scotland researching Web 2.0 technologies in Learning and Teaching. She is specifically working towards the design of an educational framework that takes account of the growing use of Web 2.0 technologies within both educational and vocational environments. She has a number of publications in this area. Thomas Hainey is a researcher in the School of Computing at the University of the West of Scotland specialising in games-based learning and particularly evaluation of games-based learning. He has a number of journal and conference publications in this area. Ana Iglesias is a member of the faculty of the Computer Science Department of Carlos III University of Madrid, since February 2006. She obtained a degree in Computer Science from Carlos III University of Madrid in 1999, and her PhD in Computer Science from Carlos III University of Madrid (UC3M) in 2004. Since 2000, I worked at the Advanced Databases Group in the Computer Science Department at Carlos III University of Madrid. I am also a member of the Spanish Centre of Caption and Audiodescription (CESyA). I have been working in several National research projects on Accessibility, Advanced Database Technologies, CASE Environments, Natural Language Processing and Information Retrieval. My research interests include Accessibility, Database Design and Advanced Database Technologies, Adaptive Intelligent Educational Systems, Natural Language Processing and Information Retrieval. (More Information: http://labda.inf.uc3m.es/doku.php?id=en:labda_personal:personal_aiglesia) Ray M. Kekwaletswe is an Associate Professor of Information in the Department of Informatics at Tshwane University of Technology, South Africa. Previously, he was the Manager of Multimedia and Digital Computing at the American University, Washington DC, where he attained the BA, BSc, and an MSc in Computer Information Systems and Technology. He earned his PhD degree in information systems from the University of Cape Town. His research areas include ubiquitous mobile computing, human computer interaction, context and presence awareness, and emerging learning and teaching technologies. Arne W. Koesling has graduated in computer science at the Carl-von-Ossietzky Universität of Oldenburg, Germany in 2003. Since 2005 he worked for Hanover Medical School supervising the central learning management system ILIAS. From 2005 to 2008 he also worked as a research assistant for the L3S Research Center. He took part in the European TENCompetence project aiming at facilitating lifelong learning. In 2008, he switched to the newly established eLearning Service Department (ELSA). His research activity is currently centered around adaptation and adaptability in regular learning management systems. Arne is a member of the Stud.IP core-group, which is steering the further development of Stud.IP.

363

About the Contributors

Phil Marc Krüger, born 1971, graduated in electrical engineering at the university of applied science in Emden, Germany. He also received a Master of Higher Education (Lehramt für Berufsbildende Schulen) at the Leibniz Universität Hannover. Since 2001, he works as researcher in the field of e-Learning as a lecturer in media- and higher education teaching methods at the L3S Research Center. In 2008, he switches to the newly established eLearning Service Department (ELSA). There, his field of activity comprises consulting and further education of university lecturers, as well as research and development tasks to explore new kinds of teaching. His focal point of work is on video-based learning. Within this topic, he did substantial examinations about self-directed and cooperative learning. You can find more information at http://www.marckrueger.de Carola Kruse solved her state examination as Primary School Teacher in 2005 at Universität Osnabrück but did not become a teacher after all. She started working at University Osnabrück at the Centre for Information Management and Virtual Teaching (virtUOS) as a counsellor, aiding professors and academic staff with using the LMS Stud.IP and finding access to technology enhanced learning. In addition she did extra-occupational distance learning academic studies in “Adult and further Education” at Technische Universität Kaiserslautern and graduated in 2009. Since 2010 Carola Kruse is working as a researcher at Technische Universität Braunschweig in the Centre of Excellence for Lower Saxon Higher Education. She is responsible for the LMS project management and offers courses for professors and academic staff. Since March 2010 Carola Kruse is member of the Stud.IP Core Group, the commission steering the general further development of Stud.IP. Paul Lam is an Assistant Professor at the Centre for Learning Enhancement and Research (CLEAR) at The Chinese University of Hong Kong (CUHK). He involves heavily in many teaching and learning research studies and services such as promotion of the outcomes-based approach to learning, the enhancement of teaching and learning spaces, and the use of technology for teaching and learning. Paul Lam’s research interests range widely across several key aspects of teaching and learning, including web-supported teaching and learning, mobile learning, case-based teaching and learning, learners’ characteristics, self and peer assessment, teaching and learning spaces, and English language teaching (ELT). José Paulo Leal is Assistant Professor at the Department of Computer Science of the Faculty of Sciences of the University of Porto (FCUP) and Associate Researcher of the Center for Research in Advanced Computing Systems (CRACS). His main research interests are eLearning system implementation, structured document processing and software engineering. He has a special interest on automatic exercise evaluation, in particular on the evaluation of programming exercises, and on web adaptability. He has participated in several research projects in his main research areas, including technology transfer projects with industrial partners. He has over 60 publications in conference proceedings, journals and book chapters. Jack Lee acquired his Master of Philosophy in 2006 and now is a Project Executive working in Centre for Learning Enhancement and Research, The Chinese University of Hong Kong. His research areas involve the correlation between student’s experience and perception on eLearning, and eLearning application and development; usage and analysis of Weblog; student motivation of peer assessment; and

364

About the Contributors

consultation and development of teaching and learning spaces etc. Recently, Jack leads a team of eight graduate students and conducted a campus-wide research project regarding students’ learning style and learning activity; and uses the SEM to model the learning environment of the university. Judy Lo is an Assistant Computer Officer in the Information Technology Centre of The Chinese University of Hong Kong. She is responsible for developing and maintaining eLearning platforms and services for the university. She suggests and implements enhancements to the eLearning platforms and services to cope with new demands. She provides consultation and technical assistance to help faculty members on eLearning course design, courseware development and use of new technology for teaching and learning at the university. She is also responsible for eLearning promotion at the university and has been involved in major eLearning projects such as eLearning Service@CUHK, Podcast@CUHK and CUHK on iTunes U. She has a BBA in Management Information Systems, an MSc in Computer Science and a Postgraduate Diploma in Education. Carla Teixeira Lopes is an invited assistant at Faculdade de Engenharia da Universidade do Porto (FEUP). She holds a degree in Informatics and Computing Engineering by FEUP in 1999 and Master in Information Management by FEUP in 2005. Presently, she is also a PhD student in Informatics Engineering at the same institution. In the past, she was a Professor in the Biomathematics, Biostatistics and Bioinformatics scientific area of a health-allied higher education institution. Before being a lecturer, she was a Project Manager in Informatics Companies. Her main research interests are Information Retrieval and Human Computer Interaction. E-learning is also one of her general interests. Carmel McNaught is Director and Professor of Learning Enhancement in the Centre for Learning Enhancement and Research (CLEAR) at The Chinese University of Hong Kong. Since the early 1970s, Carmel has worked in higher education in Australasia and southern Africa in the fields of chemistry, science education, second language learning, eLearning, and higher education curriculum and policy matters. Current research interests include evaluation of innovation in higher education, strategies for embedding learning support into the curriculum, and understanding the broader implementation of the use of technology in higher education. She is actively involved in several professional organizations and is a Fellow of the Association for the Advancement of Computers in Education; is a University Quality Assurance Auditor for both Australia and Hong Kong; is on the editorial board of 12 international journals; and is a prolific author; recent publications and activities can be viewed at http://www.cuhk. edu.hk/clear/people/Carmel.html Lourdes Moreno, earned a PhD in computer science from Universidad Carlos III de Madrid and Mathematician in the speciality of Computation Sciences for the Universidad Complutense de Madrid. I have worked with several IT companies, in R+D Departments, working in matters about Infometrics (information measurement) especially in web-channel. Since 2002 I work as an Associate Professor and since 2004 as an Assistant Professor at the Advanced Database group in the Computer Sciences Department at the Universidad Carlos III de Madrid. Presently I belong to the Spanish Centre of caption and Audiodescription (CESYA) working in the accessibility for people with different disabilities. I have developed the doctoral thesis in the Design and development of accessible Web applications. I work in several international research projects and I am co-author of several publications (More Information: http://labda.inf.uc3m.es/doku.php?id=en:labda_personal:personal_lmoreno). 365

About the Contributors

Paulo Coelho de Oliveira is an Associate Professor and Lecturer in Electrical & Computer Engineering at the Polytechnic Institute of Porto - Higher School of Engineering (IPP-ISEP. PhD in Didactics of Higher Education from University of Aveiro in 2009, MSc in Mechanic Engineer (Option Fluids and Heat Transfer) in 2001 from Faculty of Engineering, University of Porto and Diploma Degree (4 years) in Physics and Applied Mathematics in 1994 from Faculty of Science, University of Oporto. He is a member of Laboratory of Quality Evaluation in Education at the University of Aveiro. His main research interests are related with science & engineering teaching and evaluation in Higher Education. Wiesław Pietruszkiewicz received his PhD degree in Computer Science from Szczecin University of Technology (currently West Pomeranian University of Technology) in Szczecin, Poland. He is Technical Director at SDART Ltd a UK-based company in Manchester - being an innovative researching & software developing company, where he is responsible for the creation of systems incorporating scientific components. He also holds a position of Assistant Professor at Faculty of Computer Science and Information Technologies, West Pomeranian University of Technology, where teaches about the software design and programming artificial intelligence & knowledge-based systems. His research interests relate to the software development, intelligent systems and data mining & processing. He is a member of Association for Computing Machinery. Mário Pinto is an Associate professor in Informatics Department, at the Higher School of Industrial Studies and Management (ESEIG), Polytechnic Institute of Porto. PhD in Informatics from PortucalenseUniversity in 2008, MSc in Electrical and Computer Engineering in 2000 from Faculty of Engineering, University of Porto and Diploma Degree (4 years) in Applied Informatics in 1989 from Portucalense University. He is a founding member of KMILT (Knowledge Management, Interactive and Learning Technologies) research group, and researcher at Portucalense University, and member of ISPIM. His main research interests are related with Learning management systems, knowledge management systems and support decision systems. Brendan P. Post is the Manager of Online Learning Systems at The College at Brockport where he has been a member of the technology staff for over five years. Brendan has been involved in the area of learning management systems for several years, playing an active role in the ANGEL LMS user community. His participation includes multiple presentations at user conferences, founding member of the Western New York ANGEL User Group, and primary developer of software integration between ANGEL and Apple’s iTunes U. Brendan holds a Bachelor of Arts in Computer Science and Mathematics from Messiah College, Grantham, Pennsylvania. Ricardo Queirós is an Assistant Professor at the School of Industrial Studies and Management (ESEIG) in Vila do Conde, which is responsible for courses in the area of ICT and Programming Languages. He is a PhD student of the Doctoral Program in Computer Sciences in the Faculty of Sciences of the University of Porto (FCUP). His scientific activity is related with e-Learning standards and Interoperability, Languages for XML, Architectural Integration with focus on the development of e-Learning Systems. He is an associated member of the Center for Research in Advanced Computing Systems (CRACS) - an INESC-Porto Associated Laboratory - and a founding member of KMILT (Knowledge Management, Interactive and Learning Technologies) research group.

366

About the Contributors

María Dolores Redel-Macías was born in Córdoba, Spain in 1977. She received the BS Degree in Automatic and Electronic from the University of Córdoba, Spain, in 2006. She is currently as Assistant Professor in the Department of Rural Engineering, University of Córdoba, Spain. Her current areas of interest include identification models of sound, sound quality in biofuels. She is collaborating in different educational project about educational technologies, virtual labs and e-learning tools. Marina Ribaudo is an Associate Professor in Computer Science. She is currently teaching subjects on Network Protocols, Web technologies, Peer-to-Peer systems at the University of Genova. Her initial research interests were related to the study of formal methods (stochastic extensions of Petri nets and process algebras) for the qualitative and quantitative modeling and analysis of concurrent and distributed systems. More recently, she has undertaken studies related to network applications in general (clientserver, peer-to-peer, publish-subscribe systems) with particular interest to usability and accessibility issues. From July, 2005, she is one of the members of a group called Advisory Board for the ICT Educational Support, a committee suggesting strategies for the introduction of ICT technologies in education. From February 2009, she is coordinating this group. Ana Cláudia Rodrigues is an Assistant Professor and lecturer in Human Resources field at School of Industrial Studies and Management (ESEIG), a Polytechnic Institute of Porto’s faculty. Is a PhD student at ISCTE. She was responsible for creating and managing a social network of students and alumni in the Human Resources undergraduate course. She also intervened in the course curriculum change, in order to emphasize new technologies. She has been a member of a research group of Internet and LMS usage at IPP and has published several papers studying this issue. She is also a founding member of a Human Resources research group (NID_RH). Marina Rui is researcher at the Faculty of Science of University of Genova since 1983. She has a degree in Physical Chemistry and works in Theoretical Chemistry. In the last decade she also dealt with issues related to the use of ICT in education, blended learning and e-learning.In her university, she has been Coordinator of the Committee for the WEB and of the Advisory Board for the ICT Educational Support of which she is member since 2004. Together with M. Ribaudo, she is current representative of the AulaWeb Portal at University of Genova. She is also member of the Organizing Committee of the Italian Moodlemoot. Lorenzo Salas-Morera received MSc in Agricultural Engineering in 1989 and PhD in the field of Agricultural Engineering in 1993 from the University of Córdoba (Spain). He is a Professor of University of Córdoba, in the Department of Rural Engineering, Area of Projects Engineering. He is currently the Director of Academical Organization of Córdoba University and his primary areas of research are engineering design, development of intelligent user adaptive systems for automated facility layout design, projects management and educational technology. Since 2002, he has led several projects for the adaptation of teaching methodologies to the European Higher Education Area and, in particular, to the ECTS. He received the University of Córdoba Social Council award of Educational Innovation in 2008.

367

About the Contributors

Demetrios G. Sampson is an Associate Professor on eLearning at the Department of Digital Systems, University of Piraeus, Greece and a Senior Researcher at the Informatics and Telematics Institute (ITI), Centre for Research and Technology Hellas (CERTH). His main scientific interests are in the areas of Technology-enhanced Learning. He is the co-author of more than 235 publications in scientific books, journals and conferences with at least 820 known citations (h-index: 16). He has received five (5) times the Best Paper Awards in International Conferences on Advanced Learning Technologies. He is Co-Editor-in-Chief of the Educational Technology and Society Journal (impact factor 1.067, 2009), Associate Editor of the IEEE Transactions on Learning Technologies (TLT) Journal, Member of Editorial Board of (18) International Journals and Guest Co-Editor in (16) Special Issues of International Journals. He is a Senior Member of IEEE and the Chair of the IEEE Computer Society Technical Committee on Learning Technology (LTTC). Anthony Scime is a graduate of George Mason University with an Interdisciplinary Doctorate in Information Systems and Education. Currently, he is an Associate Professor of Computer Science at The College at Brockport, State University of New York. His work in data mining has been published in Expert Systems with Applications, the International Journal of Business Intelligence and Data Mining, Data Mining and Knowledge Discovery Technologies, and Public Opinion Quarterly. Idea Group Publishing published his book Web Mining: Applications and Techniques. His current research interests include the Web mining, data mining in the social and behavioral sciences, and computing education. Tyler Swanger is currently employed with Yahoo! as a Network Operations Engineer. At Yahoo!, Tyler is responsible for monitoring and troubleshooting problems that occur within Yahoo’s network, as well as assist other Yahoo! engineers to quickly resolve customer facing issues. He studied Computer Science and Computational Science at The College at Brockport State University of New York. Tyler’s interests include high performance computing, parallel computing, data mining, and their applications. His current research focuses on data mining algorithms and their computational performance. Thanh-Thu Phan Tan has studied English Literature and Culture, Pedagogical Sciences as well as Business Studies in Hannover and Liverpool and has graduated from Leibniz Universität Hannover in 2002. After that, she has worked at L3S Research Center in Hannover for the next 5 years. In 2007, she switched to the Center of Excellence in Lower Saxon Higher Education at the Technische Universität Braunschweig. In 2008, she has finished her doctoral degree on e-learning competencies in Higher Education. By now, she has led and co-worked in several projects on technology enhanced learning in Higher Education and is – next to some teaching in that field – implementing the learning management system at the university at the moment. Dimos Triantis has studied at the University of Athens. He received his BSc in Physics in 1975, the MSc in Electronic Automation in 1980 and his PhD in Solid State Physics in 1983. In 1989 he was appointed Professor in the Department of Electronics of the Technological Educational Institution of Athens, in the discipline of Semiconductor and Dielectric Physics. Since 2007 he has been Honorary Professor Associate of the School of Engineering and Design of Brunel University (UK). He was Head of the Department of Electronics from 1992 to 1997, and for a number of years Subject Area Leader in Electronics. He is Dean of Faculty of Technological Applications from the academic year 2010-11. His

368

About the Contributors

research interests include the areas of Microelectronics and Nanoelectronics, non-destructive testing of materials via electric measurements and new technologies in education. His published research work in international journals and international conference proceedings numbers over 150 papers. Erricos Ventouras got his Diploma in Engineering from the Department of Electrical and Computer Engineering of the National Technical University of Athens in 1989 and the title of Doctor in Engineering from the same Department in 1994, on Biomedical Engineering. From 1998 he is with the Department of Medical Instrumentation Technology of the Technological Educational Institution of Athens, where he is currently Professor of Biomedical Engineering. He has taught at Graduate Courses at the University of Patras and at the University of Athens. His research interests include electronic examination methods, biosignal data acquisition and processing and tomographic techniques for medical imaging. He has over 100 publications in international scientific journals, book chapters and proceedings of international conferences. He was Scientific Leader of European Union-funded research projects. Kaitlyn Whitlock graduated from The College at Brockport in Computer Information Systems. She was the Assistant Project Manager of the ANGEL Learning Management System evaluation team in The College’s Library Information Technology Services Department. Currently, she is a Network Operations Engineer at Yahoo! responsible for monitoring and troubleshooting network problems and assisting other Yahoo! engineers in quickly resolving customer issues. Luis Zaragoza. I have been working as a Journalist for Radio Nacional de España (the Spanish National Radio) in its News Service, since September 2008. There I cover news related to disabilities in its different aspects (technological among others). Born blind in 1978, I obtained a degree in Journalism from Complutense University of Madrid in 2001 and a PhD from that University in 2007. My PhD was published as a book in 2008. I also obtained a degree in Geography and History from the National University of Distance Education (UNED) in 2004, and a masters in Management of Political and Electoral Communication from the Autonomous University of Barcelona (UAB) in 2005. I worked during 2007 in Servimedia, a news agency specialized in social subjects. I have published several papers on the history of Communication, the contemporary Spanish history and the history of blind organizations in Spain. Panagiotis Zervas received a Diploma in Electronics and Computer Engineering from the Technical University of Crete, Greece in 2002 and a Master’s Degree in Computational Science from the Department of Informatics and Telecommunications of the National and Kapodistrian University of Athens, Greece in 2004. Currently, he is a PhD candidate at the Department of Digital Systems, University of Piraeus, Greece and a Researcher at the Informatics and Telematics Institute (ITI) of the Centre for Research and Technology Hellas (CERTH). His main scientific interests are in the areas of Technology-enhanced Learning. He is the co-author of more than 25 publications in scientific journals and conferences. He has received the Best Paper Award in the 2nd International Conference on Intelligent Networking and Collaborative Systems (2010), the Best Short Paper Award in IEEE International Conference on Advanced Learning Technologies (July 2009) and the Best Poster Award in IEEE International Conference on Advanced Learning Technologies (July 2007).

369

370

Index

A

B

Academic Management System (AMS) 147, 155157 accessibility 41, 120, 124, 199-217, 241, 251, 309 Accounting Information System (AIS) 7, 207, 216 active content 78, 82, 84, 93 active discussion 78, 83, 93 active quizzes 93 active websites 82, 93 ad hoc surveys 272 administrational staff 321, 334 Adobe Flash™ 223, 228, 234, 236 Adobe FlashPaper™ 223 American Society for Training and Development (ASTD) 112, 158, 179, 194, 268 ANGEL Learning Management System 94-95 ANGEL mining 94 ANGEL users 108 ANOVA 187-189, 303, 306 Application Programming Interface (API) 153-154, 157, 160 Apriori algorithm 103-104 Architecture, Arts and Design (AAD) 252 Assistive Technologies (ATs) 200, 202-204, 207, 214 association mining 94-95, 97-99, 102, 104-105, 107, 113-114 association rule 98, 102-103, 105, 109, 113 Asynchronous Discussion Forum (ADF) 298, 300, 311, 313-314 AulaWeb 272, 276-283, 285-290 authoring tools 147-149, 152, 192, 201, 203-204, 206-208, 213-215, 233, 236 automated evaluation 205 Average Variance Extracted (AVE) 130 awareness 28-32, 34-49, 203, 325

behaviorism 4, 26-27 Blackboard 17, 54, 59, 95, 109, 111-112, 117-118, 127, 135-136, 143, 145, 147, 152-153, 155, 157, 165, 167, 177, 202-203, 216, 249, 276, 293 blended learning 11-12, 27, 88, 90, 138, 174, 243244, 248, 276, 291, 314, 324, 332 blogs 13, 23, 50, 52, 54, 71, 73, 87, 99, 108, 144, 203, 254, 256, 260-262, 264-265, 269, 274, 290, 294, 314 bottom-up approach 315-319, 321, 327-328 buy-or-build dilemma 275

C C4.5 algorithm 97, 102 change management 139, 316-319, 322-323 Chinese University of Hong Kong (CUHK) 75-76, 82, 87-89 Chi-square distribution 253 Classification Based on Multiple Class-Association Rules (CMAR) 97, 110 classification mining 98, 107, 113 classification rule 102, 113 clustering 94-95, 97-98, 105-108, 113, 242, 288 clustering rule 113 co-evaluation 310, 314 cognitivism 26-27, 45 collaborative activities 280, 282-284, 289, 291, 294, 312 Collaborative Learning System (CLS) 13, 314 Communities of Practice (CoPs) 55, 71-73 Computer Assisted Instruction (CAI) 2-6 Computer-Based Learning (CBL) 51, 70, 118, 179, 183, 314 Computer Managed Instruction (CMI) 3-4, 6 computer self-efficacy 120, 123, 135

Index

Constructed Response (CR) 72, 178, 180, 184, 195 Constructed Response Questions (CRQs) 180-181, 183-190, 193 constructivism 26-27, 55 context awareness 32, 34-37, 39-40, 42-43, 45-47 contextual inquiry 30, 37, 46, 49 Control of Substances Hazardous to Health (COSHH) 62-63 Correspondence Course Learning (CCL) 4-5 Course Management System (CMS) 12, 22, 24, 118, 143, 158, 160, 164, 174-175, 177, 184, 243, 269, 273, 289

D data mines 94 data mining 92, 94-95, 97-99, 105, 107-113, 232233, 243-244 decision trees 94-95, 97, 99-102, 110 disability 199, 212, 215 distance learning 4, 10-11, 16, 23-27, 31, 95, 111, 122, 137, 158, 177, 213, 220, 244, 266, 275276, 324

E East Kilbride & District Engineering Group Training Association (EKGTA) 58-60 e-Books 220-221, 225, 228, 230, 236 e-Business 70, 221 Eclipse IDE plugin 149 Economics (ECO) 33, 243, 252-254, 256-261, 264, 278, 286, 334 e-Content 219-221, 223-224, 228-229, 231, 233, 242, 244-245 educational technology 25-26, 46, 48, 69-73, 109110, 112, 134, 136, 138, 175-176, 213, 266, 268-269, 290, 310, 312-314 e-Examination 184-185, 192 e-Learning 6, 11-14, 16, 21, 23-27, 70, 72, 88, 9093, 96, 109, 111, 113, 117-121, 123-127, 133140, 157-160, 174-177, 194-195, 199-200, 203, 213-216, 219-245, 264, 266-269, 272-275, 282, 287, 289-295, 311-314, 324, 326, 329-333 eLearning Liaison Person (eLLP) 89 e-Mail 102, 226, 232, 300 engagement 30, 55, 71, 77, 80-81, 83, 85-86, 93, 223, 244, 316-317 engineering education 159, 176-177, 196, 198, 213, 310, 313-314 epistemology 27, 33, 52, 57 e-Portfolio 70-71, 147-148

e-Postgraduate Studies 220-221 e-Tutoring 220, 312, 314 European Credit Transfer System (ECTS) 297-298, 307-308, 311-312 Evaluation Engine (EE) 148, 150 evaluation method 199-200, 204-205, 207, 244, 295

F Faculty Information System (FIS) 7-8, 13 Federal Communications Commission (FCC) 27 female students 250, 253-254, 256, 258, 260-264 Finance Information Systems (FIS) 7-8, 13

G General Education Program (GEP) 98 Germany 24-25, 136, 175, 196, 213, 242, 266, 268, 289, 315-316, 319, 321, 323-324, 329-334 Google Talk 257 Graphical User Interface (GUI) 6, 154, 206, 233234 Group Tutoring (GT) 293, 314

H Health (HEA) 58, 63, 252-254, 256-262, 264-265, 267, 296 Higher Education (HE) 9-10, 24-26, 30-31, 38-40, 46, 48, 56-57, 65, 70-73, 90-92, 120, 123, 135, 137-139, 145-146, 155, 157, 160, 162-165, 167, 173-177, 180-181, 185, 194-195, 197-200, 212, 214-215, 219, 222, 231, 241, 243, 247252, 262-263, 265-267, 269, 272, 274, 288292, 297, 300-301, 312-313, 325, 329, 331 Humanities (HUM) 85, 252, 317, 334 Human Resource Information System (HRIS) 8-9

I immediacy 30, 34, 47, 308 IMS Digital Repositories Interoperability (IMS DRI) 144, 152 IMS Learner Information (IMS LIP) 155 IMS Learner Information Services (IMS LIS) 155 IMS Learning Design (IMS LD) 151, 177, 243 IMS Learning Tools Interoperability (IMS LTI) 151, 153, 158 IMS Simple Sequencing (IMS SS) 151 Information Communication Technology (ICT) 5-12, 14, 27, 62, 91-92, 117, 120-121, 123-124, 200, 249-250, 264, 271-273, 275-277, 281, 284, 286-289, 291, 296-297, 305

371

Index

information quality hypotheses 124 Instant Messenger (IM) 40, 329, 331-332 instructional design 26, 137, 203, 213, 243, 271272, 280-282, 288, 291 Instructional Management System (IMS) 144, 149-153, 155, 157-159, 165, 177, 192, 195, 200-202, 215, 243, 311 Instructional Television Fixed Service (ITFS) 4-6, 27 Instructor’s Characteristics 131, 140 Integrated Learning System (ILS) 2, 11, 24 internet usage 247-248, 250-251, 253-255, 262, 264, 266 internetworking 6 interoperability 70, 142-145, 147-148, 150-153, 155, 157-160, 192, 195, 268-269, 319 intimacy 34

J Jorum Team 148, 158, 269

K key administrators 334 Knowledge Management Systems (KMS) 23, 25, 92, 95 knowledge sharing 28-30, 32, 34-36, 38-39, 42-46, 49, 55 Knowledge Transfer Partnerships (KTP) 69

L Laszlo Systems OpenLaszlo™ technology 227 Law and Social sciences (LSS) 252-254, 256-262, 264 Learning 2.0 50 Learning Content Management System (LCMS) 1213, 118, 143, 147, 158, 160, 179-180, 195, 201, 203, 212, 216, 266, 268-269 Learning Management Systems (LMS) 1-2, 7, 9, 11-19, 22-25, 27-30, 32, 35, 38, 41-43, 45, 48, 50-54, 56-60, 68-69, 72-73, 75-76, 78-79, 81-98, 102, 105, 107-113, 116-127, 131-136, 138-140, 142-148, 150-153, 155-158, 160-168, 173-179, 184, 192-194, 199-205, 207, 210, 212-213, 215, 219-224, 226-228, 230-233, 235, 239, 242-244, 247-251, 253-254, 258-269, 271276, 281-283, 288-291, 315-329, 331, 333 Learning Objects (LO) 13, 75, 139, 143-144, 147152, 154, 157-158, 160, 265-266, 268-269

372

Leibniz Universität Hannover (LUH) 315, 325-328, 332 Library Information System (LIS) 9, 13, 155 LMS applications 117-118 LMS evaluation tool 96, 109 LMS logs retrieval and reading system 93 LMS platforms 28, 42, 224, 248-249, 258-262, 264-265 LMS’s characteristics 131, 133, 140 LMS success factors 140

M male students 250, 253-254, 256-258, 260-264 manual evaluation 205, 207 MCQs examination method 180-181, 183, 185-188, 190-191 MCQs test 181, 183 metadata 96, 148-154, 158-161, 201, 215, 266, 268-269 Mobile Learning Management System (mLMS) 162-164, 167-168, 173-175, 177 modern apprenticeship programme 58 Moodle 17, 26, 54, 59-60, 62-69, 71, 76-77, 80, 82, 84, 92, 105-106, 109, 111-113, 118, 127, 135, 138, 143, 145, 152-155, 157-158, 162, 164165, 167-176, 199-200, 202-203, 207-213, 215, 220, 223-224, 227-228, 231, 233, 239, 243244, 249-251, 254, 261, 263, 266-267, 271, 276-277, 281-282, 285, 289, 291, 293-294, 297, 299-302, 307, 311 Multiple-Choice (MC) 143, 152, 178, 180-184, 193-198 Multiple-Choice Questions (MCQs) 178, 180-193, 195-197 MXML language 239

N nature of activities 79, 93 net generation 250, 266, 268-269

O Online Assessment (eAssessment) 51, 59, 194, 296 Online Quizzes (OQ) 77-78, 83-85, 87, 93, 293294, 296-299, 307, 309-311 organization’s characteristics 116-117, 119, 121122, 131, 133, 140 Overall Quizzes Scores (OQS) 303

Index

P Partial Least Square (PLS) 129-130, 135 PDF creator 223, 228 pedagogical strategies 19, 21, 248 pedagogy 19, 23, 26-27, 48, 55, 90-91, 250, 311 perceived ease of use 116-117, 119-120, 122-127, 131-135 personalized learning 28, 31, 38, 40 PLS approach 129 Polytechnic of Porto 248, 251-253, 256, 262-263, 265 popularity 17, 56, 77, 79, 82, 93, 144-145, 149, 152, 200, 241 Portable Digital Assistants (PDAs) 37-40, 43, 52, 166, 177 portfolio 56-57, 73, 149, 153, 159 portfolio API 153 Practical Sessions (PS) 297-298, 301 programmed learning 2, 4, 6, 11, 27 Programming Advanced Techniques (TAP) 284 PSR-MCQs method 183, 188, 190, 192 Pure LMS Use 140

Q QSR NVivo 7 253 QTI specification 192 Question & Test Interoperability (QTI) 150-152, 192

R RDF metadata 149 RDF syntax 149 repository API 153-154, 157 Resource Description Framework (RDF) 149

S Science (SCI) 24-25, 33, 46, 72, 90, 110, 134, 137-138, 158, 160, 174-177, 196, 213, 220, 243-244, 250, 252, 268, 275, 277-279, 283, 286, 296, 310-311, 313, 324, 332-333 scoring rules 182, 193-194, 198 screen reader 200, 202-203, 207, 211, 213, 216 SDART Presentation Engine (SPE) 219-220, 223, 228, 230, 233-236, 238-242, 245 self-efficacy 119-121, 123, 127, 130, 132, 134-135, 138, 269 service quality hypotheses 124 Sharable Content Object Reference Model

(SCORM) 144, 149, 151-152, 159, 165, 200, 202-203, 225, 230, 235-236, 239, 245, 250, 268, 275, 324 Short Message Service (SMS) 40, 43 Skype 257 social awareness 28-32, 34, 37-43, 45, 48-49 social presence 28-32, 34-47, 49, 136, 312 social software 55, 72-73, 87, 176, 274, 289 special needs 199, 214 SPEEG (SPE E-book Generator) 235-236 Standardized Hyper Adaptable Metadata Editor (SHAME) 149, 159 Structural Equation Model (SEM) 129, 136 Student Information System (SIS) 9, 13 Sultan Qaboos University (SQU) 116, 127, 134 Sum of Squared Errors (SSE) 106-107 Supplementary LMS Use 140 surveys 51, 77, 89, 144, 179, 203, 207, 239, 272, 285, 291-292, 294, 297, 302, 305, 309 system quality hypotheses 124

T TCC classes 98 Technische Universität Braunschweig (TU BS) 315, 318, 324-325, 329 Technological Educational Institution of Athens (T.E.I.-A.) 178, 184, 192 Technology (TEC) 5-8, 10-12, 14-16, 24-28, 39-40, 45-48, 50-51, 53-55, 58, 60, 68-73, 88, 91-92, 96-97, 102, 109-110, 112-113, 116-127, 132140, 144, 153, 160, 162, 165, 174-176, 180, 194-196, 200, 202, 211-213, 215, 220-221, 223, 227, 230, 234, 239, 242, 250-254, 256, 258-262, 264, 266-269, 272-274, 280, 289-291, 296, 310-314, 316, 323, 325, 334 template 105, 215 Theoretical Blocks (TB) 298 Thesis Continuation Credit (TCC) 98 tool consumer profile 152 top-down approach 315, 317, 319, 321, 326-328 T-Quest 279, 282, 285-286 traditional learning 9, 11, 244, 273

U ubiquitous personalized support 28, 49 Universität Osnabrück (UOS) 326-328 University of Genova 271-273, 275, 288, 290 university strategy 315 usability 25, 96, 175, 207, 211, 213-214, 244, 258259, 294, 311, 317

373

Index

usage framework 122 User-Centered Design Process (UCD) 200-202, 204, 212, 215 User Interface (UI) 6, 153, 175, 184, 201-202, 207208, 232-233, 285 Users’ Experience with the Technology (EUT) 123

V variance-based SEM 129 Video Conferencing (VC) 4-6, 27 Virtual Learning Environments (VLEs) 12, 53, 90, 92, 135-136, 158-160, 164, 175, 177, 203, 213214, 268, 334 Virtual Learning (VL) 6, 12, 53, 92, 135-138, 147, 158-160, 164, 175, 177, 203, 211, 213-214, 266, 268, 334 vocational training 50-51, 53, 55-58, 60, 69

W WAI-ARIA Suite 203

374

Waikato Environment for Knowledge Analysis (WEKA) 98 Web 2.0 50, 89 Web 2.0 Tools 51, 68-69, 88, 144, 269 web accessibility 201, 207, 212-216 WebCT 17, 25, 76-77, 80, 82, 84, 88, 113, 118, 127, 135, 138-139, 143, 145, 147, 157, 165, 249, 293 Web Enhanced Learning (WEL) 280-285, 290-291 WEL Project 280, 282-283 West Pomeranian Business School (ZPSB) 219221, 223, 227, 232-233, 235-236, 239, 242 What You See Is What You Get (WYSIWYG) 209, 215, 227 Wiki 57, 64-69, 71, 73, 226, 279, 290 World Wide Web Consortium (W3C) 152, 162, 164-167, 171-175, 201-204, 206-207, 213-215

X XML (eXtensible markup language) 150, 152, 155, 161, 233, 235-236, 238-241, 245, 269