Higher Education Institutions and Learning Management Systems: Adoption and Standardization Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal
Senior Editorial Director: Director of Book Publications: Editorial Director: Acquisitions Editor: Development Editor: Production Editor: Typesetter: Print Coordinator: Cover Design:
Kristin Klinger Julia Mosemann Lindsay Johnston Erika Carter Myla Harty Sean Woznicki Jennifer Romanchak Jamie Snavely Nick Newcomer
Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail:
[email protected] Web site: http://www.igi-global.com Copyright © 2012 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.
Library of Congress Cataloging-in-Publication Data
Higher education institutions and learning management systems: adoption and standardization / Rosalina Babo and Ana Azevedo, editors. p. cm. Includes bibliographical references and index. Summary: “This book provides insights concerning the use of learning management systems in higher education institutions, to increase understanding of LMS adoption and usage”--Provided by publisher. ISBN 978-1-60960-884-2 (hardcover) -- ISBN 978-1-60960-885-9 (ebook) -- ISBN 978-1-60960-886-6 (print & perpetual access) 1. Internet in higher education--Case studies. 2. Education, Higher--Computer-assisted instruction-Case studies. 3. Web-based instruction--Design. I. Babo, Rosalina, 1967II. Azevedo, Ana, 1977LB2395.7.H55 2012 378.1’7344678--dc23 2011022151
British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.
Editorial Advisory Board António Andrade, Universidade Católica Portuguesa, Portugal Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Schiopoiu Burlea, University of Craiova, Romania Luís Borges Gouveia, University Fernando Pessoa, Portugal Paulo Coelho Oliveira, Instituto Superior de Engenharia do Porto, Portugal Demetrios G Sampson, University of Piraeus, Greece Steve Wheeler, University of Plymouth, UK
List of Reviewers Kamla Ali Al-Busaidi, Sultan Qaboos University, Sultanate of Oman António Andrade, Universidade Católica Portuguesa, Portugal Judy van Biljon, University of South Africa, South Africa Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Burlea, University of Craiova, Romania Nicholas Caporusso, Institute for Advanced Studies, Italy Lai Yiu Chi, The Hong Kong Institute of Education, Hong Kong Thomas Connolly, University of the West of Scotland, UK Dorota Dżega, West Pomeranian Business School, Poland José Manuel Mesa Fernández, University of Oviedo, Spain Robert W. Folden, Texas A&M University-Commerce, USA Jose Albors Garrigos, Universidad Polytecnica de Valencia, Spain Luís Borges Gouveia, University Fernando Pessoa, Portugal Malinka Ivanova, College of Energetics and Electronics, Bulgaria Arpan Jani, University of Wisconsi, USA Alexandros Karakos, Democritus University of Thrace, Greece Ray Kekwaletswe, Tshwane University of Technology, South Africa Arne Wolf Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany Atik Kulakli, Beykent University, Turkey
Jack Lee, The Chinese University of Hong Kong, Hong Kong Carla Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Lourdes Moreno, Universidad Carlos III de Madrid, Spain Muhammad Abdul Mugeet, Aga Khan University, Pakistan Lino Oliveira, Escola Superior de Estudos Industriais e de Gestão, Portugal Paulo Oliveira, Instituto Superior de Engenharia do Porto, Portugal Wieslaw Pietruszkiewicz, SDART Ltd, UK Mário Pinto, Escola Superior de Estudos Industriais e de Gestão, Portugal Ricardo Queirós, Escola Superior de Estudos Industriais e de Gestão, Portugal Marina Ribaudo, University of Genova, Italy Sandra Ribeiro, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, Escola Superior de Estudos Industriais e de Gestão, Portugal Lorenzo Salas-Morera, University of Córdoba, Spain Demetrios G. Sampson, University of Piraeus, Greece Anthony Scime, State University of New York, USA Errikos Ventouras, Technological Educational Institution of Athens, Greece Steve Wheeler, University of Plymouth, UK
Table of Contents
Foreword............................................................................................................................................... xv Preface.................................................................................................................................................. xix Acknowledgment...............................................................................................................................xxiii Section 1 Generalities and Perspectives Chapter 1 General Perspective in Learning Management Systems.......................................................................... 1 Robert W. Folden, Texas A&M University-Commerce, USA Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness......... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course............................ 50 Thomas Connolly, University of the West of Scotland, UK Carole Gould, University of the West of Scotland, UK Gavin Baxter, University of the West of Scotland, UK Tom Hainey, University of the West of Scotland, UK Section 2 Implementing and Evaluating Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs........................................................... 75 Paul Lam, The Chinese University of Hong Kong, Hong Kong Judy Lo, The Chinese University of Hong Kong, Hong Kong Jack Lee, The Chinese University of Hong Kong, Hong Kong Carmel McNaught, The Chinese University of Hong Kong Hong Kong
Chapter 5 ANGEL Mining..................................................................................................................................... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems.........116 Kamla Ali Al-Busaidi, Sultan Qaboos University, Oman Hafedh Al-Shihi, Sultan Qaboos University, Oman Section 3 Trends and Challenges Chapter 7 A Comparative Study on LMS Interoperability................................................................................... 142 José Paulo Leal, CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós, CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal Chapter 8 Mobile Learning Management Systems in Higher Education............................................................. 162 Demetrios G. Sampson, University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas, University of Piraeus & Centre for Research and Technology Hellas, Greece Chapter 9 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires................. 178 Dimos Triantis, Technological Educational Institution of Athens, Greece Errikos Ventouras, Technological Educational Institution of Athens, Greece Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility..........199 Lourdes Moreno, Universidad Carlos III de Madrid, Spain Ana Iglesias, Universidad Carlos III de Madrid, Spain Rocío Calvo, Universidad Carlos III de Madrid, Spain Sandra Delgado, Universidad Carlos III de Madrid, Spain Luis Zaragoza, News Service, Radio Nacional de España, Spain Chapter 11 The Technological Advancement of LMS Systems and E-Content Software..................................... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK
Section 4 Case Studies Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education..................................... 247 Rosalina Babo, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, NID-RH, ESEIG, Portugal Carla Teixeira Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira, ISEP, Portugal Ricardo Queirós, KMILT, ESEIG, Portugal Mário Pinto, KMILT, ESEIG, Portugal Chapter 13 LMS Adoption at the University of Genova: Ten Years After............................................................. 271 Maura Cerioli, University of Genova, Italy Marina Ribaudo, University of Genova, Italy Marina Rui, University of Genova, Italy Chapter 14 Effective Use of E-Learning for Improving Students’ Skills............................................................... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain Chapter 15 Strategies of LMS Implementation at German Universities................................................................ 315 Carola Kruse, Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan, Technische Universität Braunschweig, Germany Arne Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany Compilation of References................................................................................................................ 335 About the Contributors..................................................................................................................... 360 Index.................................................................................................................................................... 370
Detailed Table of Contents
Foreword............................................................................................................................................... xv Preface.................................................................................................................................................. xix Acknowledgment...............................................................................................................................xxiii Section 1 Generalities and Perspectives Chapter 1 General Perspective in Learning Management Systems.......................................................................... 1 Robert W. Folden, Texas A&M University-Commerce, USA In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective. Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness......... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the
varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness. Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course............................ 50 Thomas Connolly, University of the West of Scotland, UK Carole Gould, University of the West of Scotland, UK Gavin Baxter, University of the West of Scotland, UK Tom Hainey, University of the West of Scotland, UK Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web2.0 technologies provided by the chosen LMS. Section 2 Implementing and Evaluating Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs........................................................... 75 Paul Lam, The Chinese University of Hong Kong, Hong Kong Judy Lo, The Chinese University of Hong Kong, Hong Kong Jack Lee, The Chinese University of Hong Kong, Hong Kong Carmel McNaught, The Chinese University of Hong Kong Hong Kong Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University.
Chapter 5 ANGEL Mining..................................................................................................................................... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation. Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems.........116 Kamla Ali Al-Busaidi, Sultan Qaboos University, Oman Hafedh Al-Shihi, Sultan Qaboos University, Oman Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’ supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS.
Section 3 Trends and Challenges Chapter 7 A Comparative Study on LMS Interoperability................................................................................... 142 José Paulo Leal, CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós, CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal A Learning Management System (LMS) plays an important role in any eLearning environment. Still, the LMS cannot afford to be isolated from other systems in an educational institution. Thus, the potential for interoperability is an important, although frequently overlooked, aspect of an LMS system. In this chapter we make a comparative study of the interoperability features of the most relevant LMS in use nowadays. We start by defining a comparison framework, with systems that are representative of the LMS universe, and interoperability facets that are representative of the type integration with other broad classes of eLearning systems. For each interoperability facet we categorize and identify the most representative remote systems, we present a comprehensive survey of existing standards and we illustrate with concrete integration scenarios. Finally, we draw some conclusions on the status of interoperability in LMS based on our study. Chapter 8 Mobile Learning Management Systems in Higher Education............................................................. 162 Demetrios G. Sampson, University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas, University of Piraeus & Centre for Research and Technology Hellas, Greece Learning Management Systems (LMS) are widely used in Higher Education offering important benefits to students, tutors, administrators and the educational organizations. On the other hand, the widespread ownership of mobile devices has lead to educational initiatives that investigate their potential as the means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This mainly aims to support the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions, towards to a truly constant and instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. In this book chapter, we address the issue of designing mLMS for Higher Education by studying and applying the W3C Mobile Web Best Practices 1.0 to a widely used existing LMS, namely, the Moodle. Chapter 9 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires................. 178 Dimos Triantis, Technological Educational Institution of Athens, Greece Errikos Ventouras, Technological Educational Institution of Athens, Greece The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems
(LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, provided that the new method proves its equivalence to traditional methods that might be considered as standard, such as constructed response (CR) tests. Research efforts to that direction are presented. Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility..........199 Lourdes Moreno, Universidad Carlos III de Madrid, Spain Ana Iglesias, Universidad Carlos III de Madrid, Spain Rocío Calvo, Universidad Carlos III de Madrid, Spain Sandra Delgado, Universidad Carlos III de Madrid, Spain Luis Zaragoza, News Service, Radio Nacional de España, Spain Currently, the great majority of institutions of higher education use Learning Content Management Systems (LCMSs) and Learning Management Systems (LMS) as pedagogical tools. In order to make these systems accessible to all students, it is important to take into account not only educational standards, but also standards of accessibility. It is essential to have with procedures and well-established method for evaluating these tools, so in this paper we propose a method for evaluating the accessibility of LCMSs and LMS based on a consideration of particular accessibility standards and other technological and human aspects. The method application is for all LMS, in order to illustrate the effectiveness of the evaluation method, we present a case study over the widely-used LMS Moodle . In the case study, the accessibility of Moodle is evaluated thoroughly from the point of view of visually-impaired persons. The results obtained from the case study demonstrate that this LMS is partially accessible. The evaluation shows that the tool provides poor support to the authors of accessible educational contents. Chapter 11 The Technological Advancement of LMS Systems and E-Content Software..................................... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK This chapter will present the practical aspects of Learning Management Systems adoption by describing this process from the perspective of evolution, observed for LMS and e-content software at West Pomeranian Business School. The chapter will address issues and found solutions relating to LMS deployment and evolution, noticed during the management of e-learning studies. In its first part, chapter will explain the requirements for different types of studies and how they influenced the shape of LMS systems. In the following sections, the chapter will analyze different technologies and software used in the e-learning process. This analysis will also describe how efficiently use the functionality of e-learning software in relation to the users’ requirements. The last part of chapter will present SPE - SDART Presentation Engine, being an innovative e-learning presentation engine, developed in form of Rich Internet Application, to overcome the limitations observed for the previously used presentation engines.
Section 4 Case Studies Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education..................................... 247 Rosalina Babo, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, NID-RH, ESEIG, Portugal Carla Teixeira Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira, ISEP, Portugal Ricardo Queirós, KMILT, ESEIG, Portugal Mário Pinto, KMILT, ESEIG, Portugal The Internet plays an important role in higher education institutions where Learning Management Systems (LMS) occupies a main role in the eLearning realm. In this chapter we aim to characterize the Internet and LMS usage patterns and their role in the largest Portuguese Polytechnic Institute. The usage patterns were analyzed in two components: characterization of Internet usage and the role of Internet and LMS in Education. Using a quantitative approach, the data analysis describes the differences between gender, age and scientific fields. The carried qualitative analysis allows a better understanding of students’ both motivations, opinions and suggestions of improvement. The outcome of this work is the presentation of the Portuguese students’ profile regarding Internet and LMS usage patterns. We expect that these results can be used to select the most suitable digital pedagogical processes and tools to be adopted regarding the learning process and most adequate LMS’s policies. Chapter 13 LMS Adoption at the University of Genova: Ten Years After............................................................. 271 Maura Cerioli, University of Genova, Italy Marina Ribaudo, University of Genova, Italy Marina Rui, University of Genova, Italy The last two decades have seen the spread of LMS among schools, universities, and companies to augment the traditional teaching process with ICT and network technologies. This chapter presents the process leading to the adoption of a Moodle based LMS at the University of Genova in the last decade. By analyzing the data collected from the LMS logs and from questionnaires proposed both to students and teachers, we found out that the needs of the stakeholders are largely limited to resource sharing and organizational support, satisfactorily provided by the current service. Further improvements could be achieved by the introduction of a policy encouraging or forcing the teachers to use the provided LMS. A project on instructional design and, as a case study, the evolution of some of the courses involved in it are also presented. Though the redesign of such courses has improved their results, the impact on the overall organization of the degree program has been negative. We infer that this is due to the excessive freedom the students enjoy in taking their exams in Italy.
Chapter 14 Effective Use of E-Learning for Improving Students’ Skills............................................................... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain The educational system promoted by the European Higher Education Area advocates the introduction of new teaching methodologies in order to improve students’ skills as well as their knowledge in the subject areas they are studying. In response to this, new teaching strategies were implemented in Industrial Engineering and Software Engineering degree courses. The main goal of the project was to improve students’ skills in areas including problem-solving, information management, group working and the acquisition of writing and speaking skills, by means of e-learning tools. In addition to implementing the new strategies, a set of assessments including surveys, forum activity analyses and group tutorial evaluations were also carried out. The combined use of these techniques proved a very useful way of improving the students’ general skills and knowledge, especially in terms of design methods and organisation and planning ability and in general academic performance. Chapter 15 Strategies of LMS Implementation at German Universities................................................................ 315 Carola Kruse, Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan, Technische Universität Braunschweig, Germany Arne Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany In Germany, a learning management system (LMS) has become an everyday online tool for the academic staff and students at almost every university. Implementing an LMS, however, can be very different depending on the university. We introduce some general aspects on the strategies at German universities on how to implement an LMS. These aspects are mainly influenced by two main approaches, the top-down and bottom-up approach, which determine the decisions and actions on different levels at the university. In order to show how the strategies are carried out, we are presenting three case studies from universities based in the German federal state of Lower Saxony. We are going to reveal that both approaches play a part in each strategy, however differently weighted. It becomes clear that networking and collaboration plays a crucial role, not only concerning the technical development of the LMS software but also in organisational and educational terms. Compilation of References................................................................................................................ 335 About the Contributors..................................................................................................................... 360 Index.................................................................................................................................................... 370
xv
Foreword
Learning Management Systems (LMS) are now ubiquitous in institutions of higher education. This has occurred very rapidly with adoption being widespread but with little standardization. LMS’s were first used to support delivery with some communication between teachers and learners, but use has now been extended to support learning activities in innovative and diverse ways. They are also being used to increase student engagement and to track student progress – a vastly different approach to the early years of pushing resources to students. Adoption of LMS’s started with experimentation by a few with small systems. Familiarization facilitated wider adoption until eventually the “institution-wide” adoption of large commercial systems became common. Recent developments have extended the range of options from a few large commercial systems to a wider selection of open source, adaptable and specialized systems. In the intervening years from early adoption to now, higher education institutions have also gained valuable expertise in selecting, implementing, using and evaluating technologies to support learning and teaching some of which has been gathered in the chapters of this book. The authors have focused on a number of areas including: implementation strategies; use of learning management systems and other eLearning technologies; technical developments; evaluation; adoption and acceptance; and supporting skills. Chapter 1 takes a broad historical focus, reflecting on the background of eLearning from the very early days of teaching machines and computer assisted instruction through to correspondence courses and video conferencing. The role of information and communication technologies in supporting higher education processes, including teaching and learning, is explored leading to the overarching concept of learning management systems and how these support different modes of learning. The major conclusion reached is that pedagogy should drive the development and use of an LMS. Chapter 2 argues that learning and knowledge is facilitated by social interactions, implying that communication should be a key component of any eLearning system. The author argues that an LMS should facilitate social interactions at a number of levels independent of temporal or geographic constraints or the context that the knowledge activity takes place in. The communication should be facilitated by whatever means are available independent of specific LMS characteristics. The use of Web 2.0 technologies to support learning in a vocational setting is explored in chapter 3. The authors posit that despite the uptake of eLearning technologies in higher education generally, there is less evidence of uptake in the vocational sector. They aim to answer questions about whether technology can supplement the hands-on approach of vocational training, in particular the use of web 2.0 technologies such as wikis and forums. They present a case study the outcomes of which suggest that although there is potential for educational technologies to offer great benefits for vocational training, there is still much work to be completed in this area.
xvi
Chapter 4 investigates the use of LMS usage logs to facilitate teaching and learning. This is not an uncommon activity in commercial settings and, in an eLearning situation, can reveal valuable information about how the LMS is being used. The authors explore how the data can be analyzed to better inform their understanding of the online activities and, as a result, inform and improve the eLearning strategies that support institutional, faculty and departmental use of the LMS. In a similar vein, chapter 5 explores usage patterns in a specific LMS, ANGEL. Here data mining is used to explore LMS feature use. Using machine learning techniques it is possible to predict: the courses will use the ANGEL system most effectively based on course specific attributes; the interaction between features or sets of features that impact on usage; and those features that are most commonly used. This leads to a set of results that can be used by the institution to inform future decision making regarding feature selection within an LMS, the design, selection and implementation of an LMS, as well as identifying areas requiring additional training. Chapter 6 further explores usage, investigating critical factors that influence the acceptance and use of an LMS by instructors. The author suggests that although LMS survival is determined by instructor and student use, adoption is initiated by the instructor’s acceptance and subsequent use. Through a comparison of instructors’ acceptance of technology, LMS characteristics and organizational characteristics for the acceptance of eLearning, a model of overall acceptance and use is developed in a distance education setting. The study also provides insights into what additional support is needed in situations where computer use and Internet literacy is not high. In chapter 7 the issue of LMS interoperability with other systems in educational institutions is explored. The authors point out that although the LMS is directed at supporting learning it cannot be isolated from other systems in the institution. Two systems have been selected for the comparison representing a significant market share. A number of facets were selected for the comparison, using currently accepted standards, including: system communication with operational environment; learning content management; and academic management. The overall conclusion is that LMS interoperability leaves a lot to be desired. Standards relating to communication and content are relatively well developed but significant work still needs to be done in the area of interoperability of academic management. The advent of mobile technologies and the impact on course management systems in higher education is explored in chapter 8. The authors address the issue of designing such a system in the context of the Moodle LMS, specifically they investigate the application of W3C Mobile Web Best Practices 1.0. A framework is designed for a server-based mobile version of Moodle that follows the W3C guidelines. The enhancing of electronic examinations is explored in chapter 9. The authors explore an extension to multiple-choice questionnaires which allow for novel grading methodologies to be employed. They suggest that simple positive scoring rules and mixed-scoring rules introduce bias for a number of reasons. However the paired scoring method introduced here overcomes some of the shortcomings of these other methods. The authors do emphasis that the initial workload associated with constructing the question bank for multiple-choice questions is high, but overall it is concluded that the enhancements suggested here add to the value of examination tools within an LMS. With the widespread adoption and use of LMS’s in higher education, the authors of chapter 10 highlight the need for these systems to be accessible to all students. An evaluation framework is developed to evaluate the accessibility of LMS’s based on particular accessibility standards as well as other technological and human criteria. The framework is tested using the widely used LMS, Moodle, using the perspective of visual-impairment. A number of findings are reported, not least of which is that Moodle does not meet accessibly standards fully.
xvii
In chapter 11, the authors describe the legal requirements of using eLearning so support distance learning in higher education in Poland. They explore the implications of the regulations and how this has shaped LMS use. They compare the educational processes required of blended- and e-learning pathways and how different technologies can support aspects of those pathways. Extensions to the LMS that are required to support learning needs are discussed. The authors also point out that there are implications for the development of materials (e-content) which increase the cost of production on a number of levels. Finally the authors present an eLearning presentation engine that overcomes some of the difficulties associated with producing e-content. A comparison of Internet and LMS use is undertaken in chapter 12. The authors propose that more attention should be given to users’ Internet skills when developing LMS strategies since most LMS’s are web-based and most students are already adept at navigating the Internet before starting to use the LMS. The results support this proposition but also indicate that consideration should be given to using collaborative platforms such as web 2.0 technologies in place of email since many students are more familiar with using wikis and blogs for example. These outcomes can inform the selection and implementation of suitable digital pedagogical processes and tools that meet both teacher and student needs as well as informing the development of eLearning policies. Chapter 13 presents the process used by one higher education institution to determine an appropriate institution-wide LMS. The institution had a history of pockets of use and innovation to base the selection on. A subsequent evaluation of the adopted system has shown that use of the system is limited to resource sharing and organizational support, the most commonly used activities in the previous collection of systems. The authors suggest that the introduction of an eLearning policy could be instrumental in enhancing and extending the use of the LMS more fully. However they have identified that any extensions of use are beyond the current capacity of the system and support services, a catch-22 situation given the imperative of supporting the modern student digitally. A case study of using eLearning strategies to improve students’ generic skills is presented in chapter 14. Problem-solving, information management, group work and communication skills are the focus of this study. A combined strategy incorporating discussion boards, group tutoring, collaborative learning and peer assessment were implemented together with a number of assessment regimes including surveys, online activity analysis and group evaluations. These have resulted in improved student performance as well as improved perceptions of the accessibility of teachers, even though this is online. The study found that teacher participation is a key factor in motivating students to engage with learning activities as well as to lead discussions. The concluding chapter, chapter 15, explores different strategies that can, and have, been used when implementing an LMS. Two main influences are whether a top-down or a bottom-up approach is adopted. These determine the different levels at which key decisions are made. The authors present three case studies which demonstrate the different approaches. All three use a blend of the two approaches but in different mixes. All institutions eventually implemented the same system despite the different foci. Regardless of strategy it seems the key to success includes good networking and communication throughout the implementation process. Jo Coldwell Deakin University, Australia
xviii
Jo Coldwell joined Deakin in 1997 where she is currently Associate Head (Teaching and Learning) in the School of Information Technology. Before joining Deakin she gained a wealth of experience in both academia (in Australia) and industry (in both Australia and the UK). Jo was eLearning Manager for the Faculty of Science and Technology for a number of years during which Deakin University undertook the first institution-wide implementation of a major learning management system (WebCT Vista). During this time she was intimately involved in the deployment at University, Faculty and teaching levels. Since 2000 she has taught extensively online and her research interests lie in a number of areas associated with engaging tertiary teachers and learners in and with technology.
xix
Preface
E-Learning plays a significant role in education, and its importance increases day by day. Learning environments can take a myriad of distinct forms. Learning Management Systems (LMS) emerge as an important platform to support effective learning environments. According to Wang and Chen (2009), “an LMS employs a range of information and communication technologies to offer an online platform over the Internet, where a whole course can be planned, facilitated and managed by both the teacher and the learner”. In their work it is presented the main functions of some of the LMS nowadays available for educational purposes such as: learning material management, discussion forums, group emailing, audio conferencing, video conferencing, text chat, and whiteboard and synchronous document sharing. For Watson and Watson (2007), the term LMS is used to describe different educational computer applications. LMS is the framework that holds all sides of the learning process, including skills gap analysis. It is responsible to deliver and manage the infrastructural content, to identify and assess individual and organizational learning or training goals, to follow the process in order to reach those goals, and to collect and present data for supervising the learning process of an organization as a whole. With the rising of Web 2.0 and Web 3.0, learning environments are also overflowing Learning Management Systems and Institutions’ boundaries. Learning Management Systems are used all over Higher Education Institutions (HEI) and the need to know and understand its adoption and usage arises. On the one hand, there are different institutional cultures and characteristics and, on the other hand, there are several distinct LMS tools. Considering this it is expected to find out distinct experiences in the adoption and usage of LMS. The richness of each of the experiences can help the worldwide community to better understand how LMS are being used. The most used LMS according to a survey (Babo & Azevedo, 2009) answered by 51 universities from 19 different countries in 5 continents, were Moodle (Moodle, 2009), Blackboard / WebCT (Blackboard, 2009), and Sakay (Sakay Project, 2009). In that study several other LMS were referred such as ItsLearning, Desire2Learn, Claroline, METU Online, Chisimba, High Learn, Formare, Learning Space, First Class, Dokeos, eCollege, Class Fronter, KEWL. The results can be seen as an evolution. In the past years, the proprietary platforms were the most used but currently there is an increase of open source free platforms usage (Bradley et al., 2007). Consequently, there are not many studies regarding the usage level of such tools, concerning students, teachers, tools functionalities, usability, and the entire technological environment. Generally, both proprietary and open source free LMS provide several functionalities, such as, electronic distribution of course syllabi, grades and teachers feedback to students, ability to post hyperlinks to websites, forum for the exchange of ideas, wikis which allows students to swap ideas and information on projects, chat rooms for real time discussion, facilitating emailing and
xx
messaging among the participants (teacher/students, students/students), facilities for students to submit work assignments electronically, the means to administer quizzes and texts online (Janossy, 2008). It is frequent to observe that despite LMS on HEI is offered and usage stimulated, only a few of those functionalities are adopted, either by teachers, or by students. LMS are a powerful technology that has not achieved its full potential yet. As far as we know, understanding the actual aspects of LMS usage in HEI is an issue that is not sufficiently explored on research. Consequently, this is an interesting aspect to be explored and studied. The primary objective of this book is to provide insights concerning Learning Management Systems on Higher Education Institutions. The book aims to increase understanding of LMS adoption and usage providing relevant academic work, empirical research findings and an overview of LMS usage on Higher Education Institutions all over the world. The target audience of this book is composed of Education government members, Higher Education Managers, researchers, academicians, practitioners and graduate students in every field of study. LMS are not limited to a specific academic area being a trend and a new learning approach in any scientific field.
BOOK STRUCTURE This book includes fifteen chapters divided into four sections, namely, LMS Generalities and Perspectives, Implementing and Evaluating, Trends and Challenges, and Case Studies. It counted with the collaboration of researchers from 40 different universities and companies from 35 countries. Despite of the overall quality of the received proposals it was not possible to include all of them. As editors and after serious consideration during the review process, supported by our reviewers’ team and by the Editorial Advisory Board, we chose the best chapters in order to achieve the proposed goals of this book and those chapters which better fits the main focus of the book. In the first section Generalities and Perspectives Robert Folden very well understood the need of a general view of all the aspects related with the Learning Management System issue. One of the problems with much of the Scientific Literature is the assumption that all readers have the same background of the writers. This author very well assumes that is rarely true presenting the readers with the necessary foundations to go further in the field to be studied in depth. This is the chapter that any editor would appreciate to open with a book. Chapter 2 lies upon the premise of the asynchrony of the learning and knowledge sharing process which is a “human-to-human process that happen independent of time and space”. A reflection is presented towards “social awareness to determine the appropriateness of a LMS tool” considering asynchrony and ubiquity through all the process.Uncommonly a social concerning contributes from a technological university author. Both chapters are an excellent treatise of LMS in education. In the line of the two previous chapters, Chapter 3 provide us with an insight of the Distance Education evolution and the evaluation of the use of web 2.0 technologies offered by a chosen LMS in an educational context. Furthermore an interesting case study was developed by University of the west of Scotland and hereby presented. The use of e-learning in vocational courses is well explored in this chapter. Adoption and evaluation are two important and related issues regarding LMS that are presented in section 2. LMS store users’ logs in specific internal databases. These logs contain an immeasurable
xxi
wealth that can and should be used to evaluate LMS usage, in order to help Higher Education decision makers to take better decisions regarding LMS policies. Paul LAM, Judy LO, Jack LEE, and Carmel NAUGHT present, in chapter 4, an interesting study developed in the Chinese University of Hong Kong. The study took place during three years, between 2007 and 2009. The authors define three levels of analysis for the e-Learning activities, namely Popularity, Nature, and Engagement. Using these levels, it was possible to become aware of the different types of LMS usage and to define strategies in order to align its usage with the policies of the institution for e-Learning. Also using users’ logs analysis, but with a different even still very interesting perspective, Tyler Swanger, Kaitlyn Whitlock, Anthony Scime, and Brendan Post study, developed in The College at Brockport, State University of New York, is presented in chapter 5. The study was developed during the academic years of 2007 and 2008. Three data mining tasks, namely classification, clustering, and association, are implemented in order to extract useful knowledge and to obtain meaningful insights on LMS evaluation. This chapter refers to a new and interesting data mining application. In chapter 6, Kamla Ali Al-Busaidi, and Hafedh Al-Shihi present a model that intend to explain instructors’ acceptance of LMS. Despite that LMS usage depends on both students and instructs, it is up to the instructor to start the process and thus it is fundamental to understand which factors affect instructors’ acceptance and consequent use of LMS. The presented model is a valuable contribution in this direction. LMS usage in higher education is gaining momentum each day. As a consequence, new trends and challenges arise. In section 3, these issues are explored. José Paulo Leal & Ricardo Queirós explore, in chapter 7, some issues concerning LMS interoperability. In order to analyze and compare some of LMS interoperability features, a framework was developed and tested using Moodle and Blackboard. The framework defines two facets for LMS interoperability, exploring the main related issues in a stimulating, methodical, and efficient manner. Chapter 8 focuses on the use of mobile devices to access LMS supported courses. Demetrios Sampson & Panagios Zervas present a device developed in order to allow the deployment of LMS courses through online devices. This is advantageously achieved by means of the implementation and validation of a mobile version of Moodle that conforms to guidelines proposed by the World Wide Web Consortium. In chapter 9, Dimos Triantis & Errikos Ventouras contribute with the presentation of an interesting grading scheme applied in multiple-choice questionnaires. The presented grading scheme intends to prevent students from guessing the correct answers and thus develop a fair grading system. The proposed methodology is compared with more traditional ones in a real situation, bringing good insights to this issue. In chapter 10, the important theme of LMS accessibility is introduced. Lourdes Moreno, Ana Iglesias, Rocio Calvo, Nuno Delgado, & Luis Zaragoza, evaluated Moodle LMS, studying in detail accessibility concerns regarding visually impaired users. A new method was developed that leads to a better perceptiveness, which can conduct to the definition of better policies and practices. Dorota Dzega & Wieslaw Pietruszkiewicz presents, in chapter 11, a new technological solution designed to address some specific necessities of a higher education institution. Those necessities are also general necessities of a vast majority of higher education institutions. The referred solution is presented as an innovative extension of Moodle LMS, and consists of a new layer in the eLearning platform, offering additional advantages. The case Study section presents four case studies among Portuguese, Spanish, Italian and German universities. While the first three chapters present deep case studies whose main concern is the better understanding how LMS are being used, the German case study focuses in adoption strategies for LMS.
xxii
Chapter 12 characterizes the students Internet Usage and their LMS usage patterns in a Portuguese University. Chapters 13 and 14 case studies present long-lasting experiments and observation made in Spanish and in an Italian Universities. Chapter 13 presents the effort made by Genova University over the last 10 years in order to adopt “ICT educational support”. Next chapter relates the adoption of an elearning system which involved new teaching strategies and take into consideration student’s workload. The last chapter discusses the taken approaches – Top-Down or Bottom-up- of three German universities regarding the LMS implementation process. Not all the universities have had the same approach. Nevertheless the overall goals were achieved.
REFERENCES Babo, R., & Azevedo, Ana (2009). Learning Management Systems usage on Higher Education Institutions. In Proceedings of 13th IBIMA Conference - Knowledge Management and Innovation in Advancing Economies: Analyses & Solutions (pp. 883-889). Blackboard (2009). Blackboard. Retrieved July 30, 2009 from http://www.blackboard.com/ Bradley, M., Carter, J., Fitzsimons, D., Graham, J., Hurlbut, N., Marshall, D., et al. (2007). Learning Management System Evaluation Report. Executive Summary, Humboldt University. Janossy, J. (2008). Proposed Model Evaluating C/LMS Faculty Usage in Higher Education Institutions. Paper presented at MBAA Conference, Chicago, IL. Moodle (2009). moodle.org - Open-source Community-based Tools for Learning. Retrieved July 30, 2009 from http://moodle.org/ Sakay Project. (2009). Sakai Project Home. Retrieved July 30, 2009 from http://sakaiproject.org/portal Wang, Y., & Chen, N. S. (2009). Criteria for Evaluating Synchronous Learning Management Systema: Arguments from the Distance Language Classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773 Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-0070023-y Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal
xxiii
Acknowledgment
We have learned a great deal from those who lead us along our academic career and gratefully acknowledge our debt to them, especially Professor João Álvaro Carvalho. Our students have contributed in a fundamental way to our work with their editorial assistance, namely: Daniela Gonçalves, Joana Oliveira and Maria Duarte. To all the members of the Editorial Advisory Board, to the reviewers and to the authors, whose names are published in this book and who have assisted us one way or another, we feel very much grateful. We would like to express our thanks to the Publishing team at IGI Global for their expert support and guidance. This book is dedicated to our respective families. Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal
Section 1
Generalities and Perspectives
1
Chapter 1
General Perspective in Learning Management Systems Robert W. Folden Texas A&M University-Commerce, USA
ABSTRACT In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective.
INTRODUCTION Before one begins an extensive study of any topic, it is to their advantage to view the topic in a general fashion. It is important to look at the historical development of the content, as well as, some the theoretical underpinnings of the subject. This helps to develop a healthy perspective for viewing the information that will be studied in depth. It can also lead one to consider those areas of greatest interest for future research. Beginning DOI: 10.4018/978-1-60960-884-2.ch001
with a birds-eye view, you will not initially see the fine detail of any individual topic or aspect of the material of interest, but you will gain an understanding of the directions that impacts have come from and being able to better understand where the subject is likely to go.
eLEARNING ROOTS In understanding any system, it is important to understand its roots (see Figure 1). As we look at the past we can understand the present and pos-
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
General Perspective in Learning Management Systems
sibly project the future (Rose, 2004). Without this knowledge, we may never understand the present and not be able to speculate effectively on where we are going (Rose, 2004). To actually understand the foundation of learning management systems, you must begin with a totally different domain of knowledge; that of psychology; most notably, educational psychology (Holmberg, 2005). One must also look at the developments that have occurred technologically (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Wagner, N., Hassanein, K., Head, M., 2008). Where we are today is an outgrowth of where we have been and it is necessary to understand that path if we are to formulate a good sense of where we are going. There are multiple generations that we have come through (Taylor, 2001).
Programmed Learning/Teaching Machines At the beginning of the twentieth century, a group of psychologists were concerned with conditioning as an explanation of behavioral adaptation. They were generally referred to as ‘behaviorists’. They believed that all behavior (learning) could be explained by the concept of conditioning. ‘Learning’, as they saw it, could be accomplished by controlling the use of stimuli and rewards, both positive and negative. An outcome of this process was the development of programmed learning tools (Rose, 2004). Initially these tools were in the form of booklets that allowed the controller to manage the stimulus by applying the appropriate reward and thus produced the desired behavior/learning. These booklets were later moved to a mechanical device or teaching machine (Rose, 2004) (Keegan, 2002). The material was presented in very small steps that were referred to as a frame. The student was then presented with a blank to fill in after which they were provided with the correct answer. In their original conception they were not officially graded or ranked, but the student worked through the material. Based upon student performance, the
2
same material would have been repeated or new material presented. This form of instruction was very linear in nature and mastery was the end goal (Baggaley, 2008). Others provided some form of grading or required mastery before the student was allowed to progress. See http://www.greenchameleon.com/gc/blog_detail/weve_been_imagined/ for pictures showing students using a selection of these machines.
Computer Assisted Instruction When computers came on the scene in the 1950’s (Watson, W. & Watson, S., 2007), they became the teaching machines and the process was referred to as computer assisted instruction (CAI) (Rose, 2004). The foundation of this learning was individualized instruction (Rose, 2004). The theory was that individual students needed to learn at his pace and in his way. This system was also referred to as an Integrated Learning System (ILS) (Rogers, L. & Newton, L., 2001) (Underwood, 1997).The instructor ensured the proper design of materials so logical organization allowed the student to move through the material in an appropriate manner. This process required that each student had access to a computer for the appropriate amount of time for learning to occur. Originally students used ‘dumb terminals’ attached to mainframe computers. Each student could access his or her own files with the results stored in a centralized database. In the 1980’s and 1990’s intelligent computers in the form of PC’s assumed this role (Keegan, 2002) (Eteokleous-Grigoriou, 2009). These computers were eventually connected through local area networks and had the software stored on a centralized server. Students worked in a networked environment with the instructor moving about the classroom to help the students over the difficult portions and to keep the students on task.
General Perspective in Learning Management Systems
Figure 1. eLearning roots
Computer Managed Instruction One of the early goals of computer assisted instruction was to free the instructor from the instructional process and enable her to become a coach for the students (Rose, 2004). Therefore, course systems were designed to allow the student to progress through a set of materials according to a plan established by the instructor (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999) (Underwood, 1997). To track the student’s progress and provide for a record keeping process, computer managed instruction (CMI) became available (Rose, 2004) (Underwood, 1997). This generally included
more than one course and allowed the students to move forward in the curriculum according to her individual time frame. Many of the CMI systems provided remediation modules that allowed the students to repeat instruction in concepts that they had not sufficiently mastered on previous attempts through the material (Underwood, 1997). Student scores, number of attempts, and other pertinent information were stored in a system database for the instructor or course administrator to view (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991) (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999) (Underwood, 1997).
3
General Perspective in Learning Management Systems
Correspondence Course Learning (CCL) Beginning in the mid-nineteenth century, students were able to study in remote locations using correspondence courses (Gaspay, A., Dardan, S., & Legorreta, L., 2008) (Homberg, 2005). These text-based programs allowed the students to receive written information and a series of questions or projects that tested or reinforced the desired learning. In most cases the students worked independently (Taylor, 2001). Instructor involvement came in the form of written communication to students after they had submitted their work for grading (Holmberg, 2005). In the 1970’s and 1980’s, these courses were enhanced with audio or video recordings to provide the student with some content that was not easily communicated by words alone. In most cases the students and instructors were not in the same location, making this a form of distance learning (Keegan, 2002). This was an attempt to enable students who could not afford, for one reason or another, to travel to the instructor to receive an education. Again, this was basically individualized instruction, but it was not as individualized as the Programmed Learning, CAI, or CMI.
Instructional Television Fixed Service (ITFS) Instructional Television Fixed Service was developed beginning in the 1950s and greatly expanded in the 1970’s and 1980’s as a course delivery platform (Saba, 2000). The students participated in classes that were often remote from the instructor. There was usually an instructor present with one group of students while the others were at one or more remote sites. In many cases this was two-way video and audio, but was mostly one-way video with two-way audio. The audio in those cases was done over an audio bridge. This allowed the students to see the instructor and interact in much the same was as they would in a
4
contained classroom environment. This approach used broadcast television to send the video and the instructor audio (Gaspay, A., Dardan, S., & Legorreta, L., 2008). Because of its use of the television broadcast medium, it tended to be quite expensive. In the 1980’s and 1990’s this was enhanced and somewhat replaced by the sending of videotapes to the students to complete their coursework (Taylor, 2001). It was a blend between correspondence courses and ITFS. While this helped to bring the cost down, it did not provide what either the instructor or the student preferred and so did not do well in the long run.
Video Conferencing (VC) This approach gave way in the 1980’s and 1990’s to video conferencing systems (Taylor, 2001). These conferencing systems provided two-way video and audio links between multiple locations. In most cases this was compressed video and an audio bridge (Baggaley, 2008). While the students could see and hear, the quality often left much to be desired. Over time, the systems improved and use increased. They were hampered by the inability of systems to communicate with other manufacturers’ systems. Theoretically, they all complied with a uniform set of standards that should have made communication possible, but each manufacturer included proprietary algorithms that made it virtually impossible. During the latter part of the decade, there was more cross-collaboration, but it was never an effective method of allowing remote students to participate in a classroom.
CIRCLE OF FOCUS Classroom Based Learning Programmed learning, CAI, and CMI focused on improving classroom teaching. They were basically built upon the model of behaviorism which emphasized individualized instruction
General Perspective in Learning Management Systems
with instructor interaction. The focus was upon improving the learning of students in a contained environment with synchronous learning. This can be seen as allowing technology to find a place in the classroom. While there was talk about replacing the instructor, it was mostly a changing of the role of the instructor. Instead of the instructor being the direct dispenser of knowledge, the instructor became the facilitator/coach in the learning environment.
Distance Based Learning Correspondence course learning, video conferencing, and ITFS focused on distance education (Guri-Rosenblit, 2005). They were attempting to solve the problem of meeting the needs of students who could not or would not participate in a selfcontained classroom (Wagner, N., Hassanein, K., Head, M., 2008) (Wagner, N., Hassanein, K., Head, M., 2008). With students distributed over great distances (Guri-Rosenblit, 2005), there was little or no interaction with one another. Their only source of interaction was with the instructor or the instructor-assigned mentor/proctor. While ITFS and video conferencing theoretically allowed for some interaction, it rarely occurred because everything generally came through the instructor and that tended to stifle any student to student interaction. In some cases, distance-based learning was mated with CAI to form a sort of a hybrid approach. The students would complete computer simulations or drill and practice but also participate in video conferences to receive input from their instructor. Methodically, distance education has adapted by using the ever increasing improvements in the information technology to meet the needs of the students and improve the interaction between and among students and faculty (Baggaley, 2008).
INFORMATION COMMUNICATION TECHNOLOGY (ICT) While all of this technology development within education was going on, information communication technology was developing rapidly (Garrison, 2000). The development of the mainframe computer allowed companies and schools to automate business knowledge acquisition, storage, and processing. It led to many improvements in the functioning of the business side of the educational enterprise. Computers began to invade every aspect of business and then began to spread throughout the lives of individuals. As a result, it is nearly impossible to find a major device that does not possess at least a rudimentary computer. Also, individuals often depend on computers to allow them to access information and share information in an open and free fashion (Hamuy, E. & Galaz, M., 2010). Most individuals find it so difficult to carry on a normal life without computer interaction that we are in the process of making wireless internet a ubiquitous factor in our lives (Gaspay, A., Dardan, S., & Legorreta, L., 2008).
Networking During the 1980’s and 1990’s personal computers became networked with one another so that they could share information across platforms. They were able to present information in a graphical manner. This provided for a more enriched environment for the delivery of instruction (Rose, 2004) (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991). The systems could blend audio and video in such a manner as to appear seamless. They also allowed for greater interactivity with the students (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991). Because they were networked, students could now work on any available computer to continue the work that they had begun previously. Instructors could view individual student work unobtrusively, providing feedback as necessary. Students could participate
5
General Perspective in Learning Management Systems
in elaborate collaborative environments to develop their individual learning. As the computer continued to improve in capability, the range of activities that the students could participate in continued to expand.
Internetworking In the mid-1990’s we saw the development of internetworking with the introduction of the World Wide Web (WWW) (Keegan, 2002). This allowed networks to communicate with other networks far removed from the original source in a reliable fashion (Rose, 2004). This whole development coincided with the development of the graphical user interface (GUI) and the use of media rich environments for the delivery of information. Technology was developing at a very rapid pace, with processors able to handle larger workloads at much faster rates than ever before. Storage space was cheaper and cheaper to provide (Gaspay, A., Dardan, S., & Legorreta, L., 2008). Sound and video delivery systems could provide greater enrichment for less expense. These developments allowed the designers to enrich their information delivery, ignoring most of the limitations of the past (Taylor, 2001). Course material could be asynchronously delivered to many individuals in an individualized manner or synchronously delivered to large masses at one time or in some other combination (Rose, 2004) (Wang, Y. & Chen, N., 2009). As the Internet became ubiquitous, the barriers to delivery of information fell away, allowing for ubiquitous access to the information (education) provided (Taylor, 2001). Students began using computers more and more in their everyday lives. They became ever more comfortable with computer delivery of information in text, video, and audio. This familiarity made it easy to move educational delivery to this platform (Keegan, 2002). Students were able to use the Internet to seek out and obtain information on their own with or without the guidance of an instructor/mentor
6
(Rose, 2004). The technology could fade into the background and allow the course material to move to the forefront, regardless of the overall purpose of the information delivery. The nature of the information delivered was more a matter of perspective than one of substance.
E-Learning/Virtual Learning This ability to move education materials electronically around the world in both synchronous and asynchronous manners created the ability to develop a classroom irrespective of time, space, and proximity (Miller, T. & hutchens, S., 2009). Even with VC and ITFS, you needed to be in fairly specific locations to participate in class activities (Guri-Rosenblit, 2005). While Programmed Learning, CAI, and CMI could be done remotely, they usually were done with the students in one location connected to the same network and using the same hardware system (Baggaley, 2008) (Guri-Rosenblit, 2005). Today, all of those limitations seem irrelevant (Dillenbourg, 2008). Operating on the platform of ICT, educational institutions are delivering all levels of education and training over the Internet (Hamuy, E. & Galaz, M., 2010) (Ozkan, S., Koseler, R., & Baykal, N., 2009). Corporations seek to reduce their training expenses by providing training over the Internet rather than sending their employees to far off places with the related expenses. Even conferences have moved to the Internet to provide similar types of experiences at the attendees’ desktops rather than requiring them to attend the conference at some exotic location. These are also recorded and made available in an asynchronous manner for those who could not or would not attend when the event was going on. ICT is more than just a delivery mechanism; it has become a shaper of what we do (Rose, 2004). It reaches into every area of our lives and has touched nearly every individual on the face of the earth (Guri-Rosenblit, 2005). As the devices that access and share the information have become
General Perspective in Learning Management Systems
ever smaller, they have transformed more and more of our lives (Taylor, 2001). They have become the medium for developing communities far and wide. Individuals can find out even the most mundane or inconsequential pieces of information in a split second (Garrison, 2000). They can also share the most intimate details of their lives with the world just as quickly. These tools also allow instructors and students to share with one another in near-real-time, no matter where they are located. Information is not limited to text or even text and still images. It is possible to share full motion video and audio in very high quality over very great distances. These videos can be of real events or simulated reality. This has opened the way to virtual reality to function in an educational environment. This pervasiveness of ICT is forcing academic institutions at all levels to move their education offerings to an electronic platform to some extent (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Eteokleous-Grigoriou, 2009). It does not require, at this point, that everything be done in an electronic medium, but it does mean that some of it must (Taylor, 2001). This push is coming from multiple sources; from almost every stakeholder group that is involved in the educational process. That pervasiveness also means that instructors have a deeper well of knowledge about their students than ever before (Eteokleous-Grigoriou, 2009). While electronic media allow a degree of anonymity, it also opens up a greater breadth of information about each participant to all of the other participants in the educational process (Dillenbourg, 2008).
INFORMATION COMMUNICATION TECHNOLOGY AND LEARNING MANAGEMENT SYSTEMS For this discussion, we will focus on academic institutions and the systems that are involved there. While they are much the same as those of
other businesses, this focus will enable a clearer correlation to learning management systems in the context that we are viewing them. We need to remember that nearly all of these ICT systems existed in paper form before they appeared in electronic form. Most of what was done in the early systems mirrored what had been done in paper forms. The basic purpose of moving these systems to digital format was to automate the redundant activities in order for the users to focus on those aspects that could not be automated, thereby creating performance improvement (Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, J., 2008) (Jones, 2004). As the systems have developed, they have increasingly moved into areas that were not possible to do or were easily done in print form (Eteokleous-Grigoriou, 2009) (Cradler, 2008) (see Figure 2).
Finance Information Systems (FIS) The first ICT system to be developed was the finance information system. This system allowed enterprises to track income and expenses (Paulsen, 2002) and use the information to make informed decisions. It is sometimes referred to as an accounting information system (AIS). It has been defined as: A system “ . . . that processes financial transactions to provide (1) internal reporting to managers for use in planning and controlling current and future operations and for non-routine decision making; (2) external reporting to outside parties such as to stockholders, creditors, and government agencies.” (Answers.com, accessed on 08-18-2010) Information was generally input using batch files and outputs were done in much the same way. As technology improved, it was possible to do input during real time and to schedule major outputs during the off hours so as not to degrade the performance of the system. These systems generally are comprised of three major components; the database and its front end, the control system, and the reporting system. The database stores all of the essential information and
7
General Perspective in Learning Management Systems
Figure 2. Information Communication Technology (ICT)
provides the input screens (front end) that allow the users to input the information. That information may be added to the database in real time or stored in a temporary ledger to be added at a more efficient time. Generally, if it is stored in a temporary ledger, it is added to the database at the close of business and the information will be available for reports after that time. The databases also allow for locking of the records so that they cannot be changed in any fashion without creating a record of the change (a part of the control system). The control system allows for the ability to develop appropriate controls on the system and users to ensure the integrity of the information contained in the system. The reporting system allows for the processed information to be displayed in an appropriate manner for the decision makers and other interested parties.
8
Faculty Information System (FIS) Faculty information systems are basically a human resource information system (HRIS) that allows an institution to maintain information on nearly everything that needs to be tracked or analyzed about faculty (current and former) and those who are applying for positions. It is able to be maintained by professionals in human resources and by the faculty themselves in order to maintain the most accurate and up to date information. Not only does it maintain information on the classes taught by each faculty member, but their academic credentials and relevant experience as well. It provides a place to maintain a record of their scholarly activity and their service involvement. These systems also allow for the publication of documents necessary for the faculty to perform their duties in an efficient and effective manner.
General Perspective in Learning Management Systems
Initially these systems did not communicate with any other systems, but more and more these systems are integrated with all of the information systems of an institution so that information only needs to be input one time.
Library Information System (LIS) The initial library information system focused on the card catalogue and the record of where an individual item resided. It was basically a large inventory tracking database for library holdings. It served to automate what a librarian was responsible to do in order to maintain control of the assets of the library. As more and more assets became electronic in nature, it also began to be a repository of those items and became then a digital library system. This system now makes it possible for patrons to access the resources of the library without having to be physically present. It also enables libraries to share their resources with one another in a seamless fashion; enabling their patrons to have access to a much broader set of materials than would be possible otherwise. While LISs served to just automate library processes in the beginning, they have now entered into the process of transforming the way that the library delivers information and the way that patrons access information. With LISs, libraries have moved from being islands of information to becoming portals to the information stored in the network of libraries worldwide.
Student Information System (SIS) Student information systems provide the ability to enter student information into a database that will provide an electronic grade book, student course schedules, and other student-related data needs for a school, college, or university (Paulsen, 2002). These systems grew out of the need to automate the various record keeping responsibilities of an academic institution. They have grown to the point of being able to provide advanced informa-
tion to the decision makers on par with enterprise resource planning systems or the HRIS systems. It is rare for an institution of any size to not have some form of an SIS system. While many of these were developed in-house initially, most are commercially available today. In the K-12 market, these systems are being outsourced to a hosted environment allowing for many other enhancements to be included in the SIS system. Most of the commercially available systems today provide the ability to integrate with all of the other ICT systems functioning in an academic institution.
Learning Management System (LMS) In a simple way learning management systems involve the application of ICT to the process of education, but it is a bit more complicated than that (Dillenbourg, 2008). An LMS provides the structure for the delivery and management of instructional content, while providing assessment of individual and organizational learning goals, tracking of progress toward those goals, and providing the data necessary for the management of the learning process of the institution as a whole (Watson, W. & Watson, S., 2007) (Kim, S. & Leet, M., 2008) (Rogers, 2001) (Perry, 2009). In today’s educational environment ICT is often involved in all types of learning systems (see Figure 3). The term was first used to refer to the management system of the PLATO K-12 learning system (Watson, W. & Watson, S., 2007). Today, there are basically four broad system designs used, as seen in the accompanying diagram. These designs are determined by the pedagogical focus of the education enterprise.
Traditional Learning Historically, this form of learning has been the staple for hundreds of years, from the very founding of higher education institutions. Students meet together in real time and in a specified location with the instructor present. This instructor cen-
9
General Perspective in Learning Management Systems
Figure 3. Learning system models
tered approach is the predominant modality of instruction in higher education. ICT can be used to enhance the educational experience with the use of technology delivering the instruction and maintaining student records. The instructor is the primary determinant of what use will be made of ICT in the delivery of instruction and to some extent to its use in the completion of assignments required for the instruction. Most information will be supplied by an instructor directly or in preprinted materials, but may be supplemented with various forms of ICT in the classroom. The students may utilize ICT outside of the classroom to produce class assignments, submit class assignments, or locate information for class, but the primary delivery of instruction is done in a face to face mode. This instruction is delivered synchronously.
10
Distance Learning Distance education dates from the mid nineteenth century. Students and instructors are separated by time, location, or both in this model of instruction (Gaspay, A., Dardan, S., & Legorreta, L., 2008). The materials may be delivered to remote locations via print or ICT, but this form of instruction does not preclude the use of remote classrooms. This form of instruction may be done synchronously or asynchronously. The emphasis of this mode of instruction is the separation of the student from the instructor and not the delivery mechanism (Gaspay, A., Dardan, S., & Legorreta, L., 2008). It does not matter that the separation is physical or temporal, just that there is some separation between the two participant groups. This mode of instruction does not focus on the delivery mechanism or the technological involvement. The
General Perspective in Learning Management Systems
focus may be the individual or a group (Keegan, 2002). Early forms of this modality focused on individual instruction (correspondence courses, etc.), with technology enabling group instruction beginning in the 1980s. This form of instruction was not highly valued until the advent of open universities in the 1970s. Most distance education in the United States is group focused, while most distance education in Europe is individual focused (Keegan, 2002). Regardless of the focus, most materials are of the pre-prepared variety. Instructors may utilize materials that were prepared by another or even by a group of others. The teaching materials may be produced up to ten years before the student interacts with them in a learning environment. Furthermore, the institution that created the materials may not be the institution that is awarding credit for the course completion. There exist clearing houses for courses that can be combined to create distance learning programs of study.
Blended Learning In the blended learning mode, parts of the instruction are delivered in a traditional format while other parts of it are delivered using ICT (Gaeta, M., Orciuoli, F., & Titrovato, P., 2009). It is this blending of the delivery modality that attempts to use the strengths of both formats to enhance the educational experience (Dillenbourg, 2008). ICT is more than just an adjunct to the process, but the students and instructor are required to utilize ICT in order for learning to occur properly. This form requires that parts of it are synchronous and parts are asynchronous. One form of blended learning is called an integrated learning system (ILS) in which networked computers or terminals with a management system monitors and records student performance results and distributes learning modules based upon those results (Dillenbourg, 2008) (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999). These are truly digital systems and can be used over a dis-
tance; they are most generally used as an adjunct to traditional learning. The primary function of these systems is to remediate performance deficits in basic skills. This form of blended learning is much more like the programmed learning roots from which all of this grew. These systems contain a management system that controls the flow of data between the other components, curriculum content that provides the tutorial, practice and assessment modules, and the student record system that maintain registration and performance information on every student enrolled in the system (Rogers, 2001). These are not usually integrated with the ICT systems of the institution. Their structure mirrors the structure of learning management systems without the broad connectivity found in those systems. Some examples of ILS are CentraOne, IntraLearn, Lyceum, and Silicon Chalk, Odyssey, and Plato (the same company that originated the name learning management system).
E-Learning E-Learning had its beginnings in late 1994 or early 1995 and really soared after 1996 when the first modern content management system was developed. In E-Learning all of the instruction is online using ICT to deliver the content. By 1998, it was viewed a mature field of distance education. Michigan Department of Education defines this as “A combination of structured, sustained, integrated, online experiences accessed via a telecommunications network.” This may involve synchronous and asynchronous delivery of instruction. It blends aspects from all of the other models into a model that is not just ICT enhanced, but one in which it would not occur without the use of ICT. In today’s implementation of E-Learning, ICT is transformative and not just supportive of the process of education (Gaspay, A., Dardan, S., & Legorreta, L., 2008). In some circles, E-Learning is used to refer to all learning that occurs through the mediation of ICT, but I use the term more formally to refer to structured/
11
General Perspective in Learning Management Systems
purposeful learning that utilized ICT for its development, delivery, and management in such a way that the instruction is transformed by the medium in which it occurs. It is much similar to the distinction between learning that occurs through independent effort of the student and that which is accomplished through the collaborative effort of instructor and student. Learning happens in both situations, but in the latter, it is substantially different than it would be otherwise because of the nature of the delivery medium. While a learning management system could be utilized in some fashion in each of these models, it is in E-Learning that it is an absolute necessity. ICT in all of these models could be referred to as learning support systems. The problem is that there are no universally accepted definitions for these learning systems that will serve to distinguish them from other ICT systems used in the educational process. For this reason, the terms are used haphazardly to refer to various systems or parts of the system as if they were the same.
COMPONENTS OF A LEARNING MANAGEMENT SYSTEM In a nutshell, LMS is the software that automates the administration and delivery of learning (Watson, W. & Watson, S., 2007) (Baturay, 2008) (Rogers, L. & Newton, L., 2001). It is the overarching ICT that provides all of the functions necessary to provide learning in a digital format at a distance (Wang, Y. & Chen, N., 2009). This does not preclude the ability for it to also serve in a blended learning environment, but it must be able to provide that learning where there is spatial, temporal separation or both. The system may have any number of components, but we will look at the major components that one should expect to find in a modern learning management system. It is important to remember that the technology is generally learning theory independent and can be utilized to function in most any theoretical
12
system (Jones, 2004). However, one must consider theoretical orientation when considering the appropriateness of any learning management system (Wang, Y. & Chen, N., 2009). While this seems pretty straight forward, there is much confusion as to what is meant by the various systems and components of systems. Some authors state that CMSs are LMS that are used by academic institutions while LMS are used in industry (Hamuy, E. & Galaz, M., 2010) (Daniels, 2009). Course management systems are also confused with content management systems, which both use the same acronym, CMS (Daniels, 2009). Course management systems and virtual learning environments (VLE) are basically the same thing with terms differing by the region in which they occur (Daniels, 2009). Course management system is the term used in North America and VLE is the term used in Europe (Daniels, 2009). Content management systems are generally systems used in industry to publish information to various audiences to support their work processes. A related term is learning content management system. It refers to the system that delivers the specific portions of learning content to the learner during the use of the LMS. We will not be referring to content management systems apart from learning and so CMS will always refer to course management systems in this chapter.
Course Management System (CMS) The CMS provides the linkage with all of the other systems involved in the learning endeavor. It is truly the brains of the whole system. This system allows the appropriate individuals to add or remove courses, to sequence courses within a curriculum, add students to the course, assign instructors to individual courses or sections of courses, to monitor other processes of the system, etc. (Watson, W. & Watson, S., 2007) (Daniels, 2009). This is probably the most frequent system confused with LMS. Many individuals believe that is all an LMS does and so what is the difference. To do its
General Perspective in Learning Management Systems
job, this system must interface with the FIS and SIS systems. It is also often linked with the LIS system. These linkages are often rudimentary and not well established. This lack of strong linkage can cause considerable problems to the end users and the institution alike. When it is functioning well, it is like a good traffic officer seeing that everyone and everything gets to where it needs to be in the most efficient manner.
Learning Content Management System (LCMS) The primary purpose of the LCMS is to develop, store, organize, and distribute multimedia content to support the delivery of E-Learning (Watson, W. & Watson, S., 2007). This over arching system has many subsystems that allow it to perform its duties. Most of these occur in the background and the users do not see them or even need to. Course content authoring is one of these background activities. While the instructor uses this system to design the course and upload the appropriate content, they really don’t need to understand what is being done to make it happen. Once the material is stored, usually in the form of learning objects (Watson, W. & Watson, S., 2007), it needs to be accessed at the appropriate time to support student learning (Watson, W. & Watson, S., 2007). These processes need to function effectively as purposed or they will negatively impact the quality of the learning delivered.
Collaborative Learning System (CLS) The CLS is the system that allows the use of the newer aspect of Web delivered content (Web 2.0). This system provides the tools for email communication, discussion groups, newsgroups, instant messaging, blogs, bookmarking, notice board, search tools, etc. Not all LMS have these capabilities, but these and many more are managed by the CLS. This is the part of the LMS that changes the most rapidly. Many times the capabilities in this
system are accomplished by third party apps that are added to the system piece meal. This process can create some very unusual difficulties if the system is not managed effectively. This is also the area where mobile connections are usually made and managed. It is within the CLS that many instructors will develop the social learning that should be part of any learning program that seeks to meet the needs of a diverse student population. However, just using collaborative tools without the requisite understanding and application of social learning theory will not produce the desired results. The importance of this area is supported by belief by some educational practitioners that only 20% of learning occurs based upon formal instruction and that the other 80% is produced by informal instruction that occurs when students interact in social environments involving each other and the instructor.
Assessment Management System All forms of assessment are managed from within assessment management system. This is also where the grade book is located. Assessments could be exams, homework, projects, etc. They could involve student submissions, as well as student participatory activities. The purpose of this subsystem is to ensure that proper delivery and recording of results for all assessment items is maintained. The system should provide the instructor with the tools necessary to assess student performance on the appropriate measures. It should allow the instructor the ability to design these measures with some degree of flexibility. The assessment management system should enable the instructor to provide the student with adequate and timely feedback on their performance so that the student will benefit from the learning and be able to improve the desired outcomes of the learning experience (Watson, W. & Watson, S., 2007).
13
General Perspective in Learning Management Systems
LEARNING MANAGEMENT SYSTEM STAKEHOLDERS There are basically four stakeholder groups that are directly involved in LMS (see Figure 4), but there is a fifth that is peripherally/externally involved. The four primary/internal stakeholders are students, faculty, administrators, and IT staff (Wagner, N., Hassanein, K., Head, M., 2008). The peripherally involved stakeholder group is that group who will hire the students once they complete the education using the system or are involved in accrediting the education institution to be able to offer a degree or credential. Each of these groups holds slightly differing sets of wants and needs, but they also share some commonalities.
Students Students are the ultimate consumers of the LMS output. Generally speaking they are either undergraduate or graduate students of the educational institution. They may be enrolled in one or more courses delivered by the LMS. While there are a number of motivations for their using an LMS, some of the primary ones are access to an education, convenience, and, for a few, because they prefer this mode of learning (Wagner, N., Hassanein, K., Head, M., 2008). Some will be using an LMS because there is no other way to complete the program of study that they desire. In a sense, then, this relates to access. Modern students have grown up with an increasing involvement with technology in their day to day lives. Few, if any, know of a time when they did not have access to the internet to further their contact with one another and with the information that is available from a broader perspective, especially in the western world. Most students today have an extensive background with ICT and use it on a daily basis. This technological background of these students will impact what they expect from any delivery system (Wagner, N., Hassanein, K., Head, M., 2008). It will also play a major role in how they
14
are able to utilize the system to improve their educational outcomes and to meet the goals that they set for themselves in the process, as well as the goals that may be set by the faculty and administrators.
Faculty Instructors, more than anyone else, determine the nature of the E-Learning experience (Wang, W. & Wang, C., 2009). Depending on their theoretical foundation, they play a larger or smaller role determining the educational experience of the students. Generally, they are the arbitrators of whether or not the students will participate in group activities, independent study, or synchronous events, to just name a few ways in which they control the E-Learning experience. If their theoretical persuasion is behaviorist, they may design instruction in such a way as to move the student in a structured manner to the desired outcome, but if they are constructivists, they will allow the students to determine how they will achieve the end result. These are two of the many facets of their role in the instructional process. Ultimately, E-Learning creates many changes for the instructor (Wagner, N., Hassanein, K., Head, M., 2008). Their role is transformed regardless of their theoretical persuasion, because the technology and distance of the students will force that change somewhat. Delivery of information, motivation of students, interaction with students, assessment, etc., will all differ because of the medium of delivery. Instructors will require a differing level of technological sophistication depending upon the level of technical support available, but all will need some level of technological knowledge to be able to function in this environment (Wang, W. & Wang, C., 2009). Instructors in the past spent a great amount of their time in knowledge creation, either in the form of original research or by consuming that research (Jones, 2004). With E-Learning, they will be spending more time in the development
General Perspective in Learning Management Systems
Figure 4. LMS stakeholders
and delivery of learning materials than they have historically done (Wang, W. & Wang, C., 2009). Some studies have indicated that they may spend twice as much time in this process, even with the aid of support staff. This shift in time commitment will result in a shift in the role and expectation of instructors in the future.
IT Staff IT staff are the individuals who have traditionally only been involved in supporting the business side of the institution. If they were involved in educational delivery, it was to support systems that were used in traditional classrooms. They now are involved in systems that must be kept
operational 24/7 (Jones, 2004). These systems are mission critical, but are used by personnel who often times do not understand the intricacies of their function. While the end users will have some technical expertise, they will not generally be technologists, but will have some strong expectations from the systems that they use. This will increase the pressure on the IT staff to provide the services desired. IT staff will have to better understand the educational theories within which the system will function in order for the system to perform according to the needs and expectations of the users (students and faculty). It will no longer be sufficient for IT staff to just understand technology, but they will have to also be cognizant of
15
General Perspective in Learning Management Systems
the environment in which the technology will be implemented (Jones, 2004). It will also force the staff to include the end user in the design process from the very beginning and throughout the lifecycle of the product. These will be different roles for the typical IT staff.
Administrators Administrators in traditional institutions are primarily seeking to create easier access to their institution and remove geographic barriers to student participation (Wang, W. & Wang, C., 2009). In this manner, they are increasing their student population and expanding their academic offerings to a wider market. They are also trying to remain responsive to the market trends and the desire of their targeted market (Jones, 2004). They are attempting to expand the institutional system while maintain fiscal responsibility. In the process of expanding their delivery modalities, they want to maintain the quality of their course offerings so as to not destroy their brand image. This often requires that they need to pay attention to the broader issues of the Environment (the outer ring of our diagram). While the other stakeholders focus on a few of the other stakeholders, this group must pay attention to and balance the needs and concerns of all of the stakeholders. They also must focus on the future needs of the institution by establishing the strategic plans for technological innovation.
Environment The environment is the hodge-podge of others that have an interest in the outcomes of the eLearning endeavor (Jones, 2004). They encompass the various accreditation bodies that must certify the quality of the instruction offered. They are primarily concerned that minimum standards are met that all courses are comparable regardless of the modality of offering. While there are a number of accrediting bodies that focus on distance
16
learning/E-Learning, all accrediting bodies must certify some E-Learning offerings through institutional members. Another group that plays a major role in this process is the potential employer of the students in E-Learning programs. They want to ensure that students have received an adequate education irrespective of manner in which it was delivered. They want the education to provide content that is relevant to the work environment and that meets minimum quality standards. They also may become involved by supporting their current employees as they pursue further education to the benefit of the employer. In this manner they are also concerned with student access. The funders of education are an often overlooked stakeholder group. These are the entities that provide various aspects of the funding for educational institutions. They may be public funders in the form of government agencies or they could be private funders who provide grant funding or give to foundations that support the educational programs of an institution. They may be the parents who are funding their child’s education. Members of this group are primarily concerned that the funds will be utilized to provide a quality education to the students and ensure a respected outcome from their investment. All of these various stakeholders interact with one another seeking to have each other’s needs faced and resolved in the E-Learning effort. None of them stand isolated from the others and individually do not have a preeminent role to play in the eLearning offering. If one is ignored or their needs and desires are not met, it creates the potential for the eLearning effort to fail.
MODELS OF LEARNING MANAGEMENT SYSTEMS The various LMS models differ according to the selection of LMS elements (see Figure 5) and how they are applied to the system and therefore
General Perspective in Learning Management Systems
to the users of that system. While there are many elements that could be utilized to discriminate one LMS from another, we will focus on three. These are the primary elements that influence most decisions concerning the system of choice. The accompanying diagram identifies the three broad elements, but one must bear in mind that most systems are not just one or the other, but are in fact somewhere on a continuum between the extremes within each element.
Source In-House Initially all LMS were developed in-house and were dependent upon the support of the institutional programmers. Most of these systems possessed limited functionality and they were difficult to modify or improve. A few of those systems, like the PLATO LMS, were later transferred to businesses that have either used them in proprietary products or marketed them as proprietary systems. Most in-house developed systems, therefore, are no longer in existence, although there are some who are advocating a return to the in-house developed systems as a way to provide quality at a more affordable cost.
Proprietary Proprietary systems comprise the largest block of systems in operation today. While they all claim to provide the functionality needed by the institution, they are by no means identical. They differ on some very important factors and it would behoove the using institution to choose carefully. While they do differ on the basis of functionality, most provide a basic set of functions along the lines discussed in course management systems. They may or may-not interface with academic administrative systems or an online library system, for instant. They may include some Web 2.0 functionality, but it may be limited in utility. There is constant change within the industry, with companies coming and going by either leaving the industry or
being absorbed by one of the competitors. Some examples of these would be: Blackboard (which recently acquired WebCT Manager and Angel), Desire2Learn, ECollege, & .LRN. There are sites where one can compare these systems in order to decide which one is right for you. With these systems, you can either have a hosted environment, where some provider supplies the hardware and software to make the system work or you, or you can host it yourself on your own servers. There are many pros and cons to each approach and should be considered carefully.
Free/Open Source These systems are supported by a network of programmers around the globe who provide updates to the systems on an almost constant basis (Pan, G. & Bonk, C., 2007). Many of these systems are free to own, but are also often available through a supplier who will provide for the installation and support of the system. They can also be available as a hosted option. They provide a great deal of flexibility to the decision making process. They have traditionally been a niche market, but with Moodle gaining in popularity, they are gaining a larger market share. The basic premise of these systems is what is commonly known as Linus’s Law: Given enough eyeballs, all bugs are shallow (Pan, G. & Bonk, C., 2007). They assume a selfcorrecting mechanism in the peer review process that open-software engenders. This further pushes the costs of these products lower and encourages users to expand their functionality. The development of these products promotes common standards that establish those that are most beneficial to the market. This process also encourages the development of expertise in the broader market arena by encouraging many individuals to be involved in the development of the product.
17
General Perspective in Learning Management Systems
Figure 5. Elements of a learning management system
Time Time refers to whether the student and instructor are involved concurrently or separated by time, in other words, synchronous or asynchronous (Wagner, N., Hassanein, K., Head, M., 2008). Most LMS are designed to be operated in an asynchronous mode. In this mode, an instructor prepares the learning content and interacts with the students mostly on a one to one basis. The interaction does not occur in real (concurrent) time, but with time lapses. This creates the possibility of instructor-student disconnect as one has to wait for a period of time for the other to respond to the inputs. As the system supports a more dispersed student population, possibly spanning the globe, this becomes more of a necessity or least more
18
easily justified. However, there are systems that focus on a synchronous delivery mechanism, such as LaunchForce and LeanLine. These systems put the instructor and students online at the same time. These systems may involve, audio only, audio and video, whiteboard, program share, and other such technologies. In this mode, there is much more the virtual classroom experience with the ability to engender the collaborative involvement of all of the students and instructor in the learning exercises. Collaboration is possible in both synchronous and asynchronous modes, but is more natural in the synchronous environment. Some systems allow for both modes of delivery to be blended together. Such systems as ECollege, Embanet, Jones e-education, and LUVIT eLearning are thought of as total solutions. They
General Perspective in Learning Management Systems
allow delivery in the most effective time frame for the individual course. Many of these systems allow for the content that is delivered synchronously to be recorded and then delivered in an asynchronous fashion. There are tools that often can be utilized with other asynchronous systems to allow for the delivery of portions of the class in a synchronous mode.
Pedagogy Pedagogy is concerned with how a course will be delivered in instructional terms. It will affect the type of interaction that may be involved in the learning process, such as, student to content, student to instructor, and student to student (Hamuy, E. & Galaz, M., 2010) (Wang, Y. & Chen, N., 2009). In the student to content, the student interacts with instructional information, either provided through the course system or accessed by other means, textbook, online, etc. In the student to instructor mode, the student interacts with the content expert or experts to retrieve the needed learning content. In the student to student mode, the students will interact with one another to retrieve information or perform learning activities in a collaborative manner. Pedagogy will also determine the nature of the course structure(see Figure 6); will the course allow the student to move around and choose the order in which material is accessed and work is completed or must they move through in a predetermined orderly sequence (Dabbagh, 2005). Will the student be required to achieve mastery of the material to some predetermined level or will the student construct their own learning with the instructor serving as a mentor/guide? Pedagogy plays a fundamental role in the determination of the nature of the learning experience, whether the learning constructor recognizes the role or not. Pedagogy involves three broad aspects in its structure (Dabbagh, 2005). Each of these aspects plays some role in the impact of pedagogy on the learning process. Pedagogical constructs provides
the foundation out of which the others flow (Dabbagh, 2005). From the constructs will flow the pedagogical strategies that determine the teaching methods that are used in the program. From the strategies one will determine the pedagogical tools that will be need in instructional delivery. It is in this manner that pedagogy will determine the nature of the LMS used in a specific setting.
PEDAGOGICAL CONSTRUCTS These are the basic formulation of the teaching/ learning process (Dabbagh, 2005). They are the models or the components of the models that will guide the development and delivery of instruction (Sabry, K. & Barker, J., 2009). They arise out of one’s understanding of cognition and knowledge. They provide the mechanism for putting theory into practice (Dabbagh, 2005). Some major examples include: behaviorist, cognitivist, and constructivist. While these are not the only models used, most others grew out of them.
Behaviorist Behaviorist pedagogical systems will provide a structured/programmed approach to the teaching/ learning process. This pedagogical system sees the mind as a ‘black box’ that is not knowable and so is not a concern. No attention is paid to the internal processes of learning, in fact, many would say internal processes don’t exist, but this pedagogical system focuses on the measurable outcomes of learning. These outcomes therefore equal learning and rewards are the motivators that support the production of desirable outcomes. Learning is linear and structured. Theorists in this paradigm are Watson, Thorndike, Skinner, and Pavlov.
19
General Perspective in Learning Management Systems
Figure 6. Pedagogical structure
Cognitivist
Constructivist
The cognitivists focus on how the mind processes and uses information to produce learning. They are interested in the mental structures and processes that are necessary to explain human behavior (Dabbagh, 2005). They pay greater attention on the learner’s thoughts, beliefs, attitudes, and values in explaining the learning outcomes (Ardichvili, A. & Yoon, S., 2009). They are more focused on the learners’ differences and seek to accommodate them by varying the instruction material and processes. To them learning is an individualized process. Group activities are relatively unimportant and rarely addressed. They seek to provide organizational structure that allows the learner the ability to move through the material in a highly individualized fashion that ties the new learning to existing informational structures. Theorists in this paradigm are Gagne, Briggs, and Bruner.
Constructivists built off the cognitivistic view that the mind is more than a ‘black box’ responding to stimuli. It instead focused on the processes involved in learning, seeing those processes as internal and, therefore, not visible directly (Homberg, 2005). Constructivists see learning as an active process that works within a context to produce (construct) knowledge (Dabbagh, 2005) rather than just acquiring it. Learners must be actively involved in both acquiring and processing information subjectively (Baggaley, 2008). As learners actively process the information, they will gain a deeper understanding of the content (Ardichvili, A. & Yoon, S., 2009). The learner needs to be encouraged and enabled to search for the knowledge or solve problems on their own rather than provided the content or solutions (Ardichvili, A. & Yoon, S., 2009). All knowledge
20
General Perspective in Learning Management Systems
is seen as social in origin, developed while engaged in activities (projects), mediated by tools (Lytras, M. & Pouloudi, A., 2006). Instructors in this tradition will provide the learners with realworld simulations, collaborative experiences with others, or by providing them with the knowledge and ability to access knowledge at the time that it is needed (Ardichvili, A. & Yoon, S., 2009). Theorists in this paradigm are Dewey, Piaget, Vygotsky, and Bruner. These models/constructs are almost never used solely. They each will often find a place in some part of the E-Learning program. Most practitioners will have a preferred approach to the design and delivery of instruction, but will generally use some of the less preferred methods within their program to meet the needs of the widest number of students or to overcome areas of difficulty in the instructional program.
PEDAGOGICAL STRATEGIES The application of pedagogical constructs to the teaching process leads to three broad pedagogical strategic systems (Dabbagh, 2005). Each system seems to lean toward a specific set of educational methodologies/tools. There is an overarching perspective that guides the choices that are made by these educators in the development of the learning program. This produces the framework for E-Learning (see Figure 7).
Behavioral/Cognitive Strategy The Behavioral/Cognitive framework/strategy promotes facilitated learning. The teacher provides the guidance and direction that is needed for the student to learn. The primary educational methods/tools will be lectures, presentation, textbooks, and teacher directed discussions. The teacher is the source of information that leads to knowledge acquisition in this process. The process is very linear and focuses on measureable
outcomes. Practitioners may or may not pay any attention to individual differences or care much about the intellectual processes that are involved in developing the learning outcomes.
Constructivist Strategy The constructivist framework/strategy focuses on individualized learning. These practitioners see the process of learning as very interactive with the student interacting with the teacher and with the learning materials. The student plays the major role in the learning outcome. Students determine what is and is not important to learn. The teacher moves from the ‘sage on the stage to the guide on the side’ in this process. Learning activities and materials must be somewhat fluid in this process, because it is not possible to know what the student will need before the student needs it. The primary educational methods/tools used will be case studies, self-instructional materials, and questions and answers. The pace of learning will be dictated by the student and their desires.
Social Constructivist Strategy The social constructivist framework/strategy focuses on collaborative learning (Huang, S. & Yang, C., 2009). Learning is seen as a social endeavor and requires the collaboration between and among all participants. The teacher will very often be just one of the learners in this process. The group will determine the nature of the learning. Learning activities will vary with the class involved; the students will bring many of the materials from their own search for answers. The primary educational methods/tools used will be case studies, problem based learning, discussion forums, teamwork, and seminar sessions. The emphasis will be upon the group participation in the learning process with the group playing a major role in the decisions about what will transpire in the process. This is a very interactive strategy requiring consistent feedback from the teacher and fellow students. Learning is
21
General Perspective in Learning Management Systems
Figure 7. Pedagogical frameworks
seen as a process rather than an end. Control rests in both the teacher and students within a context of continuous communication.
PEDAGOGICAL TOOLS Your pedagogical strategy will determine the features, pedagogical tools, that your LMS will need to provide (Dabbagh, 2005) (Sabry, K. & Barker, J., 2009). If your strategy is a behavioral/cognitive framework, you can probably get by with a simple course management system that uses prepackaged course material to deliver instruction in a linear fashion with teacher graded outcomes measures. You will need a simple email communication
22
system to allow the students to ask questions of the teacher and allow the teacher to respond to those questions. You would also benefit from having a document delivery mechanism and a means for the students to submit their work and take exams. Most systems available today provide these features and more. If your pedagogical strategy is in a constructivist framework, you will need the entire aforementioned, but also some means for the student to navigate through the material individually and to set personal learning objectives. They will also need to be able to acquire their own learning materials to provide for their individualized learning plan. There will need to be flexibility in the assessment of learning outcomes to provide
General Perspective in Learning Management Systems
for the individualized nature of those outcomes. You will want to provide for simulations, blogs, message boards, etc. to allow the students to be able to construct their own learning activities and provide mechanisms for easy feedback from the teacher/coach. If your pedagogical strategy is a social constructivist framework, you will need everything already mentioned, but also include mechanisms for multi-way communication so that all participants can communicate with each other in a free and open manner. To be most effective, you will need to provide for both synchronous and asynchronous communication modalities. Students will need to work individually and in groups; as much as possible, the groups should be self selected.
FUTURE RESEACH DIRCTIONS Too much of research focuses on finding something new or defending a current stance on a subject. We need to have researchers who are willing to integrate the findings of others into some sort of a coherent system. This should involve the gathering of information from related disciplines so that we can improve learning management systems by using the knowledge from all appropriate sources. If we become too enmeshed in our limited view of the world, either by just looking at what is being produced in distance learning publications or by looking at only what is being done in our limited geographical sphere, we will not be able to meet the transformations that will occur in the future. We also need to focus on those aspects that will improve the quality of what we do.
CONCLUSION Learning management systems, while a modern phenomenon, have very deep roots in a broad array of disciplines. Each of these disciplines is still functioning today and produces impacts
upon LMS and their use within the educational environment. The nature of that impact is influenced by pedagogy. Pedagogy will drive what the LMS will have and how it will be used in the delivery of E-Learning. In most cases, teachers will not stick with one pedagogical framework, but will use aspects of all of them at some point in the delivery of learning. Students will benefit from this eclectic use of pedagogy. We have only scratched the surface of what is involved in the function of an LMS based on pedagogical decision making. The better the teacher understands the pedagogy underlying the decisions being made, the more effective the LMS will be in meeting those needs. Many LMS implementations fail because there was not a clear understanding of the pedagogical needs of the teachers using the system (Dillenbourg, 2008). This chapter has only lightly touched on the topics presented. It is not intended to be an exhaustive treatment of any of the subjects. It is hoped the readers will continue to pursue those topics of interest and to be aware of those of lesser interest. In this manner, we should all be able to understand the changes that have occurred and be prepared for those that will come.
REFERENCES Ardichvili, A., & Yoon, S. (2009). Designing integrative knowledge management systems: Theoretical considerations and practical applications. Advances in Developing Human Resources, 11(3), 307–319. doi:10.1177/1523422309337593 Baggaley, J. (2008). Where did didtance education go wrong? Distance Education, 29(1), 39–51. doi:10.1080/01587910802004837 Baturay, M. (2008). Analysis of a learning management system model. E-Journal of New World Sciences Academy, 3(3), 464–477.
23
General Perspective in Learning Management Systems
Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L. (1999). Design and delivery of integrated learning systems: Their impact on student achievement and attitudes. Journal of Educational Computing Research, 21(4), 475–486. Cradler, J. (2008). Research on e-learning. Learning and Leading with Technology, 3(5), 54–57. Dabbagh, N. (2005). Pedagogical models for e-learning: A theory-based design framework. International Journal of Technology in Teaching and Learning, 1(1), 25–44. Daniels, P. (2009). Course management systems and implications for practice. International Journal of Emerging Technologies & Society, 7(2), 97–108.
Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K. (1991). A comprehensive study of the effects of an integrated learning system. Indiana State University. Guri-Rosenblit. (2005). ‘Distance education’ and ‘e-learning’: Not the same thing. Higher Education, 49, 467–493. doi:10.1007/s10734004-0040-0 Hamuy, E., & Galaz, M. (2010). Information versus communication in course management system participation. Computers & Education, 54, 169–177. doi:10.1016/j.compedu.2009.08.001 Holmberg, B. (1995). Theory and practice of distance education, 2nd edition. London, UK, and New York, NY: Routledge.
Dillenbourg, P. (2008). Integrating technologies into educational ecosystems. Distance Education, 29(2), 127–140. doi:10.1080/01587910802154939
Holmberg, B., & Ortner, G. E. (1991). Research into distance education. Frankfurt am Main, Germany: Peter Lang.
Eteokleous-Grigoriou, N. (2009). Instilling a new learning, work and communication culture through systemically integrated technology in education. Systems Research and Behavioral Science, 26, 707–716. doi:10.1002/sres.983
Homberg, B. (2005). The Evolution, Principles and Practices of Distance Education (Vol. 11). Hagen, Germany: FernUniversität, ZIFF.
Gaeta, M., Orciuoli, F., & Titrovato, P. (2009). Advanced ontology management sytem for personalised e-learning. Knowledge-Based Systems, 22, 292–301. doi:10.1016/j.knosys.2009.01.006 Garrison, R. (2000). Theoretical challenges for distance education in the 21st century: A shift from structural to transactional issues. International Review of Research in Open and Distance Learning, 1(1), 1–17. Gaspay, A., Dardan, S., & Legorreta, L. (2008). Distance learning through the lens of learning models: New outlets for innovation. Review of Business Research, 8(4), 53–62.
24
Huang, S., & Yang, C. (2009). Designing a semantic bliki system to support different types of knowledge and adaptive learning. Computers & Education, 53, 701–712. doi:10.1016/j. compedu.2009.04.011 Jones, D. (2004). The conceptualisation of elearning: Lessons an implications. Studies in Learning, Evaluation, Innovation and Development, 1(1), 47–55. Keegan, D. (2002, November). The future of learning: From eLearning to mLearning. ZIFF Papiere 119. Hagen, Germany: FernUniversitat, Ziff. Kim, S., & Leet, M. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24, 284–294. doi:10.1111/j.1365-2729.2007.00260.x
General Perspective in Learning Management Systems
Lytras, M., & Pouloudi, A. (2006). Towards the development of a novel taxonomy of knowledge management systems from a learning perspective: An integrated approach to learning and knowledge infrastructures. Journal of Knowledge Management, 10 6), 64-80. Miller, T., & Hutchens, S. (2009). 21st century teaching technology: Best practicess and effectiveness in teaching psychology. International Journal of Instructional Media, 36 3), 255-262. Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating learning management systems: Adoption of hexagonal e-learning assessment model in higher education. Transforming Government: People, Process and Policy, 3(2), 11–130. Pan, G., & Bonk, C. (2007). The emergence of open-source software in North America. International Review of Research in Open and Distance Learning, 8(3), 1–17. Paulsen, M. (2002). Online education sytems in Scandinavian and Australian universities_A comparative study. International Review of Research in Open and Distance Learning, 3(2), 1–14.
Sabry, K., & Barker, J. (2009). Dynamic Interactive Learning Systems. Innovations in Education and Teaching International, 46(2), 185–197. doi:10.1080/14703290902843836 Taylor, J. (2001). 5th generation distance education. 20th ICDE World Conference on Open Learning and Distance Education (pp. 1-11). Dusseldorf, Germany. Underwood, J. (1997). Integrated learning systems: Where does the management take place? Education and Information Technologies, 2, 275–286. doi:10.1023/A:1018677616969 Wagner, N., Hassanein, K., & Head, M. (2008). Who is responsible for e-learning success in higher education? A stakeholder’s analysis. Journal of Educational Technology & Society, 11(3), 26–36. Wang, W., & Wang, C. (2009). An empirical study of instructor adoption of Web-based learning systems. Computers & Education, 53, 761–774. doi:10.1016/j.compedu.2009.02.021
Perry, B. (2009, June). Customized content at your fingertips. Training & Development, 29–31.
Wang, Y., & Chen, N. (2009). Criteria for evaluation synchonous learning management systems: Arguments from the distance language classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773
Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, J. (2008). Evaluating learning support systems usability an empiracle approach. Communication & Cognition, 41(1 & 2), 143–158.
Watson, W., & Watson, S. (2007). What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-007-0023-y
Rogers, L., & Newton, L. (2001). Integrated Learning Systems -- An ‘open’ approach. International Journal of Science Education, 23(4), 405–422.
ADDITIONAL READING
Rose, E. (2004). “Is there a class with this content?” WebCT and the limits of individualization. The Journal of Educational Thought, 38(1), 43–65. Saba, F. (2000). Research in distance education: A status report. International Review of Research in Open and Distance Learning, 1(1), 1–9.
Abel, R. humes, L., Mattson, L., McKell, M., Riley, K., & Smythe, C. (2007). Achieving Learning Impact 2007. August 2007. Retrieved from Http://www.imsglobal.org/learningimpact2007/ li2007report.cfm. Bates, A. (1995). Technology, Open Learning and Distance Education. London, UK: Routledge.
25
General Perspective in Learning Management Systems
Castells, M. (2000). The Information Age: Economy, Society and Culture: Vol. I. The rise of the Network Society. Oxford, UK: Blackwell. Cooper, P. (1993). Paradigm shifts in designed instruction: From behaviorism to cognitivism to constructivism. Educational Technology, (May): 12–19. Cristea, C. (2010). Education and media in the postmodern pedagogy. Petroleum – Gas University of Ploiesti Bulletin. Education Sciences Series, 62(1A), 102–106. Elias, T. (2010). Universal instructional design principles for Moodle. International Review of Research in Open and Distance Learning, 11(2), 110–124. Freishtat, R., & Sandlin, J. (2010).. . Educational Studies, 46, 503–523. Garrison, D. R. (1990). An analysis and evaluation of audio teleconferencing to facilitate education at a distance. American Journal of Distance Education, 4(3), 13–24. doi:10.1080/08923649009526713 Green, N., Edwards, H., Wolodko, B., Stewart, C., Brooks, M., & Littledyde, R. (2010). Reconceptualising higher education pedagogy in online learning. Distance Education, 31(3), 257–273. do i:10.1080/01587919.2010.513951 Gregor, S., & Benbasat, I. (1999). Explanations from intelligent systems: Theoretical foundations and implications for practice. Management Information Systems Quarterly, 23(4), 497–530. doi:10.2307/249487 Hinostroza, J., & Mellar, H. (2001). Pedagogy embedded in educational software design: Report of a case study. Computers & Education, 37(1), 27–40. doi:10.1016/S0360-1315(01)00032-X
26
Hulsmann, T. (2004). Guest editorial – Low cost distance education strategies: The use of appropriate information and communication technologies. International Review of Research in Open and Distance Learning, 5(1), 1–14. Keegan, D. (Ed.). (1993). Theoretical principlesof distance education. London, UK, and New York. NY: Routledge. Ley, T., Kump, B., & Albert, D. (2010). A methodology for eliciting, modellling, and evaluating expert knowledge for an adaptive work-integrated learning system. International Journal of HumanComputer Studies, 68, 185–208. doi:10.1016/j. ijhcs.2009.12.001 Lockwood, F. (1995). Open and distance learning today. London, UK, and New York. NY: Routledge. Mackenzie, O., & Christensen, E. L. (Eds.). (1971). The changing world of correspondence study. University Park, PA: Pennsylvania University Press. Moore, M. G. (2003). Network systems: The emerging organizational paradigm. [Editorial]. American Journal of Distance Education, 17(1), 1–5. doi:10.1207/S15389286AJDE1701_1 Moore, M. G., & Clarke, G. C. (Eds.). (1989). Readings in principles of distance education (pp. 29–37). University Park, PA: The American Center for the Study of Distance Education. Moore, M. G., & Kearsley, G. (2005). Distance education: A systems view. Belmont, CA: Wadsworth Publications. Ozkan, S., & Koseler, R. (2009). Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation. Computers & Education, 53, 1285–1296. doi:10.1016/j.compedu.2009.06.011
General Perspective in Learning Management Systems
Paulsen, M. (Ed.). (2003). Online education and learning management systems: Global e-learning in a Scandinavian perspective. Oslo, Norway: NKI Forlaget. Sellar, S. (2009). The responsible uncertainty of pedagogy. Discourse: Studies in the Cultural Politics of Education, 30(3), 347–460. doi:10.1080/01596300903037077 Sewart, D., Keegan, D., & Holmberg, B. (Eds.). (1983). Distance education: International perspectives. London, UK: Croom Helm (Routledge). Skinner, B. F. (1968). The technology of teaching. New York, NY: Appleton-Century-Crofts. Watson, D. M. (2001). Pedagogy before technology: Re-thinking the relationship between ICT and teaching. Education and Information Technologies, 6(4), 251–266. doi:10.1023/A:1012976702296 Zawacki-Richter, O., Backer, E., & Vogt, S. (2009). Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning, 10 6), 21-50.
KEY TERMS AND DEFINITIONS Behaviorism: The theory or doctrine that human or animal psychology can be accurately studied only through the examination and analysis of objectively observable and quantifiable behavioral events, in contrast with subjective mental states. Blended Learning: The use of both classroom teaching and on-line learning in education.
Cognitivism: A theoretical approach in understanding the mind using quantitative, positivist and scientific methods that describes mental functions as information processing models. Constructivism: A theory of knowledge (epistemology) that argues that humans generate knowledge and meaning from an interaction between their experiences and their ideas. E-Learning: Learning that is done at a distance using information communication technologies for delivery. Information Communication Technology: Consists of all technical means used to handle information and aid communication, including computer and network hardware as well as necessary software. Instructional Television Fixed Service: A band of twenty (20) microwave channels available to be licensed by the U.S. Federal Communications Commission (FCC) to local credit granting educational institutions. Pedagogy: Refers to strategies of instruction, or a style of instruction. Programmed Learning: A learning methodology or technique first proposed by the behaviorist B. F. Skinner in 1958. It has three elements: (1) it delivers information in small bites, (2) it is self-paced by the learner, and (3) it provides immediate feedback, both positive and negative, to the learner. Video Conferencing: A set of interactive telecommunication technologies which allow two or more locations to interact via two-way video and audio transmissions simultaneously.
27
28
Chapter 2
Knowledge Sharing in a Learning Management System Environment Using Social Awareness Ray M. Kekwaletswe Tshwane University of Technology, South Africa
ABSTRACT The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness.
INTRODUCTION This chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms, as
learners share knowledge. The chapter argues that to be able to design LMS that ought to enable social interaction among learners not confined to particular places, we must first understand how learners interact and the tools they use. In this chapter, learners use social awareness to determine
DOI: 10.4018/978-1-60960-884-2.ch002
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Knowledge is not a fixed commodity, but a function of our interactions with external resources including tools, media, and other humans (Ryder & Wilson, 1997). This suggests that human knowledge transforms as people socially interact with others and the surrounding environment. Consequently, the chapter is premised on the notion that knowledge is created and transferred through the dynamic interactions among individuals and between individuals and their environments (Nonaka, 1994). Thus, knowledge sharing is social and sensitive to context. It is inferred from the notion of knowledge creation, sharing or transfer that knowledge can be perceived to transform when context and social presence awareness interact. In this regard, context and presence awareness influences the interaction and the problems that could be solved and how they are solved. In this chapter, social awareness is synonymous with context and social presence awareness. Although a great number of studies (e.g., Shariq, 1999; Polanyi, 1966; 1958) have shown that knowledge creation and transfer is essentially a human-to-human process or an outcome of social interaction (Nonaka, 1994), the relationships or roles of context and social presence awareness as catalysts for knowledge sharing and transformation in a learning environment has not been explored. This chapter aims to contribute to that effect. This chapter is about exploring and understanding how, through a learning management system environment, a learner uses social awareness to leverage personalized knowledge sharing. It reveals the actual nature of ubiquitous learning through social interaction where awareness of context and social presence is argued to be the underlying process of the activity. The chapter is on how varied forms of communication for knowledge sharing in an LMS learning environment are an outcome of social interaction coordinated by social awareness. Social awareness is synonymous
with awareness of context and social presence. Context is understood as the situation in which a learner or a group of learners find themselves. Accordingly, context is defined as any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application (Dey and Abowd, 1999). Social presence is re-defined and understood to be the mediated presence of another learner who could provide personalized on-demand knowledge support for a learning problem as the learner traverses varied learning contexts. Learners in contact universities come from varied social backgrounds, with diverse languages and cultures. To this point, one of the prevailing educational challenges is that of providing personalized academic support to under-prepared learners (Jaffer et al, 2006). Awareness of the social environment and social resources is, therefore, fundamental to the provision of personalized academic support to a learner. Learning is made ease when a learner has consistent awareness of context and presence of social resources (Kekwaletswe, 2009; 2007). Ubiquitous personalized knowledge support refers to the provision of context sensitive and anywhere, anytime help as learners traverse varied locations. Learners use awareness of context and social presence as a means to access ubiquitous learning support, interpret and adjust their knowledge – sharing what they know with others through social interaction. The chapter, thus, focuses on the peer-to-peer interaction and the learning environment. The social interaction whose outcome is transformed knowledge and provision of support is location and time independent. Since sharing learning experiences is a ubiquitous phenomenon, learners continuously use awareness of the environment and awareness of available social resources they can draw upon to facilitate knowledge consultation (Kekwaletswe, 2009: 2007). This chapter is about the advancement of the human-centric approach to knowledge creation
29
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
and sharing through enhanced person-to-person interaction – where context and social presence awareness is of vital significance to how learners create, use, and share knowledge. The chapter uses the contextual inquiry research method to understand how a learner goes about sharing knowledge, and the tools they use. The practical contribution of the chapter is the understanding of learning management systems environments and how learners use social awareness to model their actions for the provision of personalized academic support. The rest of the chapter is as follows: firstly, the background to the research problem is given; secondly, the theoretical foundations for the chapter are articulated followed by the research methodology; lastly, the chapter discusses, with examples of empirical evidence, how social awareness is used to leverage knowledge sharing in the learning environment. The chapter is then concluded.
BACKGROUND TO THE RESEARCH PROBLEM The practical relevance of the chapter is towards enriching personalized on-demand academic support through social awareness. To this point, this chapter focuses on LMS communication tools appropriated by awareness of learners’ context. The ideal social awareness for knowledge sharing is one that is sensitive to the background of a learner (social context includes culture and language), arranging these aspects to provide immediacy on the available social network, independent of the learner’s location and task at hand (Kekwaletswe and Ng’ambi, 2006). The focal point of the chapter is therefore how social awareness is used to support personalized social interaction for a South African university learner as s/he traverses varied learning contexts. The objective of this chapter is not to confirm previously established premises and theories but to find out, through engagement with learners, how they, as they traverse varied learning
30
locations, interpret and use social awareness for interaction whose outcome is knowledge sharing. I argue that the effectiveness of a knowledge sharing within a learning management system is fundamentally affected by the social interaction and the context in which a problem-driven learning activity takes place. In the chapter, learning through a learning management system environment is defined as any sort of learning and knowledge sharing that happens due to social awareness when the learner is not at a fixed, predetermined location – in varied learning contexts. An LMS learning environment conveys ubiquitous learning that is not confined to specific locations, and is time independent. Context aware ubiquitous learning is meant to support learning by identifying a learner’s surrounding contextual environment and social presence to provide a rounded and seamless learning experiences. In a contact university, there tend to be disproportionate access to available social resources between when learners are attending scheduled or formal classes and when they are away from scheduled classes (Kekwaletswe and Ng’ambi, 2006) especially as they move away from campus locations. The challenge for universities is that instructors and tutors are not always available to provide learners with ubiquitous support as learners move away from formal locations or contexts. The alternative for these learners is to consult with a knowledgeable peer who, for the most part, shares a background. There are three types of contexts within which a learner is mobile and for which a learner needs a learning management system and social awareness as a medium to transform knowledge. The following three learning contexts have been discussed by Kekwaletswe and Ng’ambi (2006).
Formal Learning Contexts These contexts represent formal structures – such as scheduled lectures and laboratory sessions – in which a learner’s behaviour and action is shaped according to the university class timetable. Inter-
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
action in these spaces is usually one-way from instructor to learner using English as the official language of instruction, although learners often interact with each other using their own diverse languages. In the formal learning context, the instructor delivers a lecture and a learner either takes notes or is given a handout. Even though learners are invited to ask questions, there is little time to assimilate the material and meaningfully engage with the learning materials (Ng’ambi, 2004). Learning and knowledge seeking action is, therefore, mostly passive. In this context, social presence is usually availed through wired PCs in the computer labs and the face-to-face presence of tutors, instructors and peers.
Semi-Formal Learning Contexts These contexts represent informal spaces on campus used by learners, usually while waiting for the next lecture to start or after it finishes. They include the library, cafeteria, mingling areas and walk-in laboratories. As learners begin to reflect on the previous lecture and skim through the learning materials, questions begin to arise for which clarifications are required (Ng’ambi and Hardman, 2004). The challenge a learner is faced with is how to find an instructor or tutor who is available for immediate or on-demand consultation. Most instructors schedule consultations and are often unavailable for ad hoc interactions. The dilemma for most learners is that the consultation periods are limited and not always suitable (Ng’ambi, 2004). Consequently, the learner’s alternative is to find the nearest socially present knowledgeable peer or class-mate who can provide social support (Kekwaletswe and Ng’ambi, 2006). In the semi-formal learning context social presence is still availed through wired PCs in the computer labs and face-to-face presence of peers. Since the environment is on campus, where most learners meet and come for formal classes, a learner is still very much aware of the available social network.
Informal Learning Contexts Although it is difficult to be explicit on the characteristics of an informal learning context, these contexts include working during after-hours or weekends at university residences or private homes. In these environments, a learner usually uses his or her mother tongue to consult with peers (Kekwaletswe, 2006) or may write down questions to ask the instructor when they do get into the formal context (Ng’ambi, 2004). There are three things that must be known to provide ubiquitous personalized learning and knowledge support to a mobile learner in an informal learning environment: a) Knowledge about the location of a learner so as to help identify the potential knowledgeable peer; b) the preferred language of a learner in which he or she is likely to be conversant, and c) the awareness of a peer’s social presence and contexts – including location and situation (Kekwaletswe, 2006).
The Research Problem One of the prevailing educational challenges in the new South Africa is that of providing personalized academic support to under-prepared learners (Jaffer et al., 2006). In a contact university (as opposed to distance learning university), where students attend formal lectures and scheduled laboratory sessions, there tends to be inconsistency in social presence or access to available social networks for academic support between when learners are on campus and off campus (Kekwaletswe, 2006). Based on empirical evidence, I argue that in a learning environment that has a learner population with diverse social backgrounds and languages, social awareness of peers with a shared background is fundamental in the provision of personalized academic and social support. I distinguish between a learner being on campus in a formal context and being on campus or off campus in informal contexts.
31
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
Since knowledge sharing and learning tasks are not confined to particular locations but are carried across different learning contexts, I argue that learning and social resources available to learners ought to move with a learner. I use the term social resources pragmatically to mean knowledgeable peers, tutors, instructors and the awareness of context and presence (Kekwaletswe, 2006; Kekwaletswe and Ng’ambi, 2006). In this regard, the problem is that of ensuring that the quality of resources available to learners remains consistent for supporting a knowledge transforming task or action regardless of time and location of a learner (Kekwaletswe, 2006). In other words, learners should have a consistent social awareness of presence and context. The argument is that when learners solve and engage in a learning task – problem based learning – they bring prior knowledge and experience to the social interaction situation, where the knowledge transformation outcome is influenced by social awareness. Thus, learning management systems ought to leverage this. But first, we need to understand how LMS communication tools are and could be used. In conventional interaction with the learning activity, learners have an impoverished mechanism for providing social awareness of the available social network. Consequently, personalized social support is denied when learners do not have the opportunity to access a consistent social network (ibid.). In order to understand social presence and context awareness and their role in learning system environments, we must understand both what the context is and how social awareness can be used to facilitate the ubiquitous learning. An understanding of the context and a sense of social presence enables the learner to model behavior along the expectations or the shared understanding of a social learning community.
32
THEORETICAL FOUNDATIONS This section lays out the epistemological and ontological foundations of the inquiry and reviews the existing literature that informs the chapter. The inquiry is about learning management system environments that support learning and sharing of knowledge, with context and social presence awareness as catalysts for learning actions. The theory of knowledge cannot be divorced from the situations under which knowledge and learning experiences take place and this is reviewed next, followed by a specific focus on learning environments. Social presence and context for interaction in such environments also receive special attention in this section.
Knowledge To understand knowledge and how it is shared in a learning environment as an outcome of social interaction, I unpack the logical development of how knowledge is created, retained, and used. The construction of knowledge requires processing of data into information, new information is then created which is then communicated or transferred outside of the human brain. With that premise, I briefly define data as raw facts that can be shaped and formed to create information. Thus information is data that is given a meaning within a context. Therefore, only data and information can be captured, transferred or stored outside the brain. Knowledge is created from the processing of information, and during this processing, new knowledge can be acquired or created for future use, when more or new information is acquired and processed (Van Beveren, 2002). Knowledge is transformed into information within the brain to be communicated externally through language or demonstration (ibid.. 2002). Language may include different forms of communication such as text, verbal and body gestures. What then constitutes knowledge?
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
There are probably as many definitions and explanations of knowledge as there are theories of knowledge from diverse disciplinary perspectives. Without joining the philosophical debate of what exactly knowledge is, it is encouraging to notice that literature has adopted a pluralistic epistemology, acknowledging that there are many types or forms of human knowledge. The following are only some of the definitions of knowledge. •
•
•
•
Knowledge is an individual’s stock of information, skills, experience, beliefs and memories (Alexander et al., 1991). Knowledge is the stock of conceptual tools and categories used by humans to create, collect and share information (Laudon and Laudon, 1995) Goffman (1978) explains knowledge by asserting that the “making of knowledge is a way of enacting reality, giving existence to things and events, and organizing the world.” Nonaka (1994) deems knowledge to be “justified true beliefs”. The theory of knowledge creation sees knowledge as a dynamic human process of justifying personal beliefs as part of an aspiration for the “truth”.
Generally, scholars of knowledge observe knowledge in two ways: “know how” and “knowthat”. The former is created ‘here and now’ in a specific, practical context and conveyed through analogies and metaphors; the latter is contained in manuals and procedures and oriented towards a context-free theory (Patriotta, 2003).
Knowledge as a Multifaceted Phenomenon The evolution of institutional knowledge has been informed by a wide spectrum of theoretical traditions. Knowledge is a multifaceted phenomenon which has been debated in a variety of disciplinary contexts – from philosophy and sociology, to
social psychology and cognitive science; from economics to management and organization analysis. The breadth and depth of the subject would not allow me to trace a lineage of existing knowledge theories. Nevertheless, cognitive theories have looked at knowledge as a representational phenomenon. Winograd and Flores (1986) point out that “a cognitive being ‘gathers information’ about things and builds up a ‘mental model’ which will be in some respects correct (a faithful representation of reality) and in other respects incorrect. Knowledge is a storehouse of representations, which can be called upon for use in reasoning and which can be translated into language while thinking is a process of manipulating representations” (op. cit., p73). In cognition-action theory, the foundational hypothesis is that action always possesses a cognitive basis which is reflected in the representational activities of the mind. The duality of cognition and action underlies the conceptualization of knowing as a computational activity (Patriotta, 2003). Since human behaviour is always oriented towards a goal, action is a form of problem-solving, where the actor’s problem is to find a path from some initial state to a desired goal state, given certain situations along the way. Accordingly, there is a need for researchers to understand how learners acquire new problem representations for dealing with new problems. A general theme uniting many situated approaches to cognition is a change in the way the personenvironment relationship is envisaged. Rather than a person ‘being’ in an environment, the activities of person and environment are parts of a mutually constructed whole. The inside-outside relationship between person and environment is replaced by a part-whole relationship (Bredo, 1994). Learning is centered around problem-solving and is intricately related to the context; ‘context’ here means understanding (a) the problem’s conceptual structure as well as (b) the purpose of the activity and (c) the social milieu in which it is embedded (Scribner, 1987).
33
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
The above briefly highlighted the multifaceted views on learning and knowledge and how it is created. The highlight is what informs my own definition of knowledge sharing. Having given an overview of what knowledge is and how it may be created, the following section will highlight the role of social awareness in transformational learning. In order to understand social presence and context awareness and the degree of their role in a learning management system environment, we must understand both what the context is and how the social awareness can be used to facilitate the ubiquitous learning.
Social Presence Theory In this section, the concept and Theory of Social Presence is discussed. As I examined the social presence studies and literature, it became apparent that most, if not all, of the studies on the concept employ a positivist approach. Whilst this chapter employed an interpretive approach, it is essential that I highlight the extent to which the theory and concept has been explored in previous research – which in turn informed the redefinition of the concept as used in this chapter. Short, Williams and Christie (1976) asserted that different communication media express varying degrees of social presence based on their ability to transmit nonverbal and vocal information. Thus, they initially introduced the Social Presence Theory as “technical social presence,” defining it as the capacity of the medium itself to present the “salience of the other person in interpersonal interaction” (p65). Two concepts associated with social presence are “intimacy” (Argyle and Dean, 1965) and the concept of “immediacy” (Wiener and Mehrabian, 1968). Intimacy depends on factors such as physical distance, eye contact, facial expression and personal topics of conversation. Immediacy is a measure of the psychological distance which a communicator puts between himself and the object of his communication.
34
This notion of Short et al. (1976) was, however, questioned by communication researchers (Gunawardena & Zittle, 1997; Byam,1995; Walther, 1994) who showed that perceived social presence in mediated interactions varies among participants in the same mediated conversations. That is, many of their research participants perceived mediated discourse as more personal than traditional classroom discussion. The claim by Short et al. (1976) that the quality of the communication media determines its social presence or richness was also disputed by Ngwenyama and Lee (1997) who showed that the communication richness of a media is dependent on who uses the media and how they use it. Gunawardena and Zittle (1997), for example, defined social presence as “the degree to which a person is perceived as ‘real’ in mediated communication” (op. cit., p8). They, like Ngwenyama and Lee, also argued that social presence was as much a matter of individual perceptions as an objective quality of the medium. In a survey to measure students’ perceptions of the social presence of others in a computer conference, Gunawardena and Zittle found that perceived social presence predicted more than half of the variance in students’ satisfaction with the conference. Their results also indicated that students who felt a higher sense of social presence enhanced their computer communication using emoticons to express missing nonverbal cues in textual form. Rourke, Anderson, Garrison and Archer (2001) regard social presence as one of the three fundamental “presences” that support learning, the other two being cognitive presence and teaching presence. Thus, they define social presence as “the ability of learners to project themselves socially and affectively into a community of inquiry” (p50). Rourke et al. identified three categories of social presence indicators – affective experience, cohesive experiences, and interactive experiences – and explored their use in online discussion. Affective experience contain personal expressions of emotion, feelings, beliefs, and values; Cohesive
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
experience are communication behaviours that build and sustain a sense of group commitment, such as greetings and salutations and group or personal reference; Interactive experience are behaviours that provide evidence that others are attending, such as agreement/disagreement, approval and referencing previous messages. There is, evidently, a lack of social presence research in learning management systems environment. Most studies of social presence have focused on the nature of online discussion and accordingly conceptualized social presence as a single construct with an emphasis on perceptions of the presence of peers (e.g., Swan, 2003 & 2002). As noted by Richardson and Swan (2003), there is some indication that instructor social presence may be equally important. Social presence of instructors has been considered in explorations of “teaching presence” (Shea et al., 2003; Anderson et al., 2001). Researchers have demonstrated both that students perceive the presence of others (Picciano, 2002; Gunawardena & Zittle, 1997; Gunawardena, 1995) and that they socially present themselves (Swan, 2003 & 2002; Rourke et al., 2001) in online course discussions. Nevertheless, there is lack of studies – positivist or interpretivist – looking at learners perceiving the presence of others and socially presenting themselves in a learning environment or context. This chapter addresses this shortcoming and it does so by following an interpretive tradition, diverting from the positivist tradition. In the chapter learners use awareness of a social presence for purposes of social interaction whose outcome is knowledge sharing. In this chapter, social presence is defined and understood to be the mediated presence of another learner who could provide personalized on-demand social support for a learning problem as the learner traverses varied learning contexts. Context and context awareness are also fundamental concepts in a learning environment where a learner is not fixed to particular locations. The concepts of context and context awareness are discussed in the next section, in view of the fact that they form the second phenomenon of the chapter.
Context Awareness Context and context-awareness are fundamental concepts in a learning environment where a learner is not fixed to particular locations. The following discussion of context and context awareness studies and literature is intended to show how the concepts are relevant to learning and ubiquitous social interaction. Lonsdale et al. (2003) describe context as a set of changing relationships that may be shaped by the history of those relationships. The figure below gives their hierarchical description of context as a dynamic process with historic dependencies. In Figure 1, a snapshot of a particular point in the ongoing context process can be captured in a context state. A context state contains all of the elements currently present within the ongoing context process that are relevant to a particular learning focus, such as the learner’s current project, or a learning activity. A context substate is the set of those elements from the context state that are directly relevant to the current learning and application focus, that is to say, those things that are useful and usable for the current learning system. Context features are the individual elements found within a context sub-state. Each feature is atomic and refers to one specific item of information about the learner or his/her setting. In implementing context awareness within their architecture, Lonsdale et al. (2003) derive a context sub-state and use the context features contained within it to determine what content might be appropriate for a learner. Context related to the human environment is structured into three categories (Schmidt et al., 1999): •
•
Information on the user; knowledge of habits, emotional state, bio-physiological situations. The user’s social environment; co-location of others, social interaction, group dynamics.
35
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
Figure 1. Context hierarchy (Lonsdale et al., 2003)
•
The user’s tasks; spontaneous activity, engaged tasks, general goals.
Context related to the physical environment is also structured into three categories: • •
•
Location; absolute position, relative position, co-location. Infrastructure; surrounding resources for computation, communication, task performance. Physical situations; noise, light, pressure (Schmidt et al., 1999).
An entity needs contextual information to choose between alternative strategies in order to reach its objectives, or to do useful work. Context gives hints about what is or what is not achievable (Rakotonirainy et al., 2000). If the learner’s initial knowledge sharing or transfer objectives are not reachable then they are changed to suit the current context, i.e., the objectives change or actions are taken according to the context to maintain an objective. Abstract context awareness then
36
means being aware of what you can know about your context while concrete context awareness means being aware of what you do know about your context (op. cit.).
RESEARCH METHODOLOGY Empirical Inquiry in Formal Learning Contexts Interactions of learners in formal learning contexts at the University of Cape Town – as a contact university – are usually passive (Ng’ambi, 2004). That is, learners who are mobile hardly need to actively interact with each other in a formal lecture. The assumption is that it is mostly outside the formal contexts that university learners, notably at UCT, begin to interpret and engage effectively with learning materials and therefore need social presence and context awareness about available peers for social support. Since awareness of context and social presence is not much of a need for learners in formal environments, the
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
empirical inquiry does not focus on formal learning contexts. The empirical evidence suggests that social interaction of learners, where they begin to simplify meanings and begin to apply what they have learned in class, happens mostly outside of the formal learning context. The argument is that South African learners, who traverse varied locations, need more personalized academic support through social awareness as they move away from formal learning contexts. In view of this, the empirical study and evidence does not include learning management system environments in formal learning contexts but focuses on semiformal and informal learning contexts. The empirical evidence was gathered at the University of Cape Town campus and residences using contextual inquiry methodology. Contextual inquiry is a field research framework that depends on conversations with users in the context of their work (Holtzblatt and Jones 1994). It is based on ethnography, where the researcher goes into the research participant’s own environment. It is an explicit step for understanding who the user really is and how the work progresses from day
to day (Beyer and Holtzblatt, 1998). Essentially, contextual inquiry consisted of observing learners’ actions and talking with learners in their learning environment while they were engaged in authentic learning tasks. The focus of the contextual interviews and textual interactions was on the learners’ mundane activities, notably problem-driven social interactions related to learning. In Figure 2, learners use social awareness to interact with others who are not in the same location using Web-based mediated communication to send and read emails or chat through instant messaging. Learners interacted with learning resources, posing or responding to questions via a Web-based learning management system. In the figure, learners also use mobile devices such as Portable Digital Assistants (PDAs) to interact via mobile instant messaging, and mobile phones that could be carried around to interact verbally or send short text messages (SMSs). The study began with sixty learners completing a qualitative questionnaire meant to understand communication tools, forms of interaction, social presence and context awareness. The initial sixty
Figure 2. Representation of the research framework
37
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
learners were randomly identified as they traversed the university campus and in libraries and laboratories. There were no specific criteria for selection other than that the mobile learners had to represent or give a true reflection of typical learners at the University of Cape Town. Twenty of the sixty were asked to take part in the Web-based LMS and PDA pilot based on their willingness and commitment to be available for the duration of the study. Since the study was on understanding how university learners communicate, the tools they use and how they exhibit awareness, it was not necessary to determine who should participate or not participate in the study, other than that they should be registered university learners. The WiFi-enabled PDAs were to facilitate the “anywhere anytime” online learning interactions and the opportunistic mobile instant messaging. Eight of the twenty learners were interviewed and observed as they interacted with others in authentic LMS learning environments. Although this study was not on a particular LMS platform, participants also used Vula, a university SAKAI-based online learning environment. The textual interactions were in the form of chats and instant messages. The contextual verbal interviews were recorded using a digital recorder and transcribed. The storylines were in the form of thick descriptions including all the environmental details that could be observed and documented. Pictures capturing the actions and environment situations were constantly taken. The focus of the inquiry was on how learners interact, the tools they use and how awareness is used to support personalized learning. In the next section, the empirical evidence is discussed in terms of how learners interact, the appropriateness of the tools they use and how social awareness manifest in a learning environment. It is worth noting that the aim of the chapter was not to study a specific LMS platform but to study how knowledge is shared through awareness of context and social presence – awareness determines learning actions and the appropriateness of LMS tools. 38
KNOWLEDGE SHARING IN A LEARNING MANAGEMENT SYSTEM ENVIRONMENT USING SOCIAL AWARENESS Socio-cultural theories of mediated learning suggest that what is learned will emerge from the relationship between human action, and the social, cultural, institutional and historical contexts in which action occurs. This makes it essential for us to understand these contexts and activities before we begin to investigate issues of learning (Sutherland et al., 2000). In view of what Sutherland et al. (2000) suggests, it is first important for us to understand how social awareness manifest in a learning environment if we were to design appropriate learning management systems that could leverage knowledge sharing and learning actions. This section discusses how social awareness manifest amongst learners sharing learning experiences. In earlier sections, I noted that one of the prevailing challenges in the new South African higher education is that of providing personalized support to under-prepared learners – who by the very nature of the South African population come from diverse backgrounds and cultures. In most African cultures, it is uncommon for younger people to interact with or question their elders. This phenomenon tends to apply in an educational environment where learners are not comfortable questioning the instructors. The alternative for these learners is then to consult and interact with their close friends and knowledgeable peers to provide academic support. For this reason the inquiry focused on social presence of peers and not “teaching presence”. Consequently, this chapter is about interactions amongst learners (peers) for purposes of knowledge sharing, as they traverse varied learning locations. However, it is worth noting that in South Africa, availability and access to wireless and wired computers, remains a challenge. These resources become even more inadequate as you move away from campus. The
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
study was therefore guided by the three localized learning contexts discussed in section two, which apply to a typical South African contact university. The framework depicts social awareness and presence in three different learning contexts of the mobile learning environment. Social awareness is a mental concept where a peer and a learner become aware of the social network that follows them as they move across the varied learning contexts In Figure 3, a learner is consciously aware of available knowledgeable peers should s/he encounter a learning problem for which s/he needs to consult. By the same token, a peer is consciously aware of the presence of other learners should s/he not be able to address a problem encountered by a learner. That is, a learner and a knowledgeable peer have a consistent social awareness of a social network (social resources) regardless of their location and context. Even though learners interact via Web-based environments, in this study, the social awareness and availability of social resources in
remote locations was mostly mediated by mobile technologies, e.g., mobile phones and PDAs. Knowledge sharing involves two actions: transmission (sending or presenting knowledge to a potential learner) and absorption by the audience (Davenport and Prusak, 1998). If knowledge is not absorbed, it has not been shared. In other words, merely making knowledge available is not sharing. In this regard, social interaction achieved and enhanced by awareness of context and presence is necessary. The research investigated how learners use social awareness – social presence and context awareness – to enable a knowledge sharing interaction. It sought to understand ubiquitous learning where learners support others as they traverse the three learning contexts. Communication or interaction that is mediated by technology is generally grouped into two categories: asynchronous and synchronous. Asynchronous communication occurs between learners independent of time and location. This kind of communication does not need the send-
Figure 3. A framework for mobile learner-to-learner environment (Kekwaletswe, 2006)
39
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
ing and the receiving learners to be “available” concurrently. Examples include leaving a phone voicemail, posting to or reading a discussion board, and sending and receiving email (although this could also be considered synchronous). On the other hand, synchronous mediated interaction is considered a “real-time” experience between two or more learners. Examples of tools that facilitate synchronous communication include telephones, audio-video conferencing software, instant messaging, virtual chat, virtual classrooms, and whiteboards. Asynchronous and synchronous mediatedcommunication can be used in individual or group learning situations, as well as traditional or online learning environments. Although a significant body of research validates the notion that learning is a social act, learners may still acquire knowledge in a mediated interaction. There are several forms of technology mediated communication, such as electronic mail, web-based consultation environments, instant messaging on wired networks and short message service (SMS) on mobile phones. Although they support mobility of a learner and may afford presence awareness, they do not do so in real-time. In the study, there was a need to select a mediating technology that supports mobility of a learner as well as presence awareness of available peers in real-time. Thus, mobile instant messenger (IM) Jabber client for Pocket PCs (PDAs) called iMov messenger (www.jabber.org) was selected. The mobile instant messaging on PDAs supports an immediate on demand formal or informal expressive interaction. Such real-time interaction could be one-to-one or several concurrent dyadic conversations. Mobile IM provides mobile learners with a real-time interactive space to share learning experiences – exchanging textual messages that do not require detailed email-like messages or face-to-face interactions. Dey and Abowd (1999) suggest that contextaware applications look at the who’s, where’s, when’s and what’s (that is, what the user is doing) of entities and use this information to determine
40
why the situation is occurring. In this chapter, the context-aware learning interaction environment supports (who) a learner as s/he engages with the (what) learning materials, (where) whether in semi formal or informal learning contexts (when) anytime through the use of context awareness and social presence mechanisms. In this chapter, social awareness, and not a learning management system, does actually determine why a learning situation is occurring.
Social Awareness in a Learning Environment In semi and informal contexts, learners engaged in a learning activity are able to use implicit and explicit context awareness to increase or decrease the interaction whose outcome is learners “helping” or “supporting” each other. The notion of social interaction presupposes an existence of two or more learners speaking or acting. And also, embedded in this notion is the assumption that a learning community is socially present without which interaction – “help” or “support” – is impossible. The following tables give examples of how learners use implicit and explicit context and social presence awareness to increase or decrease social interaction whose outcome is personalized learning. To help the reader understand how awareness manifests, the examples are structured as follows: what the researcher wanted to know, the learner’s experience and a brief discussion. The experiences are grouped in a table form per research theme and query. Showing the empirical evidence this way was to allow the reader to see where the low level research questions (elements of awareness) that support the main question are addressed. Table 1 shows how learners are conscious or aware of the situation and the context in which they are interacting and sharing experiences with others, e.g., how fast they would need the response, the content of interaction, etc. It also highlights how presence and availability help the decision
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
regarding how to communicate. The different mediating tools offer different opportunities and accessibility. The preceding tables showed just some of the examples of how awareness of context and social presence manifest in a learning environment. In a learning environment, the central activity is knowledge where peers provide personalized social support through social interaction. Learners
acquire knowledge and experience through social interaction, drawing upon available social resources for the solution of practical learning problems. The empirical evidence showed that social awareness of the social resources, the learning task, and the environment does influence the action a learner undertakes – hence the appropriateness of an LMS communication tool.
Table 1. How does a learner share knowledge and learning experiences with others? Researcher’s Intention
Different Learners’ Response / experiences
Researcher’s Comments
How does a learner usually communicate with peers, besides face-to-face meetings, when faced with a learning problem and/or needing help?
…through SMS, …I call or email, …I use my mobile phone, …we do online chat, …I facebook sometimes …mobile phone chat (viamiXit). NB: MiXit is a mobile phone service that allows friends to chat (send/receive text) instantly and in real-time.
Here the learner’s intention is to seek help or share knowledge about a learning problem. He or she picks up an appropriate tool that best mediates the thought.
What is a learner’s reason for communicating using a particular tool?
…Depends on how desperate I am, if desperate I phone, if not I SMS. ...Depends on what I want to say …It depends on the situation at the time …Depends on the seriousness of the situation …Availability of funds …Economic limitations and time available …It depends on where I am at the time that I need to contact them.
The experiences show evidence of the learners’ situation as awareness of context. Location, cost, need and reason to interact are taken as context awareness which influences how learners consult each other, e.g., calling will give immediate response.
How does a learner decide which method of interaction to use?
…With email, it.s efficient because I can phrase the issues better and they can review and reply with specific and thorough answers. …It depends on the issues I need to discuss or message I need to get across. ...How they will respond. …With SMS and calls, the response is faster and I can get help I need in time. …It is the fastest way to get feedback from them. …Whichever will reach the fastest in terms of being seen by the person, and the immediate need. …It.s quick and direct (Mobile phone calls). …They are always with them (peers always carry their mobile phones) ...I know they will always have their cell ...Depends on if they are available, and what will be quicker at that moment. ..If they are available (via mobile phones)
Learners use content of the interaction as context for which a mediating tool is chosen. Learners decide on the method of interaction with peers based on how quickly and fast they need the response. Where a short message is enough for the interaction, they text each other. Issues of time are noted as an influencing factor in picking a mediating tool, noting that most interactions are done on-demand to solve a learning problem instantly. Learners use social presence awareness as an influencing factor for choosing a specific mediating tool.
41
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
Table 2. How does location as context awareness influence the knowledge sharing social interaction? Researcher’s Intention
Different Learners’ Response / Experiences
Researcher’s Comments
From which locations is a learner willing to contact his/her peers for a learning task or problem?
…Home, library, hospital …University, basically anywhere on campus ...Only within university premises …Everywhere …Malls and shops …In the lab
Learning tasks and problems are not confined to fixed locations, thus learners could share knowledge about a problem from any location.
If a learner wanted to interact with a peer, how would s/he consider a peer’s present location?
.If they are in a lecture or the library and I have a small issue then I won’t bother them. In essence, I would just take into consideration my need to contact them and their availability in terms of what they’re doing and where they are. .If they were on campus or at the library or maybe at home, I’d feel free to contact them since they will probably be more flexible and relaxed to respond.
Learners use awareness of location to determine if the peer’s location favors a knowledge sharing social interaction.
How does a learner’s knowledge of where a peer is (their location) affect the knowledge sharing social interaction?
..Makes it easier because you would know who is closest to you for help ..It would be great to know if they are nearby, but if they’re not anywhere where I can reach them, then I’ll have to call regardless of where they are. …The closer he or she is, the better, because it will be easier for me to go to her when push comes to shove. …If I knew they were in a club or something, I wouldn’t bother them with school work.
The experiences show that awareness of a peer’s location would determine if it is plausible or would make sense to consult a peer.
Awareness of context and social presence is used as a tool that enables personalized sharing of experiences. An aspect of social awareness includes the location factor and context, which manifests as valuable to how the interaction is influenced. Culture and social background manifest as playing a fundamental role in learning contexts where learners often would rather seek help or support from peers who share a background. A common background allows better understanding and better communication and thus eases up learning and knowledge interactions. Awareness of context also manifests in how an interaction is influenced by a learner’s emotional and physical state, including behaviors and actions of others. Environmental situations also manifest as playing a role in a learner’s knowledge sharing decisions. Social presence of peers manifest as
42
fundamental with the role it plays in the sharing decisions. That is, awareness of a social presence enables an opportunistic learning and knowledge interaction and can also motivate a learner to engage with a learning task.
CONCLUSION This chapter did not directly address a specific learning management system (LMS) platform but addressed awareness and forms of communication that can be encountered as tools of LMS platforms. The chapter argued that to be able to design LMS that would efficiently enable social interaction among learners not confined to particular places, then there is a need to first understand how learners interact and the tools they use. In this chapter,
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
Table 3. How does a peer’s activity and emotional state as context awareness influence knowledge sharing social interaction? Researcher’s Intention
Different Learners’ Response / Experiences
Researcher’s Comments
How would knowledge of what a peer is doing (current activities) affect a learner’s interaction decision?
…Issues like a person sleeping or eating may make me wait a while. ..I need to know if they got my full attention ..If sleeping, I will leave them be, if studying will interact with them …Would respect that they are really busy and seek help elsewhere
A peer’s activity influences the decision about whether to consult with that learner or consult with someone else.
How do actions and behaviors of others affect a learner’s decision to learn or share experiences?
..I am motivated by seeing others work and tend to slack when others aren’t working …When others are discussing work, it increases chances of grasping concepts. ...It doesn’t affect me much, because we all have different learning capabilities. . ..I take seriously any feedback or advice from friends who have done courses I’m doing. ..If I am studying and people are noisy, I will not be able to concentrate. If they are serious, I too will be.
Presence of others, that is, seeing what they are engaged with influences the decision of a learner to get involved and do the same. For some, what others are doing does not influence their decision to work
To what extent do emotional states of a peer affect a learner’s actions?
…Very little, unless they’re my friends and I am concerned, or if it’s group work and they’re slacking due to moods. …You know what you can ask and can’t, their moods may limit your questions. ...Lets you know how to approach them
Learners could be sensitive to the mental state of a peer, thus altering the way they consult for help or sharing experiences.
How does the awareness of a peer’s current emotional situation (e.g., s/he is stressed, happy, sad, cheerful, etc.) help in interacting with them?
…Very helpful, you don’t want to interact and learn with a sad guy, so they ought to be happy, cheerful. If anything they can be stressed. …It’s useful in the sense that I can know whether to bother them or not. It makes little difference in group work unless their situation is very serious. .It makes me understand their behavior and the way in which they react to my questions or experience. …It makes me understand them better and assist in determining what things I can or cannot say. …Will try to approach them correctly, depending on what kind of mood they are in?
The experiences of learners suggest that emotional states and activities of peers do influence a social interaction. The social awareness determines how learners approach their peers for a learning purpose. The emotional states could be regarded as signs and rules, thereby altering the context in which a knowledge sharing activity happens. Although these are unwritten signs and rules, learners know not to disturb or pester a peer who is not in the best of emotion and spirit to help with a learning task.
learners used social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Mobile phones, SMS, PDAs, email, instant messaging were in this chapter seen as tools and forms of LMS communication, without having to study a specific LMS platform. In order to understand learning management systems
and the degree of their role in learning, we must understand both what the communication tools are and how the social awareness can be used to facilitate the ubiquitous learning. An understanding of the context and a sense of social presence enables the learner to model behavior along the expectations or the shared understanding of a social learning community.
43
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
Table 4. How do environmental situations as awareness of context influence knowledge sharing actions? Researcher’s Intention
Different Learners’ Response / Experiences
Researcher’s Comments
To what extent is a learner aware of his/her surroundings, during a learning activity, and how does that influence his/her decision to interact with peers?
..Temperature, I don’t like studying alone when it’s too cold so I normally chat a lot when it’s cold. ...The time of the day determines when I need to chat or where I need to work. ...Noise, too much movements, then I can’t concentrate on my work. …When I talk I can tell who is an understanding person from a lost one. …When I am with people I am not so aware of my surroundings in the sense that I am when I am alone, however, it is when its silent and cold that I tend to SMS/call people with my problems. ...If I see a lot of other students studying and discussing with their friends, they keep me motivated and I ask them questions.
The environmental context and situations do influence how learners act and how a learning action happens. The temperature during the day or ambience or noise levels may help a learner resort to a different knowledge action. For example, a learner saying that she prefers to do more social interaction than studying in isolation when the temperatures drop. A learner consults with peers, taking advantage of their social presence and surroundings.
Table 5. What is the role of Social Presence Awareness in the knowledge sharing environment? Researcher’s Intention
Different Learners’ Response / Experiences
Researcher’s Comments
How is it important for a learner to know a peer’s availability before he/she decides to interact with them?
Very important For me, it is quite important
Awareness of social presence or a peer’s availability is essential for knowledge interaction decisions
How does awareness of presence influence a learner’s knowledge action?
..It helps me determine who I can get assistance from ..Puts me at ease since I know my friends could help when I get stuck …I know I’m going to get help ..If they are available and willing to help, I will contact them. …I have a choice to pick any depending on who is more knowledgeable on that particular topic. …I know that when my academic questions arise I’ll be answered with certainty ...Then I could reach them whenever I need them ..It makes learning very easy, we could exchange ideas and information anytime and I will know that I am always able to ask for help
Social presence of knowledgeable peers is important for interaction whose purpose is sharing knowledge experiences. Learners are consciously at ease when they know their peers are available to help with a learning task, at anytime. Knowing a peer is socially present (awareness of social presence) gives a learner a sense of having a personalized academic support that “follows” them regardless of a learning problem and location.
How is a learner able to keep the sense of presence during a technology-mediated interaction, particularly when using textbased instant messaging.
..Make the messages as short as possible ..Instant or quick responses …I don’t want to wait for a response to my question, which is why I use IM.
Short, quick and instant responses grab a learner’s attention in a text-based social interaction.
How is a learner able to read emotions of peers during IM interaction and how can s/ he sense if the peer is interested in talking to them or not?
. ..Words, descriptive uninhibited words ... I know when they have something else in their minds . ..One word answers would indicate that he isn’t keen to talk. . ..The level of response and language used (enthusiasm)
Even though they are not in the same location, learners are able to exhibit social awareness levels during a mediated social interaction
44
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
The role of context awareness, and how it is experienced, is significant to ubiquitous social interaction whose outcome is personalized knowledge sharing. The context awareness in the learning management system environment manifests in three categories: 1) Mediating tools or applications as context of the sharing action learners use context awareness to decide on the appropriateness of the mediating tool. For example, if a learner is faced with a time-driven learning task she cannot resolve in isolation then she uses a PDA or a mobile phone to call or text message a peer as opposed to writing and sending an email to a peer. In this situation the learner is aware of the need to get instantaneous response, aware of how best to get the instant support, aware of how best to get social presence, as well as aware of the best mediating tool to exploit. 2) Location of a learner as context of the activity - learners need awareness of where they are themselves and where their peers are located, which is significant in determining the plausibility of a social interaction. 3) Learner’s situation –refers to context awareness dealing with culture and language, the learner’s current activity as well as environmental states. This chapter contributed towards an informed insight on how learners share their knowledge in a learning environment – focusing on LMS communication tools and their appropriateness. The chapter has shown that awareness of context and social presence (synonymous with social awareness) is a useful and significant characteristic for sharing learning experiences and the consequent learning support that occur independent of location and time. When learners socially interact through LMS mediation, they use context awareness to identify an appropriate tool and peer, in order to increase or decrease the knowledge sharing experience. Thus, in a learning environment, the social awareness is important since it influences the learning and knowledge decisions and actions, as the learning contexts change.
REFERENCES Alexander, P. A., Schallert, D. L., & Hare, V. C. (1991). Coming to terms: how researchers in learning and literacy talk about knowledge. Review of Educational Research, 61(3), 315–343. Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2). Argyle, M., & Dean, J. (1965). Eye-contact, distance and affiliation. Sociometry, 28, 289–304. doi:10.2307/2786027 Beyer, H., & Holtzblatt, K. (1998). Contextual design, defining vustomer-centred systems. MA, USA: Morgan Kaufmann Publishers, Inc. Bredo, E. (1994). Cognitivism, situated cognition and Deweyian pragmatism. Philosophy of Education. Retrieved June, 2010, Online at http:// www.ed.uiuc.edu/EPS/PES-Yearbook/94_docs/ BREDO.HTM] Byam, N. (1995). The emergence of community in computer-mediated communication. In Jones, S. G. (Ed.), Cybersociety. Newbury Park, CA: Sage. Davenport, T. H., & Prusak, L. (1998). Working knowledge: How organizations manage what they know. Boston, MA: Harvard Business Review Press. Dey, A. K., & Abowd, G. D. (1999). Towards a better understanding of context and context-awareness. GVU Technical Report GIT-GVU-99-22. College of Computing, Georgia Institute of Technology, Atlanta, Georgia. Goffman, E. (1978). Behavior in public places. Notes on the social organization of gatherings (4th ed.). New York, NY: Free Press.
45
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
Gunawardena, C. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2/3), 147–166. Gunawardena, C., & Zittle, F. (1997). Social presence as a predictor of satisfaction within a computer mediated conferencing environment. American Journal of Distance Education, 11(3), 8–26. doi:10.1080/08923649709526970 Holtzblatt, K., & Jones, S. (1994). Contextual inquiry: A participatory design technique for system design. In Schuler, D., & Namioka, A. (Eds.), Participatory design: Principles and practice. Englewood Cliffs, NJ: Prentice-Hall. Jaffer, S., Ng’ambi, D., & Czerniewicz, L. (2006). The role of ICTs in higher education in South Africa: One strategy for addressing teaching and learning challenges. Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa, 10-21 July. Retrieved from http:// emerge2006.net. Kekwaletswe, R. M. (2006). Social presence and context awareness for knowledge transformation in an m-learning environment. In Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa (10-21 July). Retrieved from http://emerge2006.net. Kekwaletswe, R. M. (2007). Knowledge transformation in a mobile learning environment: An inquiry of context and social presence awareness. PhD thesis. South Africa: University of Cape Town. Kekwaletswe, R. M. (2009). Conceptualizing ubiquitous learning through context-aware wired and wireless Web services. In Proceedings of the IADIS Mobile Learning conference, Barcelona, Spain (25 – 28 February).
46
Kekwaletswe, R. M., & Ng’ambi, D. (2006). Ubiquitous social presence: Context-awareness in a mobile learning environment. In Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing, Taichung, Taiwan (90-95). Taichung, Taiwan. Laudon, K. C., & Laudon, J. P. (1995). Information systems: A problem-solving approach (3rd ed.). Orlando, FL: Dryden Press. Lonsdale, P., Baber, C., Sharples, M., & Arvanitis, T. N. (2003). A context awareness architecture for facilitating mobile learning. In Proceedings of MLEARN 2003 (LSDA), London. Ng’ambi, D. (2004). Towards a knowledge sharing framework based on student questions: The case for a dynamic FAQ environment. PhD thesis. South Africa: University of Cape Town. Ng’ambi, D., & Hardman, J. C. (2004). Towards a knowledge-sharing scaffolding environment based on learners’ questions. British Journal of Educational Technology, 35(2), 187–196. doi:10.1111/j.0007-1013.2004.00380.x Ngwenyama, O. K., & Lee, A. (1997). Communication richness in electronic mail: Critical social theory and the contextuality of meaning. Management Information Systems Quarterly, 21(2), 145–167. doi:10.2307/249417 Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14–37. doi:10.1287/orsc.5.1.14 Patriotta, G. (2003). Organizational knowledge in the making: How firms create, use, and institutionalize knowledge. Oxford, UK: Oxford University Press. Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence and performance in an online course. Journal of Asynchronous Learning Networks, 6(1).
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
Polanyi, M. (1958). Personal Knowledge. Chicago, IL: The University of Chicago press. Polanyi, M. (1966). The tacit dimension. London, UK: Routledge & Kegan Paul. Rakotonirainy, A., Loke, S. W., & Fitzpatrick, G. (2000). Context awareness for the mobile environment. Retrieved Oct 11, 2009 from ftp:// ftp.cc.gatech.edu/pub/gvu/tr/2000/00-18r.ps.Z. Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1), 68–88. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2). Ryder, M., & Wilson, B. (1997). From center to periphery: Shifting agency in a complex technical learning environments. Paper presented at the Meeting of the American Educational Research Association, Chicago, IL. Schmidt, A., Beigl, M., & Gellersen, H. W. (1999). There is more to context than location. Computers & Graphics, 23, 893–901. doi:10.1016/S00978493(99)00120-X Scribner, S. (1987). Thinking in action: Some characteristics of practical thought. In Sternberg, R., & Wagner, R. (Eds.), Practical intelligence: Nature and origins of competence in everyday world. Cambridge, UK: Cambridge University Press. Shariq, Z. S. (1999). How does knowledge transform as it is transferred? Speculations on the possibility of a cognitive theory of knowledgescapes. Journal of Knowledge Management, 3(4), 243–251. doi:10.1108/13673279910303998
Shea, P. J., Pickett, A. M., & Pelz, W. E. (2003). A follow-up investigation of “teaching presence” in the SUNY Learning Network. Journal of Asynchronous Learning Networks, 7(2), 61–80. Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. Toronto, Canada: Wiley. Sutherland, R., Facer, K., Furlong, R., & Furlong, J. (2000). A new environment for education? The computer in the home. Computers & Education, 34(3-4), 195–212. doi:10.1016/ S0360-1315(99)00045-7 Swan, K. (2002). Building communities in online courses: The importance of interaction. Education Communication and Information, 2(1), 23–49. doi:10.1080/1463631022000005016 Swan, K. (2003). Developing social presence in online discussions. In Naidu, S. (Ed.), Learning and teaching with technology: Principles and practices (pp. 147–164). London, UK: Kogan Page. Van Beveren, J. (2002). A model of knowledge acquisition that refocuses knowledge management. Journal of Knowledge Management, 6(1), 18–22. doi:10.1108/13673270210417655 Walther, J. (1994). Interpersonal effects in computer mediated interaction. Communication Research, 21(4), 460–487. doi:10.1177/009365094021004002 Wiener, M., & Mehrabian, A. (1968). Language within language: Immediacy, a channel in verbal communication. New York, NY: AppletoncenturyCroft. Winograd, T., & Flores, F. (1986). Understanding computers and cognition: A new foundation for design. Norwood, NJ: Ablex.
47
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
ADDITIONAL READING Bush, M. D., & Mott, J. D. (2009). The transformation of learning with technology: Learner-centricity, content and tool malleability, and network effects. Educational Technology, 49(2), 3–20. Delta Initiative (2009). The state of learning management in higher education systems, report for the California State University System. Dourish, P., & Bellotti, V. (1992). Awareness and coordination in shared workspaces. In Proceedings of ACM CSCW Conference (pp. 107-114). Dourish, P., & Bly, S. (1992). Portholes: Supporting awareness in a distributed work group. Proceedings of the CHI ‘92 Conference on Human Factors in Computing Systems (pp. 541-547). Engeström, Y. (1987). Learning by expanding: An activity theoretical approach to developmental research. Helsinki, Sweden: Orienta-Konsultit. Engeström, Y. (1999). Activity theory and individual and social transformation. In Engeström, Y., Miettinen, R., & Punamaki, R. (Eds.), Perspectives on activity theory. Cambridge, UK: Cambridge University Press. Gardner, C. (2009). A personal cyberinfrastructure. EDUCAUSE Review, 44(5), 58–59. Gobbin, R. (1998). Adoption or rejection: Information systems and their cultural fitness. In H. Hasan, E. Gould, and P. Hyland (Eds.), Information systems and activity theory: Tools in context (pp. 109-124). University of Wollongong Press, Wollongong, N.S.W. Gould, S. J. (1987). Time’s arrow: Time’s cycle: Myth and metaphor in the discovery of geographical time. Cambridge, MA: Harvard University Press.
Gutwin, C., Greenberg, S., & Roseman, M. (1996). Workspace awareness in real-time distributed groupware: Framework, widgets and evaluation. In Proceedings of HCI on People and Computers XI (pp. 281-298). Jonassen, D. H., & Rohrer-Murphy, L. (1999). Activity theory as a framework for designing constructivist learning environments. Educational Technology Research and Development, 47(1), 62–79. doi:10.1007/BF02299477 Lane, L. M. (2009). Insidious Pedagogy: How course management systems impact pedagogy. First Monday, 14(10). Mlitwa, N., & Van Belle, J. P. (2010). A proposed interpretivist framework to research the adoption of learning management systems in universities. Communications of the IBIMA (Vol. 2010). Retrieved from http://www.ibimapublishing.com/ journals/CIBIMA/cibima.html. Morgan, G. (2003). Faculty use of course management systems (Vol. 2). ECAR Research Bulletin. Sclater, N. (2008). Web 2.0, personal learning environments, and the future of learning management systems,” ECAR Research Bulletin (vol. 2008, no. 13). Suchman, L. (1987). Plans and situated actions: The problem of human–machine communication. Cambridge, UK: Cambridge University Press. Taylor, J. (2006). Evaluating mobile learning: What are appropriate methods for evaluating learning in mobile environments? In M. Sharples (ed.), Big issues in mobile learning: Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative: University of Nottingham, UK. Retrieved November 29, 2009, from http://telearn.noe-kaleidoscope.org/ warehouse/Sharples-2006.pdf. Tollmar, K., Sandor, O., & Shömer, A. (1996). Supporting social awareness at work, design and experience. In [New York, NY: ACM Press]. Proceedings of CSCW, 96, 298–307.
48
Knowledge Sharing in a Learning Management System Environment Using Social Awareness
KEY TERMS AND DEFINITIONS Awareness: An understanding of the activities of others, which provides a context for your own activity. Context: Understood as the situation in which a learner or a group of learners find themselves. Contextual Inquiry: A field research framework that depends on conversations with users in the context of their work. Knowledge Sharing: Involves two actions; transmission (sending or presenting knowledge to a potential learner) and absorption by the au-
dience. If knowledge is not absorbed, it has not been shared. Social Awareness: Is synonymous with context and social presence awareness. Social Presence: Defined and understood to be the mediated presence of another learner who could provide personalized on-demand social support for a learning problem as the learner traverses varied learning contexts. Ubiquitous Personalized Support: Refers to the provision of context sensitive and anywhere, anytime support as a learner traverses varied locations.
49
50
Chapter 3
Learning 2.0:
Using Web 2.0 Technologies for Learning in an Engineering Course Thomas Connolly University of the West of Scotland, UK Carole Gould University of the West of Scotland, UK Gavin Baxter University of the West of Scotland, UK Tom Hainey University of the West of Scotland, UK
ABSTRACT Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web 2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web 2.0 technologies provided by the chosen LMS. DOI: 10.4018/978-1-60960-884-2.ch003
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
Learning 2.0
INTRODUCTION There has been considerable research into the perceived benefits of eLearning and Learning Management Systems (LMS) within education and it is clear that LMS now play a pivotal role in the delivery of eLearning within many educational institutions. The research literature cites many advantages of eLearning, particularly the convenience and flexibility offered by the (asynchronous) ‘anytime, anywhere, anyplace’ education. However, much less research has been carried out into the use of educational technologies and tools within vocational training environments. The purpose of this chapter is to discuss the impact on learning with the introduction of a LMS into a vocational engineering course. This chapter discusses the pilot implementation of Web 2.0 tools within an LMS and aims to answer a number of questions: (i) Can technology supplement the hands-on nature of vocational training? (ii) Can the use of wikis and forums aid vocational training? (iii) Can the pilot be considered a success? Much of the research in this area has been mainly anecdotal and has not considered the different nature of vocational training with most of the research focusing on the traditional educational environment. This chapter utilises both qualitative and quantitative surveys on the views of trainees and instructors and aims to identify the areas within the training programme where the LMS could be utilised further to aid learning. It also considers the areas where the use of the LMS has not been as successful as anticipated and the reasons for this. The next section of this chapter discusses the literature on LMS, the use of Web 2.0 technologies within education, and ePortfolios. The subsequent sections introduce the research rationale, the case study and an empirical analysis of the pilot implementation. The chapter concludes
with a discussion of the findings and provides some recommendations for the implementation of eLearning within vocational training.
PREVIOUS RESEARCH eLearning can be defined as “… any use of Web and Internet technologies to create learning experiences” (Horton, 2003, pp. 13). eLearning is essentially an evolved form of distance education, which Connolly and Stansfield (2007a) describe through a six-generation model, as depicted in Figure 1. The first generation (the ‘correspondence model’) was provided mostly through paper-based instruction, characterized by the mass production of educational materials. The difficulty with correspondence education has been the infrequent and inefficient form of communication between the instructor and the learners. Further, it was difficult to arrange for peer interaction in correspondence based distance education. The second generation (the ‘multimedia model’) was provided through integrated multimedia such as delivering courses via television or introducing material like audio and video tapes, computer-based learning (CBL) in addition to printed material. The third generation was provided through two-way communications media such as audio/video-conferencing and broadcast technology. The fourth generation of distance education (the first generation of eLearning) is defined as mainly passive use of the Internet, consisting primarily of conversion of course material to an online format, low-fidelity streamed audio/video, and basic mentoring using email. However, the educational philosophy still belongs to the pre-Internet era. The fifth generation of distance education (the second generation of eLearning) uses more advanced technologies consisting of high-bandwidth access, rich streaming media, online assessment (eAssessment) and LMS that provide access to course material, communication facilities, and learner services. The sixth generation of distance education (the third
51
Learning 2.0
generation of eLearning) is a more collaborative learning environment based much more on the constructivist epistemology, promoting reflective practice through tools like ePortfolios, Web 2.0 technologies such as blogs and wikis, online communities, and using interactive technologies such as online visualizations, games, and simulations.
We are also now starting to see the development of mLearning (mobile learning) through devices like PDAs (Personal Digital Assistants), mobile phones and smartphones, and tablet devices. For second and third generation eLearning, LMS support new approaches for people to learn and assist with the delivery but also with the way
Figure 1. Models of distance education (adapted from Connolly & Stansfield, 2007a)
52
Learning 2.0
in which information is presented leading to acquisition of new knowledge (Holmes & Gardner, 2006). eLearning and the use of LMS are now an integral part of most educational institutions with educational technologies witnessing exceptional levels of growth in recent years. To support the growth of LMS, many schools, colleges and universities have invested heavily in up-to-date technology. Also referred to as Virtual Learning Environments (VLEs), ‘learning platforms’, ‘distributed learning systems’, ‘course management systems’ and ‘instructional management systems’, LMS combine a range of course/subject management and pedagogical tools to provide a means of designing, building and delivering online learning. LMS are scalable systems that can be
used to support an institution’s entire set of teaching and learning courses. Although LMS are used extensively within educational institutions globally, their use is relatively low within vocational training companies. One reason for this may be that vocational trainers do not appreciate that some aspects of vocational training may lend itself well to online delivery. There are many different LMS products available, some at considerable cost and others available as open source. Regardless of which product is chosen, most LMS contain similar functionalities as shown in Table 1. As with most things, eLearning has advantages and disadvantages. The research literature cites many advantages of eLearning, particularly the convenience and flexibility offered by the
Table 1. Comparison of LMS functionality Features/Tools
LMS Blackboard ProSites
Moodle
Learnwise
Frog
E-portfolio
N
Y
Y
Y
File up-load
Y
Y
Y
Y
Notice/bulletin board
Y
Y
Y
Y
Course outlines
Y
Y
Y
Y
Assignments
Y
Y
Y
Y
Assessments
Y
Y
Y
Y
Multi-media resources
Y
Y
Optional extra
Y
Evidence gathering
Y
Y
Y
Y
Calendar
Y
Y
Y
Y
Administration tools
Y
Y
Y
Synchronous collaboration tools (video conferencing)
Y
Y
Optional extra
N
Forum/discussion board
Y
Y
Y
Y
Email
Y (Internal)
Y (Internal)
Y (Internal)
Y (Internal)
External links
Y
Y
Y
Y
Student home page
Y
Y
Y
Y
Real-time chat
N
Y
Y
Y
Quiz design
Y
Y
Y
Y
Costs
£6,655 per annum (200 users/licences) £3,152 per annum (100 additional users/ licences
Open source Additional costs if hosting required
Ranges from £3,300 per annum for 1,000 users to £27,500 per annum for 50,000 users
Bespoke system - £26,000 Yearly support - £4,500 Standard package - £22,500 Yearly support - £4,500
53
Learning 2.0
(asynchronous) ‘anytime, anywhere, anyplace’ education (McDonald, 2002), which gives learners time for research, internal reflection, and ‘collective thinking’ (Garrison, 1997). Moreover, the text-based nature of eLearning normally requires written communication from the learner, which along with reflection, encourage higher level learning such as analysis, synthesis, and evaluation, and encourage clearer and more precise thinking (Jonassen, 1996). In addition, eLearning courses also have the capability to present multiple representations of a concept, which allows learners to store and retrieve information more effectively (Kozma, 1987). It is also argued that increased social distance provides a number of distinct advantages to online conferences (synchronous or asynchronous). In written communications anonymity of characteristics such as gender, race, age, or social status can be preserved, which can reduce the feeling of discrimination and provide equality of social interaction among participants. In turn, this can permit the expression of emotion and promote discussion that normally would be inhibited (Gunawardena, 1993). eLearning is not without its disadvantages; for example (Connolly & Stansfield, 2007b): • •
• • •
•
54
Costs may initially exceed more traditional methods; More responsibility is placed on the learner who has to be self-disciplined and motivated; Some learners lack access to a PC/Internet or have difficulty with the technology; Increased workload for both students and faculty; Non-involvement in the virtual community may lead to feelings of loneliness, low self-esteem, isolation, and low motivation to learn, which in turn can lead to low achievement and dropout; Dropout rates tend to be higher in eLearning courses than in traditional face-to-face courses, often 10 to 20 percentage points higher.
Perhaps one of the most damaging criticisms is that some eLearning simply replicates the social organization of traditional education and training and that the potential benefits of eLearning - of personalized and accessible learning experiences - are missed. Taking this on board, this is one of the reasons this research is of particular importance. There is a high chance that the traditional nature of the organisation under investigation may simply deploy an LMS but not utilize it to its full potential.
Learner Expectations and Web 2.0 Web 2.0 is a term often associated with “… the social use of the Web which allow[s] people to collaborate, to get actively involved in creating content, to generate knowledge and to share information online” (Grosseck, 2009, pp. 478). Web 2.0 applications, such as blogs and wikis, are being introduced into LMS such as Blackboard and Moodle, providing students with increased flexibility in terms of how they communicate with fellow students and gain feedback from peers. In industry the use of Web 2.0 applications are being gradually introduced as organisations have begun to realise their potential for the purposes of learning and information sharing. The way in which education and training institutions and industrial organisations facilitate learning and information sharing is being determined by the expectations and prior learning experiences of the individuals within them. Within most learning or working environments today, there is a sense of expectancy that innovative learning and communication channels should already be in place to accommodate the diverse ways in which individuals learn. The aspects of institutional and organisational competitiveness that are closely related to the concept of the ‘knowledge economy’ means that institutions promoting learning via technological means now have an interest in developing and running learning initiatives with the minimum of effort (DeRouin, Fritzsche, & Salas, 2005). These particular factors are having an impact
Learning 2.0
upon the format and delivery of eLearning initiatives within educational and industrial contexts. Web 2.0 technologies are attractive within these contexts as they allow greater student independence and autonomy, increased collaboration and an increase in pedagogic efficiency (Franklin & Harmelem, 2007, pp. 1). For this reason, the use of Web 2.0 technologies could be an invaluable asset to those institutions delivering ‘on-the-job’ training, especially where an element of group work is undertaken. It also facilitates knowledge sharing and promotes learning in an industrial context, as well as taking account of pedagogical issues. This was important to our study given that one part of the vocational training to be delivered was a group-based project.
Social Software and Learning 2.0 It could be argued that the concept of eLearning is being enhanced by the rapid development of ‘social software’. McKelvie, Dotsika, and Patrick (2007) state that “social software is a community driven technology which facilitates interaction and collaboration and depends largely on social convention.” Though social software can be used on an individual basis it is predominately concerned with the notions of open and collective communication, dialogue and the ability to liaise with individuals collectively. Social software is having an effect upon eLearning delivery within education and industry as it has altered the way in which learning is taught and learnt. The use of social software allows the learner to generate knowledge and share their learning experiences on a collective level as well as allowing users to openly reflect upon what they have learnt. eLearning distinguishes itself from social software as it is predominately associated with electronic instruction and is better suited for education and training purposes. Web 2.0 is transforming the way in which people learn as the learning is predominately social and self-directed in nature whereas eLearning is normally associated with
individual learning. The use of social software and Web 2.0 technologies have given rise to the term ‘Learning 2.0’, sometimes referred to as eLearning 2.0, which broadly summarizes all opportunities arising from the use of social media for learning, education or training.
The Pedagogy of Learning 2.0 The interactive and collaborative nature of social software makes it highly suited towards sustaining and facilitating what are known as communities of practice (CoPs) or “groups of people informally bound together by shared expertise and passion for a joint enterprise” (Wenger & Snyder, 2000, pp. 139). In conjunction with the concept of CoPs, the learning theory of social constructivism appears to complement and accommodate the principles surrounding the use and learning benefits associated with Learning 2.0. The constructivist view of learning adopts the stance that learners do not learn individually from one another and stresses the relevance regarding the socio-cultural context of learning. Predominately, social constructivism contends that knowledge is formulated through the processes of social interaction and collaborative learning. Though the concept of social software can, in theory, support a wide range of learning approaches, it is inherently applicable towards the learning theories of social constructivism and CoPs. It has been generally regarded that one of the salient aspects of any CoP is its ability to construct and store collective knowledge in what has been referred to as a ‘shared repertoire of communal resources’ (Wenger, 2000). Additionally, CoPs are most usually distinctly defined by the concepts of collective understanding, mutual engagement and shared repertoire (Wenger, 2000). It could be argued that for this reason, the use of Learning 2.0 could be an invaluable asset to organisations. The examples given below demonstrate the growth of those technologies and tools that facilitate knowledge sharing in the wider community.
55
Learning 2.0
ePortfolios As well as the Learning 2.0 technologies, many educational institutions and organisations are utilising ePortfolios, the electronic version of the paper portfolio. As with LMS, the use of ePortfolios is also growing in popularity, particularly within vocational training. A portfolio can be defined as “a collection of documents relating to a learner’s progress, development, and achievements”; an ePortfolio simply indicates that some or all of the evidence is collected in digital form (Beetham, 2009). EDUCAUSE (2005) defines an ePortfolio as “a valuable online tool that learners, faculty, and institutions can use to collect, store, update, and share information. E-Portfolios allow students to reflect on their learning, communication with instructors, document credentials, and provide potential employers with examples of their work”. ePortfolios are well suited to education, vocational training and working environments as they capture the concept of lifelong learning and support individuals as they progress through school, college, Higher Education, training and employment (Richardson & Ward, 2005). Learners gather learning evidence and define these evidences through a self-reflection process. They attribute their competences to learning products or outcomes and reflect on how they acquired those competences. From a pedagogical perspective this process helps learners to better understand how they learn and helps them to become self-directed learners (Berlanga et al., 2008). ePortfolios can be classified into various types – assessment, presentation, learning, personal development and shared ownership (Curyer et al., 2007) but in reality most are a combination. If a standard approach was adopted for ePortfolios, institutions and organisations could share and exchange ePortfolio data, which could lead to the streamlining of the processes connected to prior learning, with student transitions through courses, and with training that involves either sequential or parallel movement through multiple institutions
56
and companies (Curyer et al., 2007). This could also help to fulfil the concept of an ePortfolio being utilised throughout lifelong learning. Richardson and Ward (2005) carried out an in-depth study of 12 different ePortfolios and found, amongst other things, that no two systems were identical or offered the same range of functions. However, it should be considered that an ePortfolio tends to be chosen on a ‘fit-for-purpose’ basis and often vendors may customise their ePortfolios to suit a particular customer, as is the experience of the authors. Most ePortfolios are driven by the learner; that is, the learner is responsible for the maintenance of the ePortfolio and decides who has access to its contents but in some environments, as in the study under investigation, this may not be desirable, as some aspects of vocational training need to be driven by the instructor, not the learner. Internal examiners and external verifiers often need access to trainee assessment material and that assessment material cannot be amended once verified. In our study, the instructors required a measure of customisation of the ePortfolio, which included allowing multiple assessors and the instructors themselves having overall control of the ePortfolio.
ePortfolio Use in Higher/ Further Education According to JISC (2008) ePortfolios represent the latest in a line of technology-based innovations that are becoming an integral part of the learning landscape in Higher and Further Education. Student ePortfolios developed out of faculty-assigned, print-based student portfolios as far back as the mid-80s. They were typically found in art-related disciplines or those that consisted of substantial written components, such as English studies, and gained greater importance in education during the mid-90s. Student ePortfolios are commonly found in college programmes where teachers use them to provide evidence of competence. This includes
Learning 2.0
communications, maths, business, nursing and engineering to record students’ learning experiences and skill set (Lorenzo & Itellson, 2005). The Open University in the UK has been using ePortfolios as an assessment tool in online courses for many years (Mason, Pegler, & Weller, 2004) and the University of Washington developed an ePortfolio in 2001 to allow students to record their entire educational learning experiences in an organised and integrated manner. A survey carried out by Strivens (2007) found that of those who participated in the survey, 20 institutions (54% of those with an ePortfolio) commented that the ePortfolio was available to all students across the institution. The increase in the use of ePortfolios in Higher Education is further supported by Beetham (2009) who states that work is currently being carried out to integrate LMS, student record systems and ePortfolio tools to provide formative feedback. It appears that educational institutions have embraced the concept of ePortfolios and appreciate their value.
ePortfolio Use in Vocational Training Portfolios (paper-based) have been utilised for many years in vocational training; for example, Austria has been using portfolios in teacher training for the past 12 years and covers topics such as supervision and professional upgrade in vocational education and is regarded as a working portfolio, as examination for teachers is impractical (Dorninger & Schrack, 2007). ePortfolios have been endorsed by some of the major vocational examiners such as City & Guilds in the UK. City & Guilds (2009) undertook a survey of 95 colleges and training providers and found that, although cost savings were significant, the main advantage was the reduction in time taken for candidates to complete their qualification when using an ePortfolio. However, although feedback was positive, only 16% of those surveyed used ePortfolios. Some centres expressed fear over technical glitches, which could result in lost work,
and some resistance to change was also noted. The literature demonstrates that ePortfolios offer a valid way to assess and ensure completion of an individual’s training, and while usage is not pervasive, uptake is growing.
METHODOLOGY The aim of this study was to determine whether eLearning could be used successfully within a vocational training environment for engineering students. More specifically, the objectives of the project were (i) to select suitable LMS and ePortfolio systems, (ii) develop some of the provision in an online format, and (iii) evaluate the success of this provision. This study started in late 2008 and the initial part of the study was completed in May 2010. Empirical data was obtained from a cohort of students going through their Induction programme, which was the first course developed in an online format, and from their final group project before completing their training. In addition, the instructors were surveyed in order to ascertain their views of the LMS and the use of the wiki and forum for the group project. The instructors’ survey also included questions relating to the use of the LMS as a learning tool. For this study, the research philosophy of interpretivism was adopted. Interpretivism is an epistemology that advocates that the researcher must understand the differences between humans in their social roles as actors. This philosophy places emphasis on conducting research among people as opposed to objects, such as cars and houses. Interpretivism stems from the intellectual tradition of phenomenology, which is concerned with the ways in which humans make sense of the world around them (Saunders, Lewis & Thornhill, 2007). According to Whisker (2001), phenomenology encourages both quantitative and qualitative methods. However, it should be considered that quantitative research is more often associated with positivism. McBride and Schostak (1994) state:
57
Learning 2.0
“Where quantitative forms of research, employing questionnaires and sampling procedures attempt to eradicate the individual, the particular and the subjective, qualitative research gives special attention to the subjective side of life...qualitative researchers are more likely to ask how it feels... they focus upon the social construction of such things...”. As such, this study is therefore adopting a mixed methodology with an Action Research and within case study approach as the authors are actively involved in the project. Gerring (2007, pp. 20) defines case study as “the intensive study of a single case where the purpose of that studyat least in part- to shed light on a larger class of cases”. It is hoped that at the termination of the project it will be possible to develop a framework that will aid practitioners in the successful implementation of an LMS in vocational training, as well as promote a rich area for future research. The next section will outline the company under investigation, the type of training delivered and how the chosen technologies were selected.
Case Study: Vocational Training This project came about when University of the West of Scotland entered into a two-year partnership programme with a local training organisation, East Kilbride & District Engineering Group Training Association (EKGTA), to explore how much of a Modern Apprenticeship programme could be delivered online. EKGTA is an employer-led training provider for the engineering industry with charitable status. Established in 1966 and recognised as one of the premier training groups throughout Scotland, it aims to serve the needs of the employer, whilst ensuring candidates have the opportunity to develop the knowledge and skills necessary in employment. The Association specialises in training Modern Apprentices at craft and technician levels, and in basic engineering skills training to national standards. EKGTA provides training in other disciplines, such as, Health & Safety, Computer Based Training, Professional
58
Development but apprentice engineering training is EKGTA’s core business and for this reason the main focus is directed towards that training. A Modern Apprenticeship Programme may involve: • • • • •
A period of training in an approved training centre (off the job). Completion of a Level 2 Vocational Qualification. Attainment of core skills to intermediate one level (minimum). Completion of a National Certificate (day release at FE College). Completion of a Level 3 Vocational Qualification in company.
Level 2 competence involves the application of knowledge and skills in a significant range of varied work activities, performed in a variety of contexts. Some of the activities are complex or non-routine, and there is some individual responsibility and autonomy. Collaboration with others may often be a requirement On the other hand, Level 3 competence involves the application of knowledge and skills in a broad range of varied work activities performed in a wide variety of contexts, most of which are complex and nonroutine. There is considerable responsibility and autonomy, and control or guidance of others is often required. There are six instructors who deliver the practical element of the qualification with the academic element delivered onsite by a local Further Education college. Once the trainees return to their company, advisors visit trainees on average every 12 weeks and oversee the continuation of the training. Management appreciate the advancements in technology that could potentially support and enhance the training programmes offered by the Association, however, as the project has progressed, it has become obvious that this sentiment is not echoed by all staff. Before the project, technology within EKGTA was used primarily for the record-
Learning 2.0
ing of completed assignments (ePortfolio) and the storing of lecture material, and was therefore used far below its potential. In the early stages of the project the pilot of the LMS was to be rolled out within the electrical area of the workshop, but on further discussion it was decided that this would not be an inclusive approach. The decision was taken to implement the LMS throughout the entire organisation in two stages: the Induction programme and the group project, thus including all instructors.
Selecting the LMS In discussion with EKGTA, a number of features were identified that the chosen LMS had to support. Existing students were also consulted. Two informal group forums were set up with ten participants in each. This helped to identify those features that students believed would enhance learning. Almost all students agreed that 24x7 access to learning materials would be advantageous, more use of multimedia, such as video demonstrations and interactive quizzes, with immediate feedback to reinforce learning. To identify a suitable LMS, it was first necessary to carry out desk research on a number of different LMS to identify the functionalities that each one supported and to match these against the company’s and the students’ criteria. A short list of potential systems was drawn up that included Blackboard ProSites, Learnwise, Frog and Moodle, Moodle being the only open source option. The outcome of the research demonstrated that there was very little difference in the functionalities of each, as shown in Table 1. The ability to automatically map student evidence of their work to individual components of assessment was regarded by the instructors as the most important function that needed to be supported by the ePortfolio component as this was their most time intensive activity, however, after much research it became apparent that this function was not readily available with any of the
shortlisted LMS systems and it would be necessary to select a separate ePortfolio system to the LMS, although it was hoped the two could be integrated together in some way. After extensive analysis and consultation with management and staff, Moodle was chosen as the LMS as it was highly modular and provided the same features as the commercial systems but at no cost to the company. The other LMS investigated were more suited to educational institutions with large numbers of students. For this project, the maximum student uptake in one year would be limited to 120 and so the need for a large commercial product could not be justified. In addition, Moodle has a large and growing community with almost 40 million users and over 50,000 sites worldwide at the time of writing (Moodle, 2010). The next stage of the project was to identify a suitable ePortfolio system.
Selecting the ePortfolio Again, a short list of potential ePortfolio systems was drawn up based on instructors’ requirements, which included Learning Assistant, One File, Pebble Pad and Mahara, which is open source and an add-on module for Moodle. EKGTA previously used two ePortfolios, one at Level 2 and one at Level 3, which were not integrated in any way. The ePortfolio at Level 2 gives on-campus students access to all lecture material, standard assessment documentation, and in some instances multimedia. At Level 3 advisors use the Modern Apprenticeship online ePortfolio system. At this level, students are off-campus, therefore regular remote communication between advisors and students needs to be undertaken. The Modern Apprenticeship online ePortfolio did not fulfil some of the criteria required by the instructors and was no longer being fully supported and so was excluded from further consideration. After in-house evaluation, it was concluded that Learning Assistant best suited the requirements. Learning Assistant is marketed as an ePortfolio and eAssessment solution for training
59
Learning 2.0
centres that deliver vocational qualifications. It has been designed specifically to meet the needs of the vocational training environment. Learning Assistant, as well as being fully integrated from Level 1 through to Level 4, is a fully supported system with regular updates to meet the changing needs of the students and any changes to the engineering qualification. In addition, the Learning Assistant vendor could customise the system to support multiple assessors and facilitate the automatic mapping of student evidence of their work to individual components of assessment.
Identifying Priority Features After researching a variety of LMS and ePortfolio systems to identify their generic features and the system(s) that is more suited to the vocational training environment, it was then necessary to survey the six workshop instructors and one training advisor to identify those features that would be regarded as a priority for implementation. This gave an indication of where the end-users see technology enhancing learning and to help in the delivery of training. Table 2 provides details of this survey. The lower the number assigned, the higher importance assigned. The data demonstrates that the top five priority features are: •
• • • •
ability to automatically map student evidence of their work to individual components of assessment; a suitable ePortfolio system; evidence gathering; uploading of assignments; a file upload capability.
These features were subsequently prioritised for implementation.
Implementation In the early stages of the project the pilot of Moodle was to be rolled out within the electrical area of the workshop, but on further discussion 60
it was decided that this would not be an inclusive approach. The decision was taken to implement Moodle throughout the entire workshop, thus adopting an inclusive approach. All trainees go through a week long induction programme - a general induction and a workshop induction. A course was developed and populated with two units: general induction and workshop induction. All lecture material and presentations were uploaded to Moodle and self-assessment quizzes designed to reinforce learning. Learning Assistant was customised by the vendor and installed for students to use and a single sign-on was developed between Moodle and Learning Assistant so students only had to log in once to access both systems. Figure 2 shows an example page in the Moodle system and Figure 3shows the reporting system within the ePortfolio.
FINDINGS AND DISCUSSION In this section, we discuss the findings from the first year of using the LMS for training the engineering students. We start with an evaluation of the results from the Induction programme followed by the group project.
Induction Programme Results A five-day Induction programme was run for a cohort of students in Autumn 2009. Demographic information was collected at the start of Induction and an evaluation questionnaire was distributed at the end.
Demographic Questionnaire 26 participants completed the demographic questionnaire. 25 (96%) of the respondents were male and 1 (4%) of respondents were female. The mean age of participants was 17 (SD = 0.748) with a range from 16 to 19. 17 (65%) of participants started an apprenticeship at EKGTA when leaving school, 9 (35%) of participants did not. The
Learning 2.0
Table 2. Survey results for priority features ePortfolio
Student home page
Course outlines
Assignments
Map evidence to assessment
Multimedia
1
6
4
7
2
2
4
13
16
3
2
14
1
13
8
6
2
10
6
16
4
5
3
1
5
12
13
8
1
7
1
12
3
5
2
4
4
10
5
1
2
8
22
82
53
35
14
46
File up-load
Evidence gathering
Notice board
External links
Quiz design
Forum
3
3
5
8
14
15
7
1
12
11
15
8
9
3
7
12
11
14
2
7
12
8
13
14
4
2
14
11
16
9
6
11
14
7
8
13
7
3
9
12
13
11
38
30
73
69
90
84
Real time chat
Calendar
Email
Admin tools
16
2
6
3
9
10
5
6
15
16
4
5
15
11
10
9
15
10
3
6
15
9
16
10
16
14
15
6
101
72
59
45
majority of the participants (18, 69%) left school in 2009, 6 (23%) left school in 2008 and 2 (8%) left in 2007. The majority of the participants (23, 88%) indicated that they had no other full-time employment in the past while 3 (12%) did. 1 participant indicated that they worked in the hospitality sector. 19 (73%) of participants indicated that they had not attended college prior to attending the training course, whereas 6 (23%) of participants indicated that they had attended college first. The courses
that were undertaken were primarily in the areas of construction (joinery, roofing, brick-laying, etc) and creative industries (art, fashion design, media, etc). These courses were undertaken at intermediate, NC and HNC level. Table 3 shows the standard grade qualifications and Table 4 the intermediate/higher qualifications achieved by the participants.
61
Learning 2.0
Figure 2. Screenshot of Moodle page
Table 3. Standard grade qualifications achieved
Table 4. Intermediate or higher qualifications achieved
Standard Grade Qualification
Number with qualification
Mathematics
21 (81%)
Higher Grade Qualification
Number with qualification
English
24 (92%)
Mathematics
20 (77%)
Physics
17 (65%)
English
15 (58%)
Craft and Design
13 (50%)
Physics
11 (42%)
French
16 (61.5%)
Craft and Design
5 (19%)
Biology
6 (23%)
French
3 (12%)
Geography
13 (50%)
Biology
1 (4%)
Chemistry
10 (38%)
Geography
5 (19%)
Modern Studies
13 (50%)
History
5 (19%)
German
2 (8%)
None
3 (12%)
Spanish
1 (4%)
Modern Studies
2 (8%)
20 (77%)
Others
14 (54%)
Other
Evaluation Questionnaire Students were asked to complete a questionnaire at the end of the Induction programme. 41 participants completed the evaluation questionnaire. Participants were asked to rate their level of interest in the training on a scale of 1 to 4 (1 being a low
62
level and 4 being a high level) for each section of the Induction course, namely Day 1 General Introduction, Workplace Environment, Tools & Maintenance, Business Improvement Techniques, ICT, COSHH & Hand Care, Fire Prevention, Risk Assessment, Electrical Safety, Measurements and Materials. The overall results were generally posi-
Learning 2.0
Figure 3. Screenshot of ePortfolio page
tive indicating that the training was sufficiently interesting for the participants. The area of the training that had the highest level of interest was Electrical Safety (Mean = 3.41, SD = 0.59). The areas of training that received the lowest levels of interest were Business Improvement Techniques (Mean = 2.93, SD = 0.65) and the General Introduction on the first day (Mean = 2.93, SD = 0.80). Participants were also asked to rate whether they found the training sufficiently challenging (1 = lowest, 4 = highest). Electrical Safety was rated as the most challenging (Mean = 3.29, SD = 0.60) while the General Introduction on the first day was rated as the least challenging (Mean = 2.75, SD = 0.71). The overall approachability of the instructors was rated highly by the participants in all areas. The average rating for approachability across all areas was Mean = 3.46, SD = 0.86 with a range of 3.32 to 3.59 out of 4. Participants were asked to rate how much they believed that their knowledge improved in each
of the areas. The area rated as having the highest level of knowledge improvement was COSHH (Control of Substances Hazardous to Health) & Hand Care (Mean = 3.51, SD = 0.60). The area rated as having the least level of knowledge improvement was the General Introduction (Mean = 3.17, SD = 0.68). Participants were also asked to rate how appropriate they considered the duration of the activities to be. Overall the results were not particularly positive. The most appropriate rated duration was for Electrical Safety (Mean = 3.15, SD = 0.76) and the least appropriate rated duration was for the General Introduction (Mean 2.75, SD = 0.74). Participant ratings for the helpfulness of teaching aids were generally positive. The highest rating for teaching aids was in the area of Electrical Safety (Mean = 3.49, SD = 0.51) and the lowest rating was in the Helpful Work Place Environment teaching area (Mean = 3.22, SD = 0.52). Participants were also asked to rate if Moodle helped in each of the teaching areas. Table 5 shows the
63
Learning 2.0
rankings of perceived help Moodle provided in each of the teaching categories. Overall, the results of the evaluation questionnaire were generally positive. Electrical Safety consistently achieved the highest ratings for level of interest, challenge, appropriate duration and perceived help provided by Moodle. The General Introduction on the first day received the highest amount of criticism primarily because of its duration as it was considered by participants to be too long and lacked participation and interaction. Participants provided some of the following qualitative comments regarding the duration of the General Introduction: Day 1 was not that interesting because there was not a lot of participation - it was all listening. I think the 3 day introduction as a whole could be delivered as a participation lesson. The first day was not interesting enough. We were just thrown information and didn’t have enough time to absorb the information. Duration - induction was too long and it could involve the trainees more to make it more exciting.
Participants also provide qualitative points for action and improvement. These primarily included: more interaction, more participation and the reduction of the duration of the General Introduction.
GROUP PROJECT The group project was designed to develop and promote team-working skills that are applicable to real-life working in the engineering industry and is the last assessment the students take before going out into industry. The overall objective of the project is for team members to build a truck within a nominal budget. Each member of the team is assigned a role, for example, project manager, chairman, secretary or quality control. To trial Moodle in the group project, it was decided to introduce a wiki for formal recording of the project, a forum to facilitate communication when not all members of the team were present and an assignment folder to allow candidates to upload all documentation for assessment. Two tests were designed: a pre-test to help identify current knowledge and a post-test to help identify new knowledge and students’ views of the wiki and forum. A total of 43 candidates participated in the
Table 5. Ratings of help provided by Moodle in each teaching category Teaching Area
64
Rank
Mean
Standard Deviation
Risk Assessment
1st
3.56
0.50
Electrical Safety
1st
3.56
0.55
Materials
1st
3.56
0.55
Measurements
2nd
3.51
0.60
Work Place Environment
3rd
3.49
0.51
Tools and Maintenance
3rd
3.49
0.55
Fire Prevention
3rd
3.49
0.55
COSHH and Hand Care
4th
3.46
0.50
Business Improvement Techniques
5th
3.44
0.59
ICT
6th
3.41
0.59
Day 1General Introduction
7th
3.33
0.58
Learning 2.0
group project. The candidates were divided into 6 teams with 7-8 students per team. The findings are discussed below.
If someone comes up with an idea then they can post it at anytime even when team members are not there.
Pre-Project Test
You can have access when you are off and record progress.
33 participants completed the pre-test questionnaire. All of the respondents were male. The mean age of participants was 17.18 (SD = 0.73) with a range from 16 to 19.8. The majority of participants (30, 91%) indicated that they had been involved in group work before, while 3 (9%) had not. The majority (27, 82%) of the participants who had participated in group work before indicated that they took part in the group work at school, 10 (30%) at college and 3 (9%) at work. None of the participants had ever used a wiki before. The majority of the participants (24, 73%) described a wiki as an ‘an online collaboration tool with built in tracking’, 7 (21%) described a wiki as ‘a social networking tool’ and 2 (6%) described it as ‘an online instant messaging tool’. The majority of participants (21, 64%) indicated that they had used an online forum before while 12 (35%) had not. 18 (55%) participants had used an online forum for social use, 5 (15%) had used an online forum for a school project, 2 (6%) had used one for a college project and as part of a work project. 12 (36%) of participants believed that an online forum was ‘a social networking tool’, 14 (42%) believed that it was ‘an online chat room’ and 4 (12%) believed that it was ‘an online collaboration tool with built in tracking’. 23 (70%) of participants believed that the use of a wiki 24x7 would encourage teamwork, while 10 (30%) believed this not to be the case. Those who indicated that a wiki would encourage team work gave some of the following reasons: Allows you to keep every member of the team up to date.
Those who indicated that a wiki would not encourage teamwork gave some of the following reasons: Better to discuss the project in person so that you are able to give them a clearer picture of what is done. The whole team would have to use it to benefit and they were not. 22 (67%) of participants believed that the use of a forum would help open up communication between members while 11 (33%) believed this not be the case. Those indicating that the forum would open up communications gave the some of the following reasons: If one team member is not in one day he/she can then see what his/her team has done that day. It can get everyone involved. 17 (52%) of the participants were not aware of any project management tools, while others listed Excel, Word, PowerPoint, efacts, wiki, graphs, flip charts, CAD, Gantt charts and Moodle as project management tools. The majority (22, 67%) indicated that they had not used any project management tools in the past while 11 (33%) had. The majority of participants (21, 64%) indicated that they thought a Project Route Map to be ‘a tool that helps to identify all areas of the project that need to be considered’, 11 (33%) believed it to be ‘a tool that is used to help plan the project and then consulted on a regular basis’ and 1 (3%) believed it to be ‘a tool that is only used to plan
65
Learning 2.0
the project but then filed away’. Participants were asked to list the project resources that they were aware of, 12 (36%) indicated that they were not aware of any, 3 (9%) indicated that they were aware of Moodle as a project resource; 4 (12%) of participants also mentioned the Internet and books as a project management resource. 18 (55%) of participants believed a budget to be ‘a set amount of money that can be spent on the project from start to finish’, 14 (42%) of participants believed that a budget was ‘an amount of money that should cover all cost of the project from start to finish’ and 1 (3%) believed that is was ‘the amount of money that is set for the first stage of the project but is ‘topped up’ as the project progresses’. 14 (42%) of participants identified the information pyramid as something that ‘allows the presenter to identify between the different types of information ensuring that the most important information is not omitted’, 10 (30%) as something that ‘helps the presenter to divide the information into equal parts’ and 9 (27%) as something that ‘helps the presenter to ensure that the correct information is conveyed during a presentation’.
Post-Project Test 26 participants completed the post-test. Participants were asked how much they agreed with the following statement: ‘everyone participated in the group project’. To get an overall rating of the perception of overall participation, strongly agree was recorded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean rating for perceived participation was 2.69 (SD = 1.19) with a range from 1 to 5. This indicates that the level of perceived participation was not particularly high with the majority of participants either disagreeing 11 (42%) or strongly disagreeing 4 (15%) that everyone was participating in the group activity. 13 (50%) of participants described a wiki as ‘an online collaboration tool with built in tracking’, 8 (31%) described it as ‘a social networking
66
tool’, 2 (8%) described it as ‘an online chat room’ and ‘an online instant messaging tool’. 10 (38%) identified a forum as ‘an online collaboration tool with built in tracking’, 7 (27%) identified it as ‘a social networking tool’, and ‘an online chat room’. 2 (8%) described it as ‘an instant messaging tool’. The majority of participants 15 (58%) said that they did not think that a wiki promoted teamwork, while 10 (38%) believed that it would promote teamwork. Those who indicated that the wiki would promote teamwork gave some of the following reasons: People could catch up on work they may have missed. Helped each other with the difficulties. Those who indicated that the wiki would not promote teamwork gave some of the following reasons: Took too long to complete and was repetitive. We didn’t use it much. Participants were asked how much they agreed with the following statement ‘if the wiki was a compulsory element of the group project all team members would have used it’. To produce a mean score of the expected participation level, strongly agree was recoded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean rating was 2.76 (SD = 1.03) with a range of 1 to 4. This means that approximately 50% of participants would use the wiki if it were a compulsory part of the group project. 13 (50%) of participants believed that the forum helped with communication and 13 (50%) did not. Those who believed that the forum helped with communication gave some of the following reasons: All people can communicate if not present.
Learning 2.0
People could see things they missed. Those who believed that the forum did not help with communication gave some of the following reasons: We didn’t use it much. Easier to talk. Participants were asked to rate their level of agreement of this statement: ‘If the forum was a compulsory element of the project, all team members would have used it more for communicating with those team members who are not present on a full-time basis’. To produce a mean score of the expected participation level, strongly agree was recoded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean score rating of expected participation was 3.23 (SD = 0.95) with a range from 1 to 5. This indicates that the forum was more popular than the wiki in terms of expected participation. Participants were asked to list all of the project management tools that they were now aware of, 6 (23%) of participants mentioned a wiki, 8 (31%) answered ‘none’, 2 (8%) mentioned Gantt charts and 1 (4%) mentioned Moodle. 13 (50%) of participants now identified a Project Route Map as: ‘a tool to help identify areas of the project that need to be considered’, 2 (8%) identified it as ‘a tool used to plan the project and is then filed away’ and 11 (42%) identified it as ‘a tool that is used to identify areas of the project that need to be considered and consulted on a regular basis’. Participants were asked to list all of the project resources that they were familiar with now that the project was complete: 6 (23%) of participants mentioned a ‘wiki’, 11 (42%) said ‘none’ and a small number mentioned talks, PowerPoint, Gantt charts and Moodle. The majority of participants (14, 54%) identified a budget as ‘an amount of money that can be spent on the project to cover all project costs from start to finish’, 11 (42%)
identified it as ‘a set amount of money that can be spent on the project from start to finish’, and 1 participant (4%) identified it as ‘the amount of money that is allocated to material for the project’. 6 (23%) of the participants identified the information pyramid as ‘a tool that helps the presenter to divide the information into equal parts’, 9 (35%) as ‘a tool that helps the presenter to ensure that the correct information is presented during the presentation’ and 11 (42%) as ‘a tool that helps the presenter identify between the different types of information ensuring that the most important information is not omitted’.
Comparison of Pre and Post-Tests 21 participants completed both the pre- and the post-tests. A Wilcoxon Signs ranked test indicated that the increase in knowledge was not significant with regards to wikis (Z = -.890, p = 0.03) and forums (Z = -.816, p = 0.414). However, when combined together, a Wilcoxon Signs ranked test indicated that there was a significant decrease in forum and wiki knowledge (Z = -1.807, p = 0.04). The mean in the pre-test was 1.14 out of 2 (SD = 0.65) and the mean in the post-test was 0.81 out of 2 (SD = 0.68). This was primarily due to a significant reduction in wiki knowledge after the training (Z = -.890, p = 0.03). This is possibly due to the fact that the instructors did not promote the wiki as it was not compulsory and instead suggested that the students used the forum to maintain the project documentation. A Wilcoxon Signs ranked test indicated that the increase in knowledge was not significant with regards to project route maps (Z = -1.000, p = 0.317), the information pyramid (Z = -.378, p = 0.705) and budgets (Z = -1.265, p = 0.206). However, when combined together there was a significant increase in the project management knowledge between the pre-test and post-test (Z = -1.615, p = 0.05). The mean in the pre-test was 1.10 out of 3 (SD = 0.94) and the mean in the posttest was 1.42 out of 3 (SD = 0.87). A significant
67
Learning 2.0
correlation was detected between undertaking group work previously and the knowledge score for budget in the pre-test (r = -.414; n = 21; p = 0.03). There was also a significant correlation between undertaking group work previously and the knowledge score for forums in the post-test (r = -.459, p = 0036). A significant correlation between using project management tools previously and knowledge scores about wikis in the post-test (r = -.539, p = 0.01) was also detected. In addition, there was a significant correlation between using project management tools previously and knowledge about Project Route Maps in the post-test (r = -.408, p = 0.03). While some of these correlations were expected, some of the others were not and these will be tested again with subsequent cohorts going through the project module.
I am keen to start uploading my lectures and restructure my lessons to incorporate Moodle All courses could be delivered online but investment in teaching material, software and manpower would have to be made to progress the project Reinforcement of the tutor lead sessions and the ability to access the information from anywhere in the centre It gives those trainees who are self motivated the opportunity to keep pace or catch Groups were able to share information with team members and use it as a note pad for ideas
It is limited to providing a storage facility for course material with the odd quiz to reinforce training
There was a definite divide between the instructors. Some could identify the support that Moodle and the Web 2.0 tools could bring to the training programme, whist others did not. Part of this could be attributed to the traditional nature of engineering; however, the instructors who were new to the company were more open to new ideas. The students who were working under the supervision of those instructors who promoted the use of the technologies were far more receptive and willing to use Moodle and the Web 2.0 tools. However, the students who were supervised by those instructors who did not encourage the use of the technologies were far more resistant to their use. This demonstrates that there must be enthusiasm and guidance in the use of technology to ensure its successful uptake. It should be noted that these issues were brought to the attention of management by the authors, but management felt they could not force instructors to use the technologies.
The wiki only helped to confuse matters in this short space of time
CONCLUSION
Some of the positive comments made by instructors were as follows:
This chapter set out to discuss the impact on learning with the introduction of a LMS into a
INSTRUCTORS RESULTS Instructors were surveyed to obtain feedback on their opinion of the Web 2.0 tools used during the group project, to gauge their acceptance of Moodle, and to identify areas where they believed the LMS could be utilised further. The course is a practical course and would not lend itself well to online delivery It was long winded and the trainees had to build it themselves, the wiki would be ideal for large academic projects that spans a university course or is used in a notational sense
68
Learning 2.0
vocational training course and to answer (i) can technology supplement the hands-on nature of vocational training, (ii) can the use of LMS and Web 2.0 tools (specifically wikis and forums) aid training, and (iii) can the pilot be considered a success? Much of the work carried out in this sector is hands-on and on-the-job training but it also consists of a substantial knowledge element that would lend itself well to the use of eLearning. From the literature review provided within this chapter, it may be argued that the use of educational technologies for practically-based courses (e.g. vocational training) should be a relatively straight forward process. Early eLearning was used in a practical capacity; however, the case study demonstrates that the introduction of learning technologies into a vocational training course was received with mixed reviews. This could largely be accredited to the traditional nature of engineering training but what was obvious from the data collected and by the authors’ experience in working with the organisation, much of the opposition to the technology use was due to the lack of flexibility and understanding from the instructors. From the analysis it can be seen that the technology was received with a mixed response. The online induction course was well received by the students (and was also accepted by the instructors). Unfortunately, the use of wikis and forums for the project module, while well received by the students was not fully adopted by the instructors, possibly as they were not confident of their knowledge and use of the technologies. As a result, they made the use of the wiki optional and instead asked the students to use the Moodle forum to maintain their project documentation. This caused the students to become confused as to the distinction between forums and wikis, resulting in a decrease in learning. The authors conclude that the research demonstrates that educational technologies could potentially aid the vocational training of apprentices and that this area could offer a rich source of future research. However, it is early days to design a framework that will aid practitioners in
the successful implementation of a LMS into a private vocational training organisation but as the empirical data grows this should become possible. At this stage the authors suggest, tentatively, that eLearning can support and in some instances replace some elements of a vocational programme. The authors acknowledge that the practical element would be very difficult to replace with technology but it could be supplemented through other tools such as multimedia and even games technology. Educational technologies offer the opportunity to open up learning in the vocational training sector, rather than restricting it to the traditional 9-5 scenario. The authors would recommend that those educators who deliver traditional vocational training courses be fully consulted during the implementation process as well as receive the necessary training needed to highlight the potential benefits that technology could bring to the overall course being delivered. If buy-in from students can be obtained and instructors can learn to appreciate the value that could be gained from educational technology then its use could diffuse throughout the vocational training sector.
ACKNOWLEDGMENT This project received financial support from the Knowledge Transfer Partnerships programme (KTP). KTP aims to help businesses to improve their competitiveness and productivity through the better use of knowledge, technology and skills that reside within the UK Knowledge Base. KTP is funded by the Technology Strategy Board.
REFERENCES Beetham, H. (2009). E-portfolios in post-16 learning in the UK: Developments, issues and opportunities. Retrieved October 2, 2009, from http://www.jisc.ac.uk/uploaded_documents/eportfolio_ped.doc
69
Learning 2.0
Berlanga, A. J., Sloep, P. B., Brouns, F., BitterRijpkema, M. E., & Koper, R. (2008). Towards a TENCompetence e-portfolio. [iJET]. International Journal of Emerging Technologies in Learning, 3, 24–28. City & Guilds. (2009). How are centres using ePortfolios. Retrieved November 12, 2009, from http://www.cityand guilds.com/44912.html Connolly, T. M. & Stansfield, M. H. (2007a). From e-learning to games-based e-learning. International Journal of Information Technology and Management, 26(2/3/4), 188-208. Connolly, T. M., & Stansfield, M. H. (2007b). From e-learning to online games-based e-learning: Implication and challenges for higher education and training. In Li, F. (Ed.), Social implications and challenges of e-business. Hershey, PA: IdeaGroup Publishing. Curyer, S., Leeson, J., Mason, J., & Williams, A. (2007). Developing e-portfolios for VET: Policy issues and interoperability. Australian flexible learning framework. Retrieved October 16, 2009, from http://e-standards.flexiblelearning.net.au/ docs/vet-eportfolio-report-v1-0.pdf DeRouin, R. E., Fritzsche, B. A., & Salas, E. (2005). E-learning in Organizations. Journal of Management, 31(6), 920–940. doi:10.1177/0149206305279815 Dorninger, C., & Schrack, C. (2007). E-portfolio’s in education-learning tools or means of assessment? In Proceedings of the International Computer-Aided Learning Conference (ICL2007), Villach, Austria. EDUCAUSE. (2005).Retrieved on February 11th, 2011, from E-Portfolios Web page, URL: http://www.educause.edu/ELI/Archives/EPortfolios/5524
70
Franklin, T., & Harmelen, M. (2007). Web 2.0 for content for learning and teaching in higher education. Joint Information Systems Committee (JISC) report. Retrieved August 8, 2010, from http://ie-repository.jisc.ac.uk/148/1/web2content-learning-and-teaching.pdf Garrison, R. (1997). Computer conferencing: The post-industrial age of distance education. Open Learning, 12(2), 3–11. doi:10.1080/0268051970120202 Gerring, J. (2007). Case study research: Principles and practices. Cambridge, UK: Cambridge University Press. Grosseck, G. (2009). To use or not to use Web 2.0 in higher education? Procedia Social and Behavioral Sciences, 1, 478–482. doi:10.1016/j. sbspro.2009.01.087 Gunawardena, C. N. (1993). The Social context of online education. In Proceedings of the Distance Education Conference, Portland, Oregon. Holmes, B., & Gardner, J. (2006). E-learning concepts and practice. London, UK: SAGE Publications. Horton, W. (2003). E-learning tools and technologies: A consumer’s guide for trainers, teachers, educators, and instructional designers. Indianapolis, IN, USA: Wiley. JISC. (2008). Effective practice with e-portfolios: Supporting 21st century learning. Joint information systems committee. London, UK: JISC. Jonassen, D. H. (1996). Computer-mediated communication: Connecting communities of learners. Computers in the Classroom (158–182). Edgewood Cliffs, NJ: Prentice-Hall, Inc. Kozma, R. (1987). The implications of cognitive psychology for computer-based learning tools. Educational Technology, 27(11), 20–25.
Learning 2.0
Lorenzo, G., & Ittelson, J. (2005). An overview of ePortfolios, Educause learning initiative. Mason, R., Pegler, C., & Weller, M. (2004). Eportfolios: An assessment tool for online courses. British Journal of Educational Technology, 35(6), 717–727. doi:10.1111/j.1467-8535.2004.00429.x McBride, R., & Schostak, J. (1994). What is qualitative research. Retrieved May 7, 2009, from http://www.enquireylearning.net/ELU/Issues/ Research/Res1Ch1.html McDonald, J. (2002). Is “as good as face-to-face” as good as it gets? Journal of Asynchronous Learning Networks, 6(2), 10–23. McKelvie, G., Dotsika, F., & Patrick, K. (2007). Interactive business development, capturing business knowledge and practice: A case study. The Learning Organization, 14(5), 407–422. doi:10.1108/09696470710762637 Moodle. (2010). Moodle Statistics. Retrieved August 25, 2010, from http://moodle.org/stats/ Richardson, H. C., & Ward, R. (2005). Developing and implementing a methodology for reviewing e-portfolio products, The centre for recording achievement (CRA). Retrieved October 10, 2009, from http://www.jisc.ac.uk/uploaded_documents/ epfr.doc Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students (4th ed.). Pearson Education Ltd. Strivens, J. (2007). A survey of e-pdp and ePortfolio practice in UK higher education. UK: Higher Education Academy. Wenger, E. C. (2000). Communities of practice and social learning systems. Organization, 7(2), 225–246. doi:10.1177/135050840072002 Wenger, E. C., & Snyder, W. M. (2000). Communities of practice: The organizational frontier. Harvard Business Review, 78(1), 139–145.
Whisker, G. (2001). The Postgraduate Research Handbook. Basingstoke, UK: Palgrave MacMillan.
ADDITIONAL READING Agostini, A., De Michelis, G., & Loregian, M. (2009). Using blogs to support participative learning in university courses. Int. J. Web Based Communities, 5(4), 515–527. doi:10.1504/ IJWBC.2009.028087 Baxter, G. J., Connolly, T. M., & Stansfield, M. (2009). The use of blogs as organisational learning tools within project-based environments. Int. J. Collaborative Enterprise, 1(2), 131–146. doi:10.1504/IJCENT.2009.029285 Berge, Z. L., & Muilenburg, L. Y. (2005). Student Barriers to Online Learning: A factor analytic study. Distance Education, 26(1), 29–48. doi:10.1080/01587910500081269 Cartelli, A., Maillet, K., Stansfield, M. H., Connolly, T. M., Jimoyiannis, A., Magalhaes, H., & Toland, J. (2008). Identifying and promoting best practice in virtual campuses and e-searning, ED-MEDIA 2008 Conference - World Conference on Educational Multimedia, Hypermedia & Telecommunications, Vienna, Austria (30 June-4 July 2008) (pp. 897-902). Chapman, C., Ramondt, L., & Smiley, G. (2005). Strong community, deep learning: Exploring the link. Innovations in Education and Teaching International, 42(3), 217–230. doi:10.1080/01587910500167910 Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52(1), 141–146. doi:10.1016/j.compedu.2008.07.003
71
Learning 2.0
Connolly, T. M., & Stansfield, M. H. (2007). Developing constructivist learning environments to enhance learning. In N. Buzzetto-More (Ed.), Principles of effective online teaching: A handbook for experienced teachers developing e-learning, 19-38.
Lorenzo, G., & Ittelson, J. (2005). An overview of e-portfolios. D.Oblinger (Ed.), Educause learning initiative paper (pp. 1-28). Retrieved October 18, 2010, from: http://www.sorteoudla.org.mx/ promueve/ciedd/CR/tecnologia/AnOverviewofEPortfolios.pdf
Gerrard, C., Connolly, T. M., & Stansfield, M. (2006). The role of staff development to enhance the integration of e-learning within the HE curriculum, European Conference on e-Learning (ECEL) 2006, 11-13 September 2006, University of Winchester, UK.
Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4), 134–148.
Grippa, F., & Secundo, G. (2009). Web 2.0 projectbased learning in higher education: some preliminary evidence. Int. J. Web Based Communities, 5(4), 543–561. doi:10.1504/IJWBC.2009.028089 Hall, H., & Davison, B. (2007). Social software as support in hybrid learning environments: the value of the blog as a tool for reflective learning and peer support’. Library & Information Science Research, 29(2), 163–187. doi:10.1016/j. lisr.2007.04.007 Illeris, K. (2002). The Three Dimensions of Learning: Contemporary learning theory in the tension field between the cognitive, the emotional and the social. Copenhagen: Roskilde University Press. Kim, S. W., & Lee, M. G. (2007). Validation of an evaluation model for learning management systems, Journal of Computer Assisted Learning, Blackwell Publishing Ltd. Larusson, J. A., & Alterman, R. (2009). Wikis to support the “collaborative” part of collaborative learning. Computer-Supported Collaborative Learning., 4(4), 371–402. doi:10.1007/s11412009-9076-6 Lawless, N., & Allan, J. (2004). Understanding and reducing stress in collaborative e-learning. Electronic Journal of e-Learning, 2(2).
72
Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books. Schön, D. A. (1987). Educating the reflective practitioner: Towards a new design for teaching in the professions. San Fransisco, CA: Jossey-Bass Inc. Shin, J., & Bickel, B. (2008). Communities of practice: Creating learning environments for educators (Kimble, C., & Hildreth, P., Eds.). Information Age Publishing. Stansfield, M. H., & Connolly, T. M. (Eds.). (2009). Institutional transformation through best practices in virtual campus development: Advancing e-learning policies organizations. Hershey, PA: IGI Global Publishing. doi:10.4018/978-160566-358-6 Stansfield, M. H., & Connolly, T. M. (2009). An exploration into key issues relating to the adoption of good practices in e-learning and virtual campuses. In Mayes, T., Morrison, D., Mellar, H., Bullen, P., & Oliver, M. (Eds.), Transforming higher education through technology enhanced learning. Higher Education Academy. Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice. Boston, MA: Harvard Business School Press.
Learning 2.0
Williams, J. B., & Jacobs, J. (2004). Exploring the use of blogs as learning spaces in the higher education sector. Australasian Journal of Educational Technology, 20(2), 232–247.
KEY TERMS AND DEFINITIONS Communities of Practice (CoPs): Groups of people informally bound together by shared expertise and passion for a joint enterprise (Wenger & Snyder, 2000, pp. 139). E-Portfolios: A valuable online tool that learners, faculty, and institutions can use to collect, store, update, and share information. E-Portfolios allow students to reflect on their learning, communication with instructors, document credentials, and provide potential employers with examples of their work (EDUCAUSE, 2005). eLearning: Any use of Web and Internet technologies to create learning experiences (Horton, 2003, pp. 13).
LMS: The components in which learners and tutors participate in online interactions of various kinds, including online learning (Becta, 2003) Portfolio: A collection of documents relating to a learner’s progress, development, and achievements (Beetham, 2009). Social Software: Social software is a community driven technology which facilitates interaction and collaboration and depends largely on social convention (McKelvie et al., 2007). Web 2.0: The social use of the Web which allow[s] people to collaborate, to get actively involved in creating content, to generate knowledge and to share information online (Grosseck, 2009, pp. 478). Wiki: A website (or other hypertext document collection) that allows users to add content, as on an Internet forum, but also allows anyone to edit the content (G. Avram, 2006)
73
Section 2
Implementing and Evaluating
75
Chapter 4
Evaluations of Online Learning Activities Based on LMS Logs Paul Lam The Chinese University of Hong Kong, Hong Kong Judy Lo The Chinese University of Hong Kong, Hong Kong Jack Lee The Chinese University of Hong Kong, Hong Kong Carmel McNaught The Chinese University of Hong Kong Hong Kong
ABSTRACT Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized Web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University.
INTRODUCTION Learning management system (LMS) is a broad term that is used for a wide range of systems that organize and provide access to online learning services for students, teachers and administrators. These services usually include access control,
provision of learning content, communication tools, and organizations of user groups (Paulsen, 2002). Jovanovic et al. (2007) defined an LMS as “a software environment that enables interactive web-based teaching and supports administration of distance courses, allowing instructors to distribute information to students, producing course mate-
DOI: 10.4018/978-1-60960-884-2.ch004
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
Evaluations of Online Learning Activities Based on LMS Logs
rial, preparing assignments and tests, engaging in discussions, and managing courses and distance classes” (p. 46). In 2005, 95% of all higher-education institutions in the UK were using an LMS (Browne, Jenkins, & Walker, 2006). At The Chinese University of Hong Kong (CUHK), two LMS (WebCT and Moodle) are centrally supported. Indeed, the majority of the University’s eLearning activities are supported by these central services; apart from one faculty (Engineering), there are relatively few non-centrally-hosted course websites. Effective record-keeping, and extraction and interpretation of eLearning logs, can reveal valuable information on how these LMS are used to facilitate teaching and learning. As noted by Sen, Dacin and Pattichis (2006), the use of logs for tracking user activities is quite common in commercial settings where customer habits and trends are traced and monitored. Reading user logs also applies in educational settings – for example, the study by Black, Dawson and Priem (2008) on how to obtain measures of ‘community’ in online courses. In universities with centralized web-based teaching and learning systems, the logs can be monitored through inbuilt mechanisms to track and record a certain amount of information about online activities. Colace and De Santo (2003) commented that monitoring an LMS can enable detailed and useful information on the LMS’s utilization and efficacy. This information can include trend data if the logs have been collected regularly over time. Such information can provide the basis for various decisions related to the implementation and promotion of eLearning. However, these inbuilt web-log tracking systems do not normally provide institution-level data. The weblogs are reported in an interface designed for individual teachers to get a summary of activities recorded in individual courses (Mazza & Milani, 2005) rather than for analyses of online learning activities at higher levels (e.g. department, faculty or institution). Retrieval of
76
data in WebCT (version 3.8) is even more challenging. As it does not employ a database structure, these records or logs of activities can be extracted using the provided display logs functions which are very limited in functionality. The locations where the record information is stored are not clear because of the lack of a database structure, and so time and effort are needed for: 1) testing through trial and error for the allocation of the desired information; 2) checking whether the data are accurate; and 3) developing software to extract the information for all courses in the University. After retrieval of information, additional work is needed to standardize and automate the integration, interpretation and reporting processes of the log data so that we have a common ground to compare and contrast eLearning uses over time. Romero, Ventura and Garcia (2008) discussed using data-mining techniques to explore the raw log data of servers in order to understand various aspects of learner activities. However, such strategies, though allowing great flexibility in the topics of study, are technically complex. As noted by Black, Dawson and Priem (2008) “server logs are plagued with a low signal-to-noise ratio: simply preparing the data for modeling can consume 80% to 95% of a project’s resources” (p. 67). A system that is more powerful than the LMS inbuilt activity log systems, and can regularly retrieve and interpret the logs into a number of fixed representations for year-by-year comparison and contrast, seems to be what we need. Zhang et al. (2007) reported a similar system called Moodog which monitor students’ activities live on the Moodle LMS. Our system looks at the issue more from an institutional point of view. We may not need to monitor student activity every moment but retrieve and analyze the logs once every semester. Also, because of our particular context, our system reads and integrates as much as possible of the logs from both of the LMS we host – WebCT and Moodle. Earlier development and the framework of the system were reported in Lam et al. (2006).
Evaluations of Online Learning Activities Based on LMS Logs
The work at that time was more focused on the data-retrieval stage. The present paper extends our discussion to the automation of the data interpretation and reporting processes. As noted in Lam, Keing, McNaught, and Cheng (2006), system logs data can provide information on: 1) popularity; 2) the nature of the functions/ strategies in use; and 3) engagement of teachers and students. 1. The notion of popularity is a very simple yes/no specification for each course in the University whether any eLearning activities are recorded in our logs or not. 2. The nature of the eLearning activities recorded for each web-enabled course refers to uses of forums, assignment-submission service, course-content delivery, online quizzes or surveys, grade-book facility, etc. 3. Engagement reflects how involved teachers and/or students are in these activities. This is the level among the three that convey the finest amount of detail about a site. After the recognition that there is a course website (popularity), more information can reveal the actual features and activities having occurred on the site (nature). After learning about the nature of the website, further information can gauge how active teachers and students are on site (engagement). The data report the actual activities recorded and, to a certain extent, they fit the requirements of the naturalistic research paradigm (Alexander & Hedberg, 1994; Grasha, 1990) that collection of evaluation data is usefully done in non-intrusive ways (Lam, McNaught & Cheng, 2008) and should, where possible, be situated in authentic educational settings (Froehlieh, 1994). We do not claim that weblog data represent a comprehensive solution to all evaluation needs. More comprehensive evaluation studies would consider evidence, both quantitative and qualitative in nature, from a variety of sources. The
present paper serves only to illustrate the advantages as well as limitations to using weblogs as a data source. Apart from relatively easy access to the data, the extraction of system logs is also completely non-intrusive to both teachers and students. Furthermore, repeated measures can be taken for a long period of time so that comparison of usage over time is possible. The largely automatic extraction of the logs (once the software for logs extraction is devised), and the standardized methods in the follow-up analysis, reporting, and interpretation of the data, enable an institution to build a mechanism which is suitable for administration on a regular basis (e.g. annually). A comparable system of logs extraction and analysis can then be used in subsequent monitoring. One limitation of this approach, however, is that it monitors only uses of the web that utilize the central system. Also, it has a bias on quantity rather than quality as logs are numbers rather than a full picture of the educational quality of the content on course websites. Also, not all online activities and teachers’/ students’ engagement in these activities can be effectively recorded by the logs. For example, the availability of course outlines on course websites is an online activity that is of interest to universities. However, ‘online course outline’ is not an activity separately recorded by the logs of WebCT or Moodle. It is impossible to identify course outlines unless the researchers go into individual websites and read all the documents. The picture portrayed by the logs is only a partial representation of the total online learning activities and the engagement in these activities. Even if the logs reveal certain information on popularity of, nature of and engagement with activities in websites, care has still to be taken to understand the exact meanings of these logs based on the characteristics of the platforms and how logs are kept in them. Very often, minor adjustments have to be made or there are decisions to make
77
Evaluations of Online Learning Activities Based on LMS Logs
concerning the cut-off points beyond which the records are deemed to fall into another category. In this paper, we: 1. Explain how the system works. 2. Use empirical evidence recorded from 2007 to 2010 to show how the data could be employed to achieve various levels of analysis; and 3. Discuss how the better understanding of the online activities have enabled decisions in a number of eLearning support initiatives in our University. We consider that the data have assisted in better understanding and refining our eLearning strategies and supports at both an institutional level and faculty/ department level.
HOW THE SYSTEM WORKS The following measures have been taken to refine and standardize the data so that we identified the right types of logs for the types of activities we targeted. Clear definitions are also necessary, particularly in our case where we have merged weblog readings from two different LMS. The standardization makes sure the readings from the two platforms record the same underlying activities and record them in a compatible manner. Not all websites created on the servers should be considered ‘active’ websites; this means that there are some sites that are developed but not made accessible to students. For example, some of the websites might have been created by the teachers or the teaching assistants for testing purposes (e.g. perhaps to be used in the next term) and there were actually no real student activities on the site. In our weblog study, therefore, we took care to isolate only the so-called ‘active’ websites – websites that had at least one student access during the course period would be included in the data. The similar ‘active’ concept was also applied in the study of the other four eLearning strategies
78
in our study. A website that had none of these active features may mean a site that was used by the teacher for only announcing course information such as examination dates or news of events. •
• •
•
Whether or not a particular course used ‘forum’ for ‘active discussion’, for example, was judged not only from the existence of a forum on the site, but rather on whether there was at least one student access to the forum as well. ‘Active online quizzes’ meant quizzes that had at least one student attempt. ‘Active assignment submission’ meant that at least one student submitted work to the platform. ‘Active content’ meant the website contained at least one document (could be PowerPoint, Word or PDF documents, or any other multimedia files) for download, and there was at least one student download recorded in the logs.
The use of eLearning strategies is more meaningful in courses that have considerable class sizes. We have introduced a mechanism to filter cases based on class size. In our web logs study, we only studied classes with a class size equal to or larger than 10. Lastly, it is worth noting that the unit of analysis in our study was normally a course. In most cases, it was relatively easy to decide whether a certain course had used eLearning strategies or not. However, some decisions had to be made in many other cases. Sometimes one course was run in a number of sessions (at different times and even by different teachers). Our standpoint was to consider a course eLearning-enabled IF at least one of the sessions had an active website on any one of the LMS. Figure 1 shows the report interface of the weblog system. The interface is comprised of several parts such as searching filters (Boxes 1 and 2), result window (Boxes 3a and 3b) and graph
Evaluations of Online Learning Activities Based on LMS Logs
Figure 1. Inquiry layout of weblog system
window (Box 4). Once a set of filters is submitted into the search engine, results and a graph will be shown in the result and graph windows. Also, in order to provide the flexibility to enable multiple levels of analysis, different parameters can be set for the filters. For example, we can ask for a report on a certain year/ term or a report that contrasts activities in multiple years or terms. Reports can be on institution, faculty, department, or course levels. We can call for records of undergraduate courses, postgraduate courses, or both. The unit of analysis is normally a course but there is an option to drill down into the use of LMS by the individual sessions of a course (taught by the same or different teachers). Also, the system can report actual numbers (e.g. how many active course websites and how many courses we have in the University) or report percentages (e.g. the percentage of courses that have an active website). These parameters are available for the administrators in the Box 1 area in Figure 1.
In Box 2, administrators can specify the exact year or term to report on, as well as the exact faculty, course type and even the course code the report should be limited to if desired. 2009 on the system means the 2009–2010 academic year which is from September 2009 to May 2010 in our context. In addition, there is a choice about class size, i.e. the minimum student number in a class before it can be included (the default is 10). Boxes 3a and 3b in Figure 1 show the weblog results of the whole University in the year 2009. Both the ‘popularity’ (Box 3a) and the ‘nature of activities’ (Box 3b) types of information are shown. Table 1 further explains the sub-categories in these two areas. Box 4 contains graphical representations of the various numerical data reported in Box 3. They are usually sufficient for normal reporting purposes. However, the data and graphs can be exported as an Excel file for further processing and analysis if needed.
79
Evaluations of Online Learning Activities Based on LMS Logs
Table 1. Definitions of the sub-categories in the report Notion
Popularity
Nature of activities
Sub-categories
Definition/ Description
Course Total
Total number of courses
None Total
Courses without using any LMS
WebCT Total
Used WebCT
Moodle Total
Used Moodle
Multi Total
Used both WebCT and Moodle
Active website (AW)
At least one student access to LMS and view website page during the course period
Active content (AC)
At least one document on the active website for access and at least one student download recorded in the logs
Active discussion (AD)
At least one student posts a topic or a thread
Active assignment (AA)
At least one student submitted work to assignment drop-box
Active quiz (AQ)
At least one student attempted quiz on active website
Figure 2 illustrates how the weblog system represents teachers’ and/or students’ ‘engagement’ in various eLearning activities (called level 3 analysis in the system: Box 5). In this example, the data show the amount of activity recorded in using the various functions over all our active WebCT and Moodle websites in the year 2009.
Box 6 shows the tabs that lead to analyses of various types of activities. In the result-grid region now showing (Box 7), we can tell that among all the courses in the year, 128 of them had a website in which student access over the year was between 1 to 100, 731 of them had 101 to 1000 visits, 858 had 1001 to 10,000 visits and 167 had over 10,000 visits in the year. 1586 courses did not have a
Figure 2. Demonstration of result window reporting engagement data
80
Evaluations of Online Learning Activities Based on LMS Logs
website in any of our two platforms or their websites did not record any student visits at all. Similarly, data and graphs concerning engagement in the various activities can be exported as an Excel file for further processing and analysis.
VARIOUS LEVELS OF ANALYSIS The flexibility of the system allows us to conduct various levels of analysis easily. Six main types are distinguished in our framework as shown in Figure 3. The framework is an extended version of the one reported in Lam et al. (2006) as a result of enhanced analytical power through recent developments. In brief, the system provides a general institution-wide overview of eLearning activities (Types A, B and C analysis in Figure 3); it also
has the capability to differentiate faculty- or department-level eLearning practices (Types D, E and F analysis in Figure 3). Type A refers to how popular LMS are across the whole University. Type B refers to the nature of the features used in the LMS. Type C refers to how students and/ or teachers in general are engaged in the various activities. Type D refers to how popular LMS are used in a certain faculty/ department or even a certain course. Type E refers to the analysis of how various eLearning functions are used in the different faculties/ departments or courses. Lastly, Type F analysis distinguishes the engagement in activities by students and/or teachers in various faculties/ departments. Below, we use empirical evidence recorded from the years 2007 to 2009 to illustrate how the data could be employed to achieve these various levels of analyses. We looked at all undergraduate
Figure 3. Model of the monitoring mechanism through system logs
81
Evaluations of Online Learning Activities Based on LMS Logs
and postgraduate courses that had class sizes of 10 or above at our University. These illustrations are not meant to be a comprehensive discussion of the eLearning activities in our University. Only a few key features are discussed. We also only report up to the faculty level, rather than drilling into what happened in individual departments or even courses. The actual names of the faculties are also hidden as the interest of the present paper is not on detailed disciplinary differences. The data represent a few snapshots of some of the analyses the system allows us to do that, as will be discussed later, reveal useful information about the actual use and usage of the LMS in our University.
Popularity of LMS Across the Whole University Figure 4 shows the overall websites built on the WebCT and the Moodle platforms across three years.
The overall impression is that an increasing number of courses began to have a web presence in our LMS. The courses that had a website in one or both of our LMS grew steadily from around 42% (1432 active websites) in 2007, to 46% (1555 active websites) in 2008 and then to over 54% (1891 active websites) in 2009. More websites were built on the WebCT platform because Moodle was not introduced at CUHK until 2007. Nevertheless, the increase in Moodle use over the years is impressive.
Comparison of Two Functions What are the common functions used on these sites? Data show that the main use of these websites was for content delivery. The more interactive functions such as quizzes and discussion were used less. We illustrate this large contrast by highlighting the use of ‘active content’ and ‘active quiz’ in three years in Figure 5. Over 90% of the active websites contained some sorts of content (various types of files such
Figure 4. Usage of an LMS in CUHK courses in the academic years 2007–2009
82
Evaluations of Online Learning Activities Based on LMS Logs
Figure 5. LMS functionality used in the academic years 2007–2009
as doc, PowerPoint or PDF) which have been accessed by students in the course of study. In contrast, only a small percentage of courses had employed online quizzes as an active strategy. The percentage of quiz-containing websites actually decreased quite significantly over the years. In fact, low use was also observed for other interactive functions such as ‘active discussion’ and ‘active assignment’. On average, active-online discussions were found only in about 21.5% (2007), 20.9% (2008) and 14.8% (2009) of the total websites respectively. Also, 27.6% of the websites used the active-assignment feature in 2007. It gradually decreased to 24.0% in 2008 and then to 14.6% in 2009.
Engagement in Four Areas Figure 6 shows students’ visits to the websites and their engagement in some of the website functions in 2009. The data first of all confirmed that LMS are actively used by students. The ‘active website’ analysis showed that more than 15,000 students (over 90% of the total student population) visited one or more of the websites. Many of the students paid quite frequent visits (more than 30% of the students accessed the LMS more than 20 times
during the year). A few even visited the websites over 1000 times. However, the data also showed that students did not visit the websites for the interactive functions. Only 1995 students (recalling that over 15,000 students ever visited the LMS that year) had written anything on any of the online forums (many of these 1995 students wrote only one or two postings). Similarly, relatively few students used the assignment submission function and the quizzes. Many of the students who used the quizzes, however, had completed multiple online quizzes. 4742 students used quizzes and about one-third of them attempted 11–20 quizzes on the LMS. One-tenth even made 20 times or more attempts.
Use of LMS in Four Faculties Different faculties used LMS quite differently. ELearning experience was not the same across disciplines. Figure 7 shows the use of the two LMS in four of our eight faculties. First of all, we observed that an increasing number of courses had a course website and this was true in all of the faculties. This finding is in line with the general picture portrayed on Figure 4. However, the interest in using LMS
83
Evaluations of Online Learning Activities Based on LMS Logs
Figure 6. Frequency of usage of function within the LMS (WebCT) in 2009
varied quite a lot from one faculty to another (e.g. over 90% of courses in Faculty B versus around 50% to 60% in the other three faculties). It is also interesting to note that students were likely to be told to use two LMS because some of the teachers used WebCT while some used Moodle even if they were in the same faculty. This was particularly true in Faculty A where the preferences of WebCT and Moodle by teachers were roughly half and half. Using two different LMS and getting used to the navigation and controls of two systems might have imposed considerable confusion and unnecessary workload on students.
Use of Three Functions in Four Faculties Similar to the University-wide phenomenon as portrayed on Figure 5, using LMS for content delivery was also a common practice across the
84
four faculties inspected (Figure 8). Most of the websites (around 90% or more) in these faculties had active content that had been accessed by students in the respective years. This content-delivery focus was similar among the faculties. What makes faculties different from each other is their use of the other functions of LMS. In Faculty B, we found the use of online discussion a popular strategy (nearly half of the courses used it) in 2007 but the enthusiasm of using this function then dropped significantly (dropped to only about 5% in 2009). The interest of teachers in this faculty in the use of online quizzes also dropped to zero in the period. On the contrary, the use of online discussion in Faculty A and Faculty C courses has been relatively stable. Comparatively, teachers in Faculty D never seemed to have any great interest in online communication.
Evaluations of Online Learning Activities Based on LMS Logs
Figure 7. Use of LMS in different faculties (A–D)
Faculty C impressed us by their interest in using the quiz function. Percentage of courses having quizzes in this faculty has been well above the University mean all the years. Based on personal communications the authors have had with teachers in these faculties, there are many factors involved in explaining these observed differences. Faculty B, for example, was newly established in 2007 and then had significant expansion in teacher and student numbers in the three-year period. The faculty boosted the use of eLearning strategies in the early years but the focus was quickly shifted to other matters as more and more new teaching staff members were recruited and teaching loads became heavy. Faculty C is a professional discipline which requires students to remember and understand significant numbers of terms and information. Online quizzes seem to be a useful tool to facilitate factual learning so that classroom time can be used for activities that require higher-order thinking. Faculty A focuses on social and humanities subjects where discussion may be a more suitable online activity than quizzes. Topics in Faculty D centre on established laws and formulae and there appears to a perception that online discussions are not needed. The use of the online discussion function in the websites of this faculty has been decreasing over the years.
Engagement in Two Functions in Two Faculties Figure 9 selectively looks at two online activities in two of the faculties. It shows that, as expected, students’ engagement in content-oriented uses was similar but their involvement in other online activities could differ from one faculty to another, basically reflecting the uses of these functions as indicated in Figure 9. More than half of the websites in both Faculties A and B had 16 or more content files that were accessed by students in 2009. Websites were content-heavy and accessing of the content by students was popular in both faculties. Their use of the quiz function was, however, quite different. Websites in Faculty C obviously had a higher concentration of quizzes in them: over 10% of the websites contained 6 or more quizzes that have been actively used by students in the year. In general, thus, there is a ‘cultural’ difference in eLearning use in different faculties; perhaps not so much in using the web for content but much more when using more interactive functions.
85
Evaluations of Online Learning Activities Based on LMS Logs
Figure 8. Use of LMS functions in four faculties
REFINING OUR eLEARNING STRATEGIES Monitoring the LMS, or weblogs in our case, can achieve the objective of evaluating students’ online learning activities. By analyzing these data,
valuable guidelines in terms of eLearning design can be developed. Effective record-keeping, and extraction and interpretation of eLearning logs can reveal valuable information on standards of design and development, and program delivery.
Figure 9. Websites and students’ engagement in activities in two faculties
86
Evaluations of Online Learning Activities Based on LMS Logs
Using LMS for More than Content Delivery There are both positive and negative trends in our data concerning adoption of eLearning at CUHK. The overall impression seems to be that more courses have begun to have a web presence in our LMS. However, when we looked at these course websites closely, we did not find many functions were being used actively. Most of the websites had content for students to download. There might also be course information and announcements, but the other discussion, assignment and quiz functions were not popular (35% or below). Moreover, many of these functions did not seem to be sustainable in the sense that we observed a decrease in use of the more interactive functions such as online discussion and online quizzes over the years. Many scholars have also commented on this phenomenon that teachers used LMS mainly for delivery of content; our findings thus are more generally applicable outside our own context. Folajimi (2009) remarked that “computers aren’t fulfilling their potential to effect significant changes in education, are under-utilized, and are not being implemented in very effective or creative ways” (p. 1617). Ginsberg and McCormick (1998) also remarked that computers have only been used mostly to complement traditional teaching but are not designed as tools for effective learning activities on their own. One major drawback of existing LMS is that they are content-centric. Many teachers simply move all their teaching materials to the system. The materials are presented uniformly to all learners regardless of their background, learning styles and preferences. Modern trends in education are for learner-centric design where learners are facilitated to actively engage in the learning process to construct their personal knowledge. Teachers play the role of ‘facilitator’ who guides the learning process instead of being the sole information provider (Ong & Hawryszkiewycz, 2003). The
findings thus suggest that institutional eLearning support should not merely focus on having a web presence in courses or using the Web for courseware delivery. Attention also needs to be on the diffusion and sustained use of interactive online learning activities. Sustainability of these strategies seems to impose particular challenges in our context as the data show not only underuse of the related functions but also a general decrease of use for some of them. The question still remains about whether teachers are reluctant to interact with students on the Web in general, or whether this is only a limitation imposed by LMS. It can be an LMS problem; at CUHK, we are beginning to see a trend that teachers are interested in Web 2.0 strategies and social-media software and services. Web 2.0 or social software tools such as blogs, wikis, podcasts and media-file-sharing systems such as YouTube are supplementing or even “supplanting” (Gray, Chang & Kennedy, 2010, p. 33) the basic Web 1.0 strategies such as emails and forums in education. Users world-wide produce text and other media files for sharing and these materials can be easily adopted as teaching and learning resources elsewhere. The communications that are facilitated in social software also have the potential to extend communications to a scope far beyond the boundary of any single university. For example, Dillon, Wong and Tearle (2007) described internationalization of teaching and learning defying physical boundaries in terms of teachers and students in various places of the world teaching and learning together. We also know of a number of instances at our University where teachers have begun communicating with students through blogs and Facebook. Pituch and Lee (2006) observed that, although factors such as perceived usefulness influence LMS use, the strongest influence on student use might be system characteristics. Do these socialmedia services possess certain characteristics that make them better choices than the LMS, espe-
87
Evaluations of Online Learning Activities Based on LMS Logs
cially when implementing interactive activities? Is the LMS losing ground to these popular Web 2.0 technologies? Integration between the ‘Web 1.0’ LMS and Web 2.0 technologies seems to be needed (Boulakfouf & Zampunieris, 2008). However, teachers may simply refuse online interactions for teaching and learning in general. Levin and Arafeh (2002) suggested that a gap exists between students and teachers in the manner they use technology for teaching and learning purposes. Students use the internet for daily academic tasks, e.g. the internet as a virtual textbook and reference library, or virtual tutor, etc. However, most teachers are slower to adopt the more complex and interactive features of the Web (Morgan, 2003). If this is the case, our support should focus on motivating teachers to rethink teaching rather than just to promote functionalities of the LMS only. In a traditional, largely face-to-face university such as CUHK, blended learning (e.g. Garrison & Vaughan, 2007) offers possibilities that allow teachers to retain their traditional roles while gradually increasing the suite of online tools and strategies they use. In our context eLearning and LMS are at present supplementary tools to assist in the process of teaching and learning. What we want to achieve is that eLearning becomes more integrated into course design and thus complementary, rather than merely supplementary, to classroom teaching. Our support should not be limited to teachers only. Students may use technology in their everyday lives, but they may not be able to use the technology wisely for learning purposes. Kirkwood and Price (2005) pointed out that students may have ability with different computer applications, but only a few can transform and apply their digital competence to learning processes. For instance, students being “familiar with the use of email does not imply expertise in rigorous online debate and discussion” (p. 271). Support for meaningful eLearning thus requires professional development and support for both teachers and students.
88
The fact that students at CUHK use a number of LMS throughout their studies has led the University to plan to adopt a single LMS in the future. At the time of writing the paper, an evaluation of different LMS is underway; the imminent demise of WebCT is a facilitating factor. LMS logs have been important information in this evaluation process. For example, we understand that the future LMS should be very strong in course management and content delivery as these features are used by most teachers. However, we will also examine LMS for user-friendly interactive and communicative functions, and for convenient integration with existing Web 2.0 tools so that student-oriented eLearning strategies might be better used in the future.
Disciplinary Differences: Differentiating Support Strategies The LMS logs showed differences in the use of eLearning strategies between the faculties. This piece of information is crucial to refinement of our approaches to supporting eLearning in individual faculties. For example, we will focus on exploring the decline of interest of using certain eLearning strategies in places where the strategies do not seem to be sustained (e.g. Faculty B in our study). The support in that particular context will focus on known challenges rather than simply explaining and promoting eLearning in a traditional fashion. Another key factor is the perceived intrinsic return on effort in terms of better teaching practice and students’ learning, as well as the expected extrinsic benefit during performance appraisal. Our support to teachers in this circumstance includes both assisting them in lowering the effort in using technology, as well as showing them the usefulness of the strategies in fulfilling one or more of their needs. However, Thomas et al. (2009) interviewed eight teachers at CUHK and noted how these teachers considered innovative e-learning projects to be lonely and time-consuming. Being self-motivated and intellectually committed was a key factor that
Evaluations of Online Learning Activities Based on LMS Logs
sustained these teachers and allowed them to stay engaged. Our University is currently reviewing its strategies for assessing teaching quality and innovation in teaching will gain more prominence. This extrinsic reward should support a move to more active uses of eLearning. In disciplines where teachers seem to have a growing interest in using certain eLearning strategies, we can strengthen our support for these strategies and then also disseminate these teaching ideas to other teachers in related fields who are newer to eLearning strategies. Another structural condition that supports diffusion is the support provided by the department or by the University – in terms of monetary development grants, technical services, and consultative teaching and learning support in terms of eLearning pedagogical design. This support, though somewhat limited, still has significant impact on workload. The contents of this chapter will be presented at the University committee that considers policy for eLearning at CUHK. We know that differences in practice are also at the department level. We have devised an ‘eLearning Liaison Person’ (eLLP) strategy to better inform us the needs of individual departments. ‘eLLPs’ are representatives of individual departments who act as liaisons to inform central services of departmental needs and also act as conduits for conveying information about central services and events. To conclude, the LMS logs are useful in assisting us to continually adjust our eLearning support strategies. Some of the influences include: •
•
•
informing us the need to further investigate teachers’ attitudes and habits of using interactive e-functions; leading us to focus on meaningful uses of the functions rather than merely on explaining the functions of LMS in our promotion and support; informing us the important considerations when planning our future LMS;
•
•
informing us students’ online learning experiences which started a process to revisit support and services needed, including the discussion of a single platform; and assisting us in formulating faculty/department- specific support; eLLPs provide another source of information on disciplinespecific eLearning needs and habits.
FUTURE RESEARCH DIRECTIONS As mentioned above, LMS logs provide convenient data about web uses but are limited to showing the quantity rather than quality of these uses. Evaluating the LMS weblogs cannot provide information on the mode of online participation, interaction patterns and group dynamics. The interaction between users is of most importance when evaluating the function and effectiveness of an LMS (Gunawardena, Carabajal, & Lowe, 2001). The data highlight areas where further exploration seem to be warranted and we need other evaluation strategies (e.g. surveys and focus groups) to confirm these trends and patterns and look for the reasons behind them. One important area that is certainly worth exploring further is the observed decline in the use of communicative functions in the LMS. Is there a genuine decline of e-communication for learning or there are other software and services (such as Web2.0 technologies) that are taking teachers’ and students’ attention away from similar functions in the LMS?
CONCLUSION Effective record-keeping, and extraction and interpretation of eLearning logs can reveal valuable information on eLearning use. In universities with centralized web-based teaching and learning systems, monitoring the logs can be accomplished because most eLearning platforms have inbuilt mechanisms to track and record a certain amount of information about online activities. We reported
89
Evaluations of Online Learning Activities Based on LMS Logs
our strategy in building a system to automatically retrieve and then interpret the logs of eLearning activities in our two centralized LMS. The method has become a convenient and repeatable method for the evaluation of the richness of eLearning resources and interactions over time. In so doing, however, we have to acknowledge that the weblog analyses have limitations and should be interpreted with evidence about eLearning uses collected by other means and from other sources. We explained how the system works, and we used empirical evidence recorded from the academic years 2007 to 2009 to show how the data could be employed to achieve various levels of analyses. Weblogs represent a comparatively easy, automatic, and non-intrusive method to provide relatively quick and accurate data. We found that the logs have informed us the pattern of use of various online activities as well as the change in these activities over time. The new understanding has facilitated decisions in a number of eLearning support initiatives in our University, including the evaluation of new LMS systems for the future, and an increase in sensitivity towards faculty and departmental needs.
REFERENCES Alexander, S., & Hedberg, J. (1994). Evaluating technology-based learning: Which model? In Beattie, K., McNaught, C., & Wills, S. (Eds.), Multimedia in higher education: Designing for change in teaching and learning (pp. 233–244). Amsterdam, Netherlands: Elsevier. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11, 65–70. doi:10.1016/j. iheduc.2008.03.002
90
Boulakfouf, S., & Zampunieris, D. (2008). Integrating Web 2.0 features into a learning management system. Conference on e-Learning 2008. Retrieved December 29, 2010, from http://publications.uni.lu/record/1695/files/DZ_Pages%20 from%20ecel08-cd1.pdf Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177–192. doi:10.1080/10494820600852795 Colace, F., & De Santo, M. (2003). Evaluating online learning platforms: A case study. Proceedings of the 36th Hawaii International Conference on System Sciences. Retrieved December 29, 2010, from http://www.hicss.hawaii.edu/HICSS36/ HICSSpapers/ETWBE03.pdf Dillon, P., Wang, R., & Tearle, P. (2007). Cultural disconnection in virtual education. Pedagogy, Culture & Society, 15, 153–174. doi:10.1080/14681360701403565 Folajimi, Y. (2009). Transforming distant pedagogical learning to Web based collaborative system: An Intelligent Tutoring Systems Architecture. In T. Bastiaens et al. (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009 (pp. 1618–1622). Chesapeake, VA: AACE. Froehlich, T. J. (1994). Relevance reconsidered Towards an agenda for the 21st century: Introduction to special topic issue on relevance research. Journal of the American Society for Information Science American Society for Information Science, 45(3), 124–134. doi:10.1002/(SICI)10974571(199404)45:3<124::AID-ASI2>3.0.CO;2-8 Garrison, D. R., & Vaughan, N. D. (2007). Blended learning in higher education: Framework, principles, and guidelines. San Francisco, CA: Jossey-Bass.
Evaluations of Online Learning Activities Based on LMS Logs
Ginsberg, R., & McCormick, V. (1998). Computer use in effective schools. Journal of Staff Development, 19(1), 22–25. Grasha, T. (1990). The naturalistic approach to learning styles. College Teaching, 38(3), 106–113. Gray, K., Chang, S., & Kennedy, G. (2010). Use of social Web technologies by international and domestic undergraduate students: Implications for learning and teaching in Australian universities. Technology, Pedagogy and Education, 19(1), 31–46. doi:10.1080/14759390903579208 Gunawardena, C., Carabajal, K., & Lowe, C. A. (2001). Critical analysis of models and methods used to evaluate online learning networks. Education Resources Information Center (ERIC). Retrieved December 29, 2010, from http://eric. ed.gov/PDFS/ED456159.pdf Jovanovic, J., Devedzic, V., Gasevic, D., Hatala, M., Eap, T., Richards, G., & Brooks, C. (2007). Using Semantic Web technologies to analyze learning content. Internet computing, 11(5), 45–53. Kirkwood, A., & Price, L. (2005). Learners and learning in the 21st century: What do we know about students’ attitudes and experiences of ICT that will help us design courses? Studies in Higher Education, 30(3), 257–274. doi:10.1080/03075070500095689 Lam, P., Keing, C., McNaught, C., & Cheng, K. F. (2006). Monitoring e-learning environments through analyzing Web logs of institution-wide e-learning platforms. In L. Markauskaite, P. Goodyear, & P. Reimann (Eds.), Who’s learning? Whose technology? (pp. 429–440). Proceedings of the 23rd Annual Australian Society for Computers in Learning in Tertiary Education 2006 conference, Sydney, Australia. Retrieved December 29, 2010, from http://www.ascilite.org.au/conferences/sydney06/proceeding/pdf_papers/p62.pdf
Lam, P., McNaught, C., & Cheng, K. F. (2008). Pragmatic meta-analytic studies: learning the lessons from naturalistic evaluations of multiple cases. ALT-J, 16(2), 61–79. doi:10.1080/09687760802315879 Levin, D., & Arafeh, S. (2002). The digital disconnect: The widening gap between Internet savvy students and their schools. Pew Internet & American Life Project. Retrieved December 29, 2010, from http://www.pewinternet.org/~/ media//Files/Reports/2002/PIP_Schools_Internet_Report.pdf.pdf Mazza, R., & Milani, C. (2005). Exploring usage analysis in learning systems: Gaining insights from visualizations. 12th International Conference on Artificial Intelligence in Education (AIED 2005). The Netherlands: Amsterdam. Retrieved December 29, 2010, from http://linux3.dti.supsi. ch/~mazza/Web_area/Pubblicazioni/AIED05/ aied-ws2005.pdf Morgan, G. (2003). Faculty use of course learning systems. Educause center for applied research reports. Retrieved December 29, 2010, from http://net.educause.edu/ir/library/pdf/ecar_so/ers/ ers0302/ekf0302.pdf Ong, S. S., & Hawryszkiewycz, I. (2003). Towards personalised and collaborative learning management systems. Third IEEE International Conference on Advanced Learning Technologies (ICALT’03). Retrieved December 29, 2010, from http://www.computer.org/portal/web/csdl/ doi/10.1109/ICALT.2003.1215113 Paulsen, M. F. (2002). Online education systems: Discussion and definition of terms. Retrieved December 29, 2010, from http://www.porto.ucp. pt/open/curso/modulos/doc/Definition%20of%20 Terms.pdf
91
Evaluations of Online Learning Activities Based on LMS Logs
Pituch, K. A., & Lee, Y. K. (2006). The influence of system characteristics on eLearning use. Computers & Education, 47(2), 222–244. doi:10.1016/j. compedu.2004.10.007 Romero, C., Ventura, S., & Garcia, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51, 368–384. doi:10.1016/j.compedu.2007.05.016 Sen, A., Dacin, P. A., & Pattichis, C. (2006). Current trends in Web data analysis. Communications of the ACM, 49(11), 85–91. doi:10.1145/1167838.1167842 Thomas, K., Lam, P., & Ho, A. (2009). Knowledge diffusion in eLearning: Learner attributes and capabilities in an organization. In G. Siemens & C. Fulford, (Eds.), ED-MEDIA 2009 (pp.493–497). Proceedings of the 21st Annual World Conference on Educational Multimedia, Hypermedia and Telecommunications, Honolulu, Hawaii. (pp. 22–26). Chesapeake, VA: Association for the Advancement of Computers in Education. Zhang, H., Almeroth, K., Knight, A., Bulger, M., & Mayer, R. (2007). Moodog: Tracking students’ online learning activities. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 4415–4422). Chesapeake, VA: AACE.
ADDITIONAL READING Alavi, M., & Leidner, D. E. (2001). Review: Knowledge management and knowledge management systems: conceptual foundations and research issues. Management Information Systems Quarterly, 25(1), 107–136. doi:10.2307/3250961
92
Becker, R., & Jokivirta, L. (2007). Online learning in universities: Selected data from the 2006 Observatory survey–November 2007. Observatory on borderless higher education (OBHE) online. Retrieved December 29, 2010, from http://www. obhe.ac.uk/documents/view_details?id=15 Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Collis, B., & van der Wende, M. (2002). Models of technology and change in higher education: An international comparative survey on the current and future use of ICT in higher education. Center for Higher Education Policy Studies (CHEPS). Retrieved December 29, 2010, from http://doc. utwente.nl/44610/1/ictrapport.pdf O’ Reilly, T. (2005). What is web 2.0? Retrieved December 29, 2010, from http://www.ttivanguard. com/ttivanguard_cfmfiles/pdf/dc05/dc05session4003.pdf Piccoli, G., Ahmad, R., & Ives, B. (2001). Webbased virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. Management Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989 Selim, H. M. (2007). Critical success factors for eLearning acceptance: Confirmatory factor models. Computers & Education, 49(2), 396–413. doi:10.1016/j.compedu.2005.09.004 Wang, W. (2007). Features of future learning management system. In T. Bastiaens & S. Carliner (Eds.), Proceedings of World Conference on ELearning in Corporate, Government, Healthcare, and Higher Education (pp. 1332–1335). Chesapeake, VA: AACE.
Evaluations of Online Learning Activities Based on LMS Logs
Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75–79. doi:10.1145/986213.986216
KEY TERMS AND DEFINITIONS Active Content: Content on the website that has been accessed (meant downloading in our study) by students in the specified period of time. Active Discussion: Online forums that have been accessed (meant posting in our study) by students in the specified period of time. Active Quizzes: Online quizzes that have been accessed (meant attempting the quizzes in our study) by students in the specified period of time.
Active Websites: Websites that have recorded student access (meant viewing the front page of website in our study) in the specified period of time. Engagement: LMS logs revealing the amount of activities (e.g. frequency of use) recorded for the various functions. LMS Logs Retrieval and Reading System: An automatic system to retrieve suitable logs from an LMS server, interpret the logs and then represent them in ways that facilitate our understanding of the types and level of activities happened on the LMS. Nature of Activities: LMS logs revealing the common functions being used on the course websites. Popularity: Using LMS logs to find out how many courses opened an active website on one or more of the LMS in our study.
93
94
Chapter 5
ANGEL Mining Tyler Swanger Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock Yahoo!, USA Anthony Scime The College at Brockport, State University of New York, USA Brendan P. Post The College at Brockport, State University of New York, USA
ABSTRACT This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation.
INTRODUCTION A Learning Management System (LMS) is a course independent framework that provides, delivers, and manages instructional content, identifies and assesses learning, and tracks and records progress towards those goals. It may also provide course
registration and administration, as well as skills gap analysis, tracking, and reporting (Sclater, 2008; Watson & Watson, 2007; Paulsen, 2003). A Learning Management System generally has the same interface and features for all courses at a given school. Typically they have discussion forums, calendars, quiz capability, group work and chat spaces, and gradebooks, and perhaps
DOI: 10.4018/978-1-60960-884-2.ch005
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
ANGEL Mining
some customization capability for the instructor or individual student (Feldstein & Masson, 2006). Learning Management Systems undergo evaluation for a number of reasons. Evaluations of LMS data may be conducted to assist instructors in understanding how to enhance student learning (García, Romero, Ventura, & Calders, 2007, Romero, Ventura, & García, 2008). Schools may evaluate existing LMS options to select a particular initial choice (Sclater 2008), or when a decision is being considered on whether to replace or not the current LMS; replacement may be due to the current system no longer being supported (Sturgess & Nouwens, 2004). An investigative analysis of the usage of A New Global Environment for Learning (ANGEL), learning management system was conducted at a comprehensive 4-year college. This implementation of ANGEL has been used by the college’s faculty, staff, and students for eight years. The ANGEL Learning Management System “enables efficient and effective development, delivery and management of courses, course content and learning outcomes. Engaging communication and collaboration capabilities, enhance instruction to deliver leading edge teaching and learning” (ANGEL Learning, 2008). This investigation was prompted by the merger of the ANGEL company with a competitor, Blackboard, Inc; and Blackboard’s decision to stop support of the college’s version of ANGEL in 2012 (Blackboard, 2010). These changes present the college with the opportunity and need to assess the usage of the system. These events have forced the college to examine the need for a LMS, and if so what features are necessary in the event a new product needs to be purchased. Given these circumstances, data mining techniques were applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Data mining techniques are applicable to situations where large amounts of data exist, and the data may contain
internal relationships and patterns that characterize the data set as a whole. In the case of this study, the usage of ANGEL by students and instructors is described by the ANGEL data. This study clearly demonstrated that data mining techniques can be applied to find unknown patterns, interesting patterns, confirm assumptions, and consider statistical results for making decisions on the future of ANGEL or another LMS at a college. The data mining methods of classification, association, and clustering were applied to analyze the data. Classification produces a decision tree which predicts which courses will use the LMS system in the future, based on course specific attributes such as course type, discipline, and the number of students enrolled. Association mining discovers the usage of one feature’s effect on the usage of all other sets of features. Grouping features with similar values is the process of clustering. From the results of these analyses, metrics are formed to indicate usage of features in the LMS.
BACKGROUND There is a similarity between LMS and Knowledge Management Systems (KMS); both provide a repository for knowledge which is valuable for the user. In a KMS the knowledge is kept and used by an organization’s employees. In a LMS the purpose is to disseminate knowledge from instructors to students and to share knowledge in a way to enhance student learning (Haldane, 1998). There are also similarities between Learning Management Systems and distance learning. Distance learning uses LMS like software to provide students with learning materials and activities while tracking student activity (Falvo & Johnson, 2007). Whether the system is an LMS, a KMS, or a distance learning system the organization needs to select and implement the system best suited to their needs. Studies have been done to compare different Learning Management Systems (LMS)
95
ANGEL Mining
prior to installation or to evaluate effectiveness (Beatty & Ulasewicz, 2006). There have been many approaches to system evaluation and selection. Traditional systems requirements analysis has been used to evaluate and select LMS. Functionality desired by faculty has been achieved through requirements analysis followed by system selection and modification (Sclater, 2008) and functionality has been achieved through the use of design patterns (Avgeriou et al., 2003). Studies have been done to determine frameworks for evaluating LMS. Kim and Lee (2008) validated a model for evaluating Learning Management Systems (LMS) used in e-learning fields using exploratory factor analysis. Factor I was instruction management, screen design, and technology and factor II covered interaction and evaluation. Roqueta (2008) used Moore’s (1993) transactional distance theory, the diffusion of innovations theory (Rogers, 2003), and Malikowski’s (2007) model for evaluation of learning systems to conclude that there is a difference between learning management systems and course management systems, and that LMS are preferable in most cases. A commercial tool for evaluation has been developed by 3Waynet Inc. The LMS Evaluation Tool is spreadsheet like and designed to assist in selecting a LMS. With this LMS Evaluation Tool users specify criteria for evaluation, which are considerations that are important to the institution adopting the LMS, these criteria are cost of ownership, maintainability and ease of maintenance, usability, ease of use, user documentation, user adoption/ vendor profile, openness, standards compliancy, integration capacity, learning object metadata integration, reliability and effectiveness, scalability, security, hardware and software considerations, and multilingual support. Each criterion is expanded with relevant questions to be answered. The candidate LMS undergoing evaluation are also rated on their features. The features considered
96
are administration, security, access, integration with other systems, course design, development and integration, course monitoring, assessment design, online collaboration and communications, and productivity tools. These features each have a number of sub-features that are given weights to measure their relative importance. The user rates each candidate LMS on each criterion and feature. The evaluation tool computes an overall score for each LMS in each criterion (Commonwealth of Learning, 2004). Additionally, the independent research company Brandon Hall Research, has completed a study of over 90 LMS with custom comparisons across more than 200 features. The report of this study is available on the World Wide Web in an interactive format allowing users to specify their requirements and complete side-by-side comparisons of selected LMS (Chapman, 2010). Studies have shown that most evaluations of LMS look toward the technology of the system. Specifically, evaluations are done on communication, education management, and file management features and how effectively these features are implemented in the technology. Hall (2003) provides a list of technically oriented considerations in evaluating learning management systems. Lewis, et al. (2005) evaluated nine LMS by considering and comparing the features and capabilities of the systems in terms of content development, group work and participation, the calendar, communication capability, study tools available to students, handling of audio and video, monitoring of student participation and progress, and usability in terms of navigation, interface design and administration. Evaluations often done on the technology itself are conducted by experts and not participants. Feldstein and Masson (2006) outline features that teaching professionals, as well as technical staff, should consider when selecting an LMS. Specifically they suggest looking for the ability to customize the LMS to meet the pedagogical methods of individual instructors, which may vary
ANGEL Mining
by discipline, instructor, and learning styles of individual students. Sturgess and Nouwens (2004) report on a different approach where participation in the evaluation includes an understanding of the organizational culture and sub-cultures. Specifically they identified academics, information technology staff, multimedia development specialists and managers/administrators, as participant groups in selecting a new learning management system. Evaluation of student use of LMS found that the LMS needs to be integrated with other educational activities and other aspects of everyday life. This is more important than the interface design. Students’ idea of what an LMS should provide differs from the impression of the technical staff and other stakeholders who select and implement the system. While students effectively use an LMS remotely this is only undertaken when there is a practical advantage to the student, not just for some educational benefit (Alsop & Tompsett, 2002). Student learning style and its impact on learning management systems is investigated by Graf and Kinshuk (2006). Bayesian Networks have been used to discover user preferences to allow adaptation by the LMS to those preferences (Kritikou, Demestichas, Adamopoulou, Demestichas, Theologou, & Paradia, 2008). Instructors can use data mining techniques such as classification, association and clustering on student LMS data to improve instruction. Clustering techniques can find groups of similar students so that the instructor can direct instruction to the group’s particular needs classification will identify the characteristics of the students in each group. Association mining may discover relationships between these characteristics and student attributes (Romero, Ventura, & García, 2008). There are a number of data mining techniques and associated algorithms. A common technique shared with statistics is classification analysis. A decision tree model is constructed which finds a path to a predetermined class attribute for each data record. A classification decision tree contains
branches that can be converted to rules unique to the data set. Machine learning research produced a number of classification models, the best known of which is the C4.5 algorithm (Quinlan, 1993), which uses information gain to define the model’s structure. A product of machine learning research, association mining is used to find patterns of data where sets of attribute-value pairs occur frequently in the data set. With association mining what class attribute should be in the results is not predetermined. Apriori (Agrawal, Imieliński, & Swami, 1993) is the predominate association mining algorithm. It is an algorithm that produces many rules, and special techniques are needed to reduce the rule set to those that are interesting. The final technique used in this chapter is clustering. Clustering represents each attribute as a dimension and shows where data records occur in this multidimensional problem space. The most popular clustering algorithm is k-means (MacQueen, 1967). Again, analysis of the clusters needs special techniques. In an attempt to increase the robustness and reliability of rules, often a combination of data mining methodologies are applied. Deshpande and Karypis (2002) and Padmanabhan and Tuzhilin (2000) improved classification rules by first using association mining. Li, Han, and Pei (2001) classified records using CMAR (Classification Based on Multiple Class-Association Rules), which identifies frequent patterns and associations between records and attributes. Jaroszewicz and Simovici (2004) employed user background knowledge and a Bayesian Network to determine the interestingness of sets of attributes prior to association mining. Fu and Wang (2005) reduced data dimensionality using a separability-correlation measure that ranks the importance of attributes to improve classification and the usefulness of rules. Scime and Murray (2007) and Murray, Riley, and Scime (2007) used expert knowledge to reduce data dimensionality while iteratively creating classification models. Rajasethupathy, Scime,
97
ANGEL Mining
Rajasethupathy, and Murray (2009) improved the usefulness of rules by identifying “persistent rules,” which are those rules identified by both classification and association mining.
DATA MINING ANGEL DATA Data mining is a process of inductively analyzing data to find interesting patterns and previously unknown relationships in the data. Data mining is used not only to predict the outcome of a future event but also to provide knowledge about the structure and interrelationships among the data. The data mining process identifies relationships that are expressed as classification rules, association rules, and clusters of records. Data mining algorithms are used to create models that describe existing data and relationships within the data. These methodologies create rules used to analyze new data and predict future outcomes. The data set used in this analysis contains information across two years. Data is recorded in the ANGEL data base for every class offered during Fall, Spring, and Summer for every feature. This data consisted of class specific data, e.g. semester, department, course number, and student count, and feature data, e.g. feature (files, messages, folders etc.) and frequency of feature usage. The data set was processed using classification, association, and clustering algorithms as implemented by the Waikato Environment for Knowledge Analysis (WEKA, Witten and Frank, 2005). The intent of this study is not to compare ANGEL to other LMS but rather to assess the usage of the ANGEL features by students and instructors. Basing this study on recent historical data provides a good indication of the expected ANGEL use in the near term future. The college’s course offering and student and faculty profiles are not expected to change in the next few years. Classification mining determines the influence of combinations of attributes on a specific goal. In the case of this ANGEL data, multiple classification
98
models determine the usage of each feature based on student count, department, level, and semester. Association mining finds patterns in the data that reoccur frequently. This is an effective method for finding features that co-occur. Association rule mining was preformed to find the relationships between the features of ANGEL. This could lead to acceptance or rejection of sets of features. Clustering finds natural clusters occurring in data. These multi-dimensional clusters divide the data into groups; sufficiently large groups indicate the most important features in the data. In the ANGEL data a large cluster will indicate the most commonly used features. The WEKA data mining tool is used to apply the data mining techniques, and Microsoft’s Excel was used to prepare the data.
Preprocessing Data often contains attributes and records that are not pertinent to the analysis being conducted. In the case of the ANGEL data set, it consisted of records from all classes offered by the college for the Fall and Spring semesters from Fall of 2007 to Spring of 2009. For each of these classes the feature usage was collected. This original data set contained 9430 records, one for each class section, and 20 attributes (features). From this data set records for Thesis Continuation Credit (TCC) classes were removed, because TCC classes are an extension of a previous thesis course but were not identified by department. The information gained from these sections would have been inconclusive since the original class is not identified. The General Education Program (GEP) class records with sections of 0 or 99 were also removed. These sections are used to place all incoming transfer students in the computer skills exam, passing the exam is a graduation requirement, but an actual class does not exist. Five attributes were removed from the data set. The attribute identifying the section number was not relevant to the scope of our analysis. Determining the difference between section 1
ANGEL Mining
and section 2 is trivial. The attributes indicating the use of wikis, games, blogs, and assessments features were removed due to their extremely low usage. For example, assessments, which posted the highest usage of the four, was only used by 64 classes or .68% of total classes. After removal of the records and attributes the data set was reduced to 9312 department-course level- semester records with 15 attributes (Table 1). Some attributes had their values adjusted. The course number was changed to a nominal Level number, for example 436 became L400, Semester was changed to a nominal value, for example 200709 became S200709 for the Fall semester 2007. Other attributes had values for the number of times the feature was used in the course. For all empty attribute values a zero (0) was inserted, because the absence of a value meant the feature was used zero times for that class. These adjustments created the base data set. The data mining algorithms required slight variations from this base data set. For classification and association mining a nominal data set was needed. The numerical feature data was
discreteized to nominal data using six bins with equal depth binning. This method creates six bins for each attribute where the number of records in each bin is kept close to equal. Nominal data for student count used the college’s Common Data Set from 2008-2009. This document binned student count by the ranges 2-9 (very low), 10-19 (low), 20-29(medium low), 30-39 (medium), 40-49 (medium high), 50-99 (high), 100+ (very high). Classes of a single student are not specified in the Common Data Set, therefore they were assigned a value of ‘single’. Preprocessing resulted in a data set characterized by department-course level-semester records with binned counts of feature usage. Beyond this data preparation, preprocessing done on the data set is specific to the data mining method.
Classification Using Decision Trees The goal of classification of the ANGEL data is to examine the influence of semester, department, level, and student count on each feature. This requires classifying the data with respect to each
Table 1. Attributes Name
Description
Department
The academic department of the course
Drop Box
Number of drop boxes used to electronically collect assignments
Files
Count of files available for download
Folders
Number of folders used to organize content
Form
Number of surveys to be completed by students
Forum
Number of online discussions
Grade Book
Number of grade books (usually one)
Links
Number of URLs to resources outside the LMS
Level
The level of the course (e.g. 100, 200, 300, etc.)
Messages
Number of emails sent from within the course
Pages
Number of content pages available for students to view
Quizzes
Number of LMS administered tests
Semester
The semester in which the section was taught
Student Count
Number of students in the section
TurnItIn
Number of assignments collected and evaluated for originality
99
ANGEL Mining
feature individually. From the data set a feature specific data set was created consisting of semester, department, level, student count and one feature. These data sets were processed individually to create feature specific decision trees. Decision tree analysis creates a tree structure, where each internal node denotes a test on an attribute, each branch represents an outcome of the test, and every leaf represents a class or class distribution (Bagui, 2006). Decision tree analysis allows for easy conversion to classification rules. A decision tree starts as a single node. If all records in that node are the same it becomes a leaf. If they are not all the same, a selection algorithm uses entropy, a measure of the inconsistency of the data, to decide how to divide the records. An attribute is chosen that best divides the records into further purer nodes. The attribute chosen is labeled as the decision attribute. Branches are then created for the known values of the decision attribute, and the records are divided accordingly into the child nodes. This process is performed Figure 1. Generic decision tree
100
recursively on all child nodes (Bagui, 2006). For example, if given a data set consisting of both boys and girls and their favorite sport, using entropy the algorithm selects gender as the first decision attribute producing two child nodes. The data set records are divided at this node. Each subset undergoes the same algorithmic process at the individual gender nodes to further divide the records. This process continues until only pure nodes occur, creating leaves. An example of a completed classification decision tree is presented in Figure 1. This generic decision tree has four nodes or points at which decisions are made. A record would be classified depending on the values of its attributes into one of the leaf nodes, where the class attribute has a specific value. In this example the possible class attribute values are w, x, y, and z. A record with the attribute values (Attr1 = a then Attr2 = 4) would be classified as the class attribute with the value x. This record is classified by following the Attr1- Attr2 edge of the tree and then the
ANGEL Mining
Attr2-ClassAttr=x edge, reaching a ClassAttr=x leaf. Other records may also reach a leaf with ClassAttr=x (e.g., Attr1 = c then Attr3 = g then Attr4 = s), but via a different path. After the decision tree is constructed, each branch of the decision tree is converted into a rule. The tree presented in Figure 1 can be converted into the following Rules: • • • • • •
Rule 1: IF Attr1 = a AND Attr2 ≤ 5 THEN ClassAttr = x Rule 2: IF Attr1 = a AND Attr2 > 5 THEN ClassAttr = y Rule 3: IF Attr1 = b THEN ClassAttr = z Rule 4: IF Attr1 = c AND Attr3 = g AND Attr4 = s THEN ClassAttr = x Rule 5: IF Attr1 = c AND Attr3 = g AND Attr4 = t THEN ClassAttr = w Rule 6: IF Attr3 = h THEN ClassAttr = y
The rules provide insight into how the class attribute’s value is, in fact, dependent on the other attributes. A complete decision tree provides for all possible combinations of the attributes and their allowable values reaching a single, allowable class attribute value. The decision tree algorithm C4.5 builds the decision tree from a training data set using information entropy (Quinlan, 1993). The goal of the algorithm is to classify all the training set records according to the goal or class attribute. The algorithm first determines the attributes available in the first node. The normalized information gain with respect to the class attribute is calculated for each attribute to determine how the records should be split into subsets. The attribute with the highest normalized information gain is selected as the decision attribute. The records are then broken into branches on the values of the decision attribute, creating branches of the tree ending in child nodes. This process is performed recursively (Quinlan, 1993). The class attribute and its values constitute the leaf nodes of the decision tree.
Information gain is the change in information entropy from the current state of the set of records to the proposed state of the set of records. Entropy is a measure of the randomness of the distribution of records in a subset of records with respect to the class attribute. The attribute with the greatest information gain is selected as the node for dividing the data in the decision tree at each node. The information gain with the overall highest gain with respect to the dependent attribute is the root of the tree. The confidence that a record is correctly classified provides the accuracy of splitting the record set. Confidence is calculated by dividing the number of records correctly classified by the total number of records that are classified at the node. Confidence is used to prune the decision tree, ensuring a tree with a reasonable number of leaf nodes and acceptable accuracy. The level of pruning is controlled by setting a minimum confidence level. If a node falls below the minimum confidence that branch is pruned; that is, the records are rolled up to the parent node until the minimum confidence is met This allows for only rules that meet the required accuracy to remain on the final tree. The data set was divided into training and test sets, where the training set contained two-thirds of the records, randomly selected. The remaining records are in the test set. The decision tree was constructed using the training set and validated with the test set. To determine the appropriate confidence level multiple trees were constructed using individual features as the class attribute. Evaluation of multiple decision trees is common in classification problems (Osei-Bryson, 2004). After several trials of varying confidence, 55% confidence was selected. This choice was determined after trials focusing on minimizing the percent incorrectly classified, while maintaining quality leaf nodes. However, the direct relationship of these features needed to be taken into account. As the number of leaf nodes decreases, the num-
101
ANGEL Mining
ber of incorrectly classified records decreases, as well as the quality of rules formed from the nodes decreasing. The goal of classification of ANGEL data is to examine the influence of semester, department, level, and student count on a feature. To accomplish this task, our fully nominal data set was broken into eleven data sets, which in turn were divided into training and test sets. Each data set consisted of semester, department, level, student count and one feature, each having a different feature. For example, semester, department, level, student count, and file. Dividing the data set into these smaller data sets allowed each feature to be a class attribute. Records with zero values for the class (feature) attribute were removed from the respective feature data set. The C4.5 algorithm was executed on each data set with a confidence level of 55%. The decision tree is converted into IF-THEN rules. Each rule represents one branch of the decision tree from the root node to a leaf node. The intermediate nodes provide additional information in the form of branching on the branches. There is one rule for each of the leaf nodes of the tree. The results of the classifications followed a pattern. For the top five features (messages, files, folders, grade book, and links), the attribute with the highest information gain was student count followed by department. From this several rules were formed that concluded that as class size increased the usage of the feature increased. Figure 2 provides a sample of the rules found. For the little used features (drop box, form, forum, page, quiz and turn it in) the attribute with the highest information gain was first department followed by level. From this classification a definite conclusion pertaining to all classes cannot be made. Several departments have lower level classes with heavier users, while other departments have higher level classes with the heavier users (See Figure 3 for rule examples). From classification it is found that features with a less traditional online use such as, admin-
102
istering a test or having students turn in a paper, have usage that is directed more by the department and level than the class size. This suggests users who are more willing to experiment with the new technology create the demand. Those users then have an effect on their colleague’s use of that feature. Whereas, with the more commonly used online features, such as e-mail messages or file sharing, usage is directed by need. This suggests that the more students in a class, the more likely a professor is to post files rather than print the files to hand out in class. These findings support Nichols’(2003) hypothesis that the movement to eLearning, including the use of LMS, is an evolutionary, not revolutionary process. As success is seen in one course where an LMS is used the LMS will be adopted in other, similar courses.
Association Rule Mining The goal of association mining the ANGEL data was to evaluate each feature’s affect on the usage of the other features. This required modification to the original data set. Non-feature attributes were removed so that patterns of feature usage would show the use of one feature with another in the association rules. Association rule mining, unlike classification rule mining, does not need a class attribute in order to find meaningful results. Association mining allows for rules to be generated for any combination of attributes within the data set. Association Rules are not intended to be used together (Witten and Frank, 2005). Each rule generated in association mining implies a different regularity within the data set, and each could result in a very different conclusion. With the ANGEL data, relationships are found between the different features used for a class on campus. To construct association rules itemsets found in the data set are used. An itemset is a collection of attribute-value pairs (items) that occurs in the data set. Each itemset can be converted into a
ANGEL Mining
Figure 2. Sample classification rules of top features
number of rules, where each item in an itemset implies and is implied by every other item and combination of items in the itemset. This results in a very large number of rules. Since association rule mining results in a very high number of rules, restrictions need to be applied to specify whether a rule is significant and valid. The metrics of support and confidence are used to determine the significance of a rule. The support of an itemset is the count or percent of records that contain all the items in the itemset. A support threshold is set which must be met or exceeded for a itemset to be considered a frequent itemset. In the example above the support count threshold was set to two.
Rules that come from the same itemset all have the same support value. A rule contains a premise (IF-part) and consequent (THEN-part) and states that when the premise is true the consequent will be true. The confidence that this rule is correct is also calculated. The confidence is a conditional probability that a record containing the premise will also contain the consequent. It is calculated by dividing the support for the rule by the support just for the itemset that is the same as the premise. Only those rules meeting the user-specified confidence threshold are kept. Each rule is an observation of the data’s behavior. The algorithm chosen for this investigation is the Apriori algorithm. The Apriori algorithm uses itemsets to generate rules, and support based
103
ANGEL Mining
Figure 3. Sample classification rules for forums
pruning to help control the growth of rules. It also uses the Apriori Principle. This principle states that if an itemset is frequent, then all of its subsets must also be frequent (Agrawal, Imieliński, & Swami, 1993). The Apriori algorithm first considers the single itemsets, those with one only item and counts how frequently the item appears in the data. The itemsets that do not meet the minimum support threshold are discarded from the possible one itemsets. The Apriori Principle ensures that all supersets of the one itemsets that are infrequent, are also infrequent. A list of two itemsets is generated from the list of one itemsets. The frequency of the two itemsets is determined and those not meeting the minimum support threshold are removed. Three itemsets are created from the two itemsets and the process continues until no more frequent itemsets can be found. Once all of the frequent itemsets are generated, the rules are generated. If a rule does not meet the minimum confidence
104
level, it gets discarded as well as any rule that is a subset of that rule. The goal of association mining the ANGEL data was to evaluate a feature’s affect on the usage of one or more other features. To create the data set necessary to achieve this goal several attributes (semester, department, level and student count) were removed from the original data set. These attributes were removed to generate only rules pertaining to the use of one feature’s influence on another. For example, the use of folders indicates the use of files. The minimum value for support was set to 0.7, the value for confidence was set to 0.9, and the number of rules was set at 3000. Support and confidence were set at such high values to ensure presentation of rules with certainty. The number of rules was set so as to be exhaustive and find all possible rules within the bounds of support and confidence. The algorithm generated 2,638 rules.
ANGEL Mining
The useful rules can be defined by applying a template (Klemettinen et al. 1994). Only those rules where the predicate consisted of a single feature were kept. The rules then had the appearance of, if attribute X is used, then these other attributes are used. These rules show how using one feature of the ANGEL system influences usage of other features. However, only rules where the feature count was zero were found. Nevertheless, this still is an interesting, and meaningful result. The rules now have the form, if you do not use this feature, then you will not use these other features. In order to reduce the number of rules further, if there existed a rule X where the consequence was a subset of the consequence of rule Y, then X was removed (Rajasethupathy, Scime, Rajasethupathy, & Murray, 2009). After the filtering of rules, 37 rules remained. (Appendix A). Inspecting the rules from association rule mining leads to a few general conclusions. Given the level of confidence and support in this study, the use, or non-use, of email messages or files uploaded to ANGEL have no affect on the other features. There is no correlation between those two features and any of the others. One might assume that if a particular class on ANGEL uploads a large number of files that this high frequency of usage could be seen with other features, however this investigation did not find that to be the case. Another conclusion is that if a class does not use the drop box feature, then they will not use any feature, or any combination of features. When rules were being purged, there existed only one rule with drop box in the predicate. This rule can be used as a metric for ANGEL usage. (Note: it is not logically correct to say that if they do use the drop box feature then they will use other features.) At the theoretical level, García, Romero, Ventura, and Calders (2007) determined that in association mining critical factors in determining usage are the number of messages, emails, documents, and Web pages on the course site. However, the findings of this study indicate that
the number of messages and emails do not influence the feature usage. In association mining student Moodle activity in a course, Romero, Ventura, and García (2008) found that students that do not send messages (or email) do not read them either, which then leads to course failure. While there is no relationship between using the message components of ANGEL clearly use of a LMS features, while not ensuring a passing grade, does mitigate the likelihood of failure.
Cluster Rule Mining The goal in clustering the ANGEL data set was to provide an indication of the most used features in larger classes and therefore by the most students. Additionally, indications of infrequent use of some features, while identifying where these features are used, is found in smaller clusters. In conventional terms, clustering is when you group similar objects together. In data mining, this definition still holds. Clustering in data mining is taking similar records and grouping them together into clusters based on a measurable distance between them. Once these clusters are created, similarities can be determined within the data set that may not have been previously apparent. In clustering each attribute is a dimension in the problem space. Each record is placed as a point in the problem space based on the values of the record’s attributes. The process of clustering is based on comparing the records, calculating the distance between them, and then grouping like records together. The collection of clusters (known as a clustering) has clusters consisting of records that are very similar to records inside the cluster, but dissimilar to those within other clusters. The goal of clustering is to indentify common feature groupings found in ANGEL, as well as, the count of students associated with them. The algorithm used for clustering for this investigation is k-means. Since clustering is graphical the data must be fully numeric. Therefore, the
105
ANGEL Mining
pre-discretized data set was used. This data set contained numerical data for feature usages and student count. The nominal attributes (semester, department, and level) were removed. The k-means algorithm requires an input for the number of clusters (k). The algorithm randomly selects k points to be the initial cluster centroids. The records are formed into these k clusters based on to which centroid a given record is closest, closeness being determined by the distance measure. The centroid of each cluster is recalculated using the records in the cluster. The distance from each record to each centroid is recalculated and records may change clusters. This process continues until the difference between the current centroid and the previous centroid is zero or near zero for each cluster. The value of k determines the number of clusters, varying k results in different clusterings or grouping of the records. To determine the optimal clustering the sum of squared errors (SSE) are compared to the number of clusters. As the number of clusters increases the SSE should decrease, when the SSE begins to remain nearly constant the smallest number of clusters in typically chosen. This process was performed on the ANGEL data set. Fifteen clusterings were computed, increasing k by two from two to thirty. For each analysis of k the SSE was recorded and graphed (Figure 4). From the graph, 23 clusters were chosen for cluster rule mining. This cluster size was chosen as it is the point where the graph levels out. The area of the graph where leveling occurs is considered to be optimal for cluster size because once the graph begins to level off the measure of SSE from one k to the following k+2 is less meaningful. Therefore, the analysis is producing sub optimal clusters, or with less meaningful information than their predecessor. For example, in our data set using twenty-six clusters would not produce better results. It would be more likely to divide good clusters.
106
The goal in clustering the data set was to find natural feature groupings in the data set and the class size associated with that grouping. In the 23 cluster clustering, five clusters were formed that contained 72% of the records, and revealed files and messages as the only features used by those clusters. Furthermore, clustering revealed seven clusters totaling 3.7% of records, that used nine to ten of eleven features. Lastly, the largest cluster holding 24% of records contained an average student count of 1.78 and average of 0 to 0.45 for feature use. That cluster formed the rule that if a class has two or less students they will not use ANGEL. The results received by clustering show that the most commonly used features in ANGEL are files and messages. Most classes do not use the more advanced features in ANGEL, yet there is a small subset of instructors that heavily use advanced features. Finally, classes with two or less students do not use any feature in ANGEL; this may imply that it is not worth the instructor’s time to create content for just a few students. Romero, Ventura, and García’s (2008) study of types of Moodle students found three clusters defining types of students: very active, active, and non-active. Very active students are characterized as having sent more than one message, read about three messages, passed a large number of quizzes, and spent time on Moodle doing assigned work. Non-active students did no assignments, read few messages, completed very few quizzes, and did not spend time on Moodle. Active student’s behavior was between the other two groups. While messages are one of the most commonly used features in ANGEL, it is up to the individual student to define themselves as very active, active, or non-active.
FUTURE RESEARCH DIRECTIONS Today, the use of the computer has become common for statistical analysis of data. Software packages are easy to use, inexpensive, and fast. But
ANGEL Mining
Figure 4. Number of clusters vs. SSE
today’s vast stores of data with immense data sets make comprehensive analysis all but impossible using conventional techniques. A solution is data mining. Classification mining finds the rules that can predict future behavior. Association mining discovers patterns in the data. Clustering provides insight into the data. Currently data mining requires some specialized skill – the ability to understand the processes, the data, and expertise in the domain. In the case of Learning Management Systems, how the system operates and instruction techniques may be necessary to understand the results. This combination of skills is difficult to find in a single person. In the future data mining tools will become easier to use, as statistical and spreadsheet tools are today. Then domain experts will be able to apply data mining to their data without the need for a data mining expert. The rules found and conclusions drawn can then be used with more confidence when making decisions within the domain. Data mining is commonly conducted against transactional data, but data have gone beyond simple numeric and character flat file data. Future LMS data will come in many forms: image, video, audio, streaming video, and combinations
of data types. Data mining research is being conducted to find interesting patterns in data sets of all these data types. Beyond the LMS data, other data sources may supplement the LMS data. Data from corporate and government data warehouses, transactional databases, and the World Wide Web (Scime, 2008) may be added to enhance the LMS. The data mining of an LMS of the future will be multidimensional, accessing all these data sources and data types to find the optimal use of a school’s LMS. In addition to the new forms of data that will need to be mined in future iterations of the LMS, the very nature of the LMS is likely to see significant changes in approaching years. As colleges and universities become more attune to the needs of both regional (Middle States) and program accreditation (NCATE, AACSB, ABET) the usage of the LMS for purely learning or course management activities is likely to transform to put greater emphasis on assessment and accreditation activities. Where the use of specific system features has not been traditionally mandated, it is likely that there could be a shift to the mandated use of features such as rubrics and assessment tools.
107
ANGEL Mining
With this potential shift comes the opportunity for data mining to expand beyond evaluating just the usage of the LMS but to also provide deep insights into the performance of students against institutional and program objectives. Correlating the use of LMS features with student performance could provide the opportunity to enhance the use of the LMS to provide stronger educational outcomes. This in turn, could provide guidance in improving college and university programs to produce stronger and more competitive classes of graduates. Finding more interesting and supportive rules for a domain using multiple methods places constraints on the mining process. This is a form of constraint-based data mining. Constraint-based data mining uses constraints to guide the process. Constraints that can be used can specify the data mining algorithm. Constraints can be placed on the type of knowledge that is to be found or the data to be mined. Dimension-level constraints research is needed to determine what level of a summary, or the reverse, detail is needed in the data before the algorithms are applied (Hsu, 2002). Research in data mining will continue to find new methods to determine interestingness. Research is needed to determine what values of a particular attribute are considered to be especially interesting in the data and in the resulting rule set (Hsu, 2002). Currently there are 21 different statistically based objective measures for determining interestingness (Tan, Steinbach & Kumar, 2006). A leading area of research is to find new, increasingly effective measures. With regard to subjective measures of interestingness, research in domains that is both quantitative and qualitative can lead to new methods for determining interestingness. Further data mining research will find new methods to support existing knowledge and perhaps find new knowledge in domains where it has not yet been applied (Scime, Murray & Hunter, 2010).
108
CONCLUSION In this investigation, statistical methods were used to determine whether or not features were being used in the ANGEL system. Data mining was able to generate results, and from these results conclusions were made from the three different data mining techniques; classification, association, and clustering. A few general conclusions can be made, the majority of classes at this college do not use the more advanced features of ANGEL. The only parts of the system being used by a large number of classes are the messaging and file sharing features. Another conclusion is that there exists a relationship between need and use. If there are a large number of students, then using ANGEL to send messages or share files alleviates the instructor from a certain amount of additional effort. For example, instead of printing off enough copies for the class, the file is shared on ANGEL. Whereas, classes with a low student count do not use this feature as frequently. Association suggests that there is not a relationship between the use of one feature to the use of another feature. However, association does conclude the non-use of features implies the non-use of other features. Furthermore, during pre-processing, it was found that there are features that are not being used at all (wikis, blogs, assessments, and games). Nevertheless, it has been found that there are a small number of instructors who are heavy ANGEL users. These users often utilize many of the features available. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation or to identify feature areas needing additional training in their use.
ANGEL Mining
REFERENCES Agrawal, R., Imieliński, T., & Swami, A. (1993). Mining association rules between sets of items in large databases, Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington, DC. (pp.207-216). Alsop, G., & Tompsett, C. (2002). Grounded theory as an approach to studying students’ uses of learning management systems. ALT-J Research in Learning Technology, 10(2), 63–76. doi:10.1080/0968776020100207 Avgeriou, P., Papasalouros, A., Retalis, S., & Skordalakis, M. (2003). Towards a pattern language for learning management systems. Journal of Educational Technology & Society, 6(2), 11–24. Bagui, S. (2006). An approach to mining crime patterns. International Journal of Data Warehousing and Mining, 2(1), 50–80. doi:10.4018/ jdwm.2006010103 Beatty, B., & Ulasewicz, C. (2006). Faculty perspectives on moving from blackboard to the moodle learning management system. TechTrends, 50(4), 36–45. doi:10.1007/s11528-006-0036-y Blackboard (2010). Blackboardlearn+ ANGEL edition. Retrieved on April 19, 2010, from http:// www.blackboard.com/Teaching-Learning/LearnResources/ANGEL-Edition.aspx Chapman, B. (2010). LMS KnowledgeBase 2010: In-depth profiles of 90+ learning management systems, with custom comparison across 200+ features. Brandon Hall Research, Retrieved on November 9, 2010, from http://www.brandon-hall. com/publications/lmskb/ lmskb.shtml Commonwealth of Learning (2004) LMS Evaluation Tool User Guide, 1.2, 3waynet Inc. and the Commonwealth of Learning.
Deshpande, M., & Karypis, G. (2002). Using conjunction of attribute values for classification. Proceedings of the 11th International Conference on Information and Knowledge Management, McLean, VA (pp. 356-364). Falvo, D. A., & Johnson, B. F. (2007). The use of learning management systems in the United States. TechTrends, 15(2), 40–45. Feldstein, M., & Masson, P. (2006). Unbolting the chairs: Making learning management systems more flexible. E-learn magazine. Retrieved from http://www.elearnmag.org/subpage.cfm?section= tutorials&article=22-1 on April 19, 2010. Fu, X., & Wang, L. (2005). Data dimensionality reduction with application to improving classification performance and explaining concepts of data sets. International Journal of Business Intelligence and Data mining, 1(1), 65-87. García, E., Romero, C., Ventura, S., & Calders, T. (2007). Drawbacks and solutions of applying Association Rule mining in learning management systems. Proceedings of the International Workshop on Applying Data mining in e-Learning, Crete, Greece (pp. 13-22). Graf, S., & Kinshuk (2006). Considering learning styles in learning management systems: Investigating the behavior of students in an online course. Proceedings of the 1st IEEE International Workshop on Semantic Media Adaptation and Personalization, Athens, Greece (pp. 25-30). Haldane, A. (1998). The convergence of e-learning and knowledge management: Fusion or confusion? People development in a knowledge economy, 2. Learning Futures Ltd., Retrieved on April 26, 2010 from http://www.learningfutures.co.uk/LFdocs/ MKelleher_06-01-03_15-29-27.pdf. Hall, J. (2003). Assessing learning management systems. Chief learning officer. Retrieved on April 19, 2010, from http://www.clomedia.com/ features/2003/January/91/index.php.
109
ANGEL Mining
Hsu, J. (2002). Data mining trends and developments: The key data mining technologies and applications for the 21st century, Proceedings of the19th Annual Conference for Information Systems Education (ISECON 2002), San Antonio, TX (Art 224b). Jaroszewicz, S., & Simovici, D. A. (2004). Interestingness of frequent itemsets using Bayesian networks as background knowledge. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (pp. 178-186). Seattle, WA. Kim, S. W., & Lee, M. G. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24(4), 284–294. doi:10.1111/j.13652729.2007.00260.x Klemettinen, M., Mannila, H., Ronkainen, P., Toivonen, H., & Verkamo, A. I. (1994). Finding interesting rules from large sets of discovered association rules. Proceedings of the 3rd International Conference on Information and Knowledge Management (CIKM’94) Gaithersburg, Maryland, USA (pp. 401-408). Kritikou, Y., Demestichas, P., Adamopoulou, E., Demestichas, K., Theologou, M., & Paradia, M. (2008). User profile modeling in the context of Web-based learning management systems. Journal of Network and Computer Applications, 31(4), 603–627. doi:10.1016/j.jnca.2007.11.006 ANGEL Learning (2008). The ANGEL learning management suite. Lewis, B. A., MacEntee, V. M., DeLaCruz, S., Englander, C., Jeffrey, T., & Takach, E. …Jason W. (2005). Learning management systems comparison. Proceedings of the 2005 Informing Science and IT Education Joint Conference, Flagstaff, Arizona.
110
Li, W., Han, J., & Pei, J. (2001). CMAR: Accurate and efficient classification based on multiple class-Association rules. Proceedings of the 2001 IEEE International Conference on Data mining (pp. 369-376). MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability (pp. 281-297). Malikowski, S. R., Thompson, M. E., & Theis, J. G. (2007). A model for research into course management systems: Bridging technology and learning theory. Journal of Educational Computing Research, 36(2), 149–173. doi:10.2190/10021T50-27G2-H3V7 Moore, M. G. (1993). Theory of transactional distance. In Keegan, D. (Ed.), Theoretical principles of distance education (pp. 33–38). New York, NY: Routledge. Murray, G. R., Riley, C., & Scime, A. (2007, May). A new age solution for an age-old problem: Mining data for likely voters. Paper presented at the 62nd Annual Conference of the American Association of Public Opinion Research, Anaheim, CA. Nichols, M. (2003). A theory for elearning. Journal of Educational Technology & Society, 6(2), 1–10. Osei-Bryson, K.-M. (2004). Evaluation of decision trees: A multicriteria approach. Computers & Operations Research, 31(11), 1933–1945. doi:10.1016/S0305-0548(03)00156-4 Padmanabhan, B., & Tuzhilin, A. (2000). Small is beautiful: Discovering the minimal set of unexpected patterns. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data mining, Boston, MA (pp. 54-63).
ANGEL Mining
Paulsen, M. F. (2003). Online education and learning management systems. Bekkestua, Norway: NKI.
Sturgess, P., & Nouwens, F. (2004). “Evaluation of online learning management systems. Turkish Online Journal of Distance Education, 5(3).
Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco, CA: Morgan Kaufmann.
Tan, P.-N., Steinbach, M., & Kumar, V. (2006). Introduction to data mining. Boston, MA: Addison Wesley.
Rajasethupathy, K., Scime, A., Rajasethupathy, K. S., & Murray, G. R. (2009). Finding “persistent Rules”: Combining association and classification results’, Expert Systems With Applications, 36(3P2), 6019-6024. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j. compedu.2007.05.016 Roqueta, M. (2008). Learning management systems: A focus on the learner. Distance Learning, 5(4), 59–66. Scime, A. (2008). Web page extension of data warehouses. In Wang, J. (Ed.), Encyclopedia of Data Warehousing and Mining (2nd ed., pp. 1211–1215). Hershey, PA: Idea Group Reference. Scime, A. & Murray, G. R. (2007). Vote prediction by iterative domain knowledge and attribute elimination. International Journal of Business Intelligence and Data mining, 2(2), 160-176. Scime, A., Murray, G. R., & Hunter, L. Y. (2010). Testing terrorism theory with data mining. International Journal of Data Analysis Techniques and Strategies, 2(2), 122–139. doi:10.1504/ IJDATS.2010.032453 Sclater, N. (2008). Large-scale open source e-learning systems at open university UK. Research Bulletin EDUCAUSE Center for Applied Research, 2008(12), 1-13.
Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques (2nd ed.). San Francisco, CA: Morgan Kaufman.
ADDITIONAL READING Anand, S. S., Bell, D. A., & Hughes, J. G. (1995). The role of domain knowledge in data mining. Proceedings of the 4th International Conference on Information and Knowledge Management, Baltimore, MD (pp. 37-43). Andoh-Baidoo, F. K., & Kweku-Muata, O.-B. (2007). Exploring the characteristics of internet security breaches that impact the market value of breached firms. Expert Systems with Applications, 32(3), 703–725. doi:10.1016/j.eswa.2006.01.020 Ankerst, M., Martin, E., & Kriegel, H.-P. (2000). Towards an effective cooperation of the user and the computer for classification. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (pp. 179-188). Boston, MA. Bremer, D., & Bryant, R. (2005). A comparison of two learning management systems: Moodle vs blackboard. Proceedings of the 18th Annual Conference of the National Advisory Committee on Computing Qualifications (pp. 135-139). Tauranga, NZ.
111
ANGEL Mining
Brusilovsky, P. (2003). A distributed architecture for adaptive and intelligent learning managament systems. Proceedings of the AIED 2003 Workshop Towards Intelligent Learning Management Systems (pp. 5-13). Sydney, Australia.
Li, J., Tang, J., Li, Y., & Luo, Q. (2009). RiMOM: A dynamic multistrategy ontology alignment framework. IEEE Transactions on Knowledge and Data Engineering, 21(8), 1218–1232. doi:10.1109/ TKDE.2008.202
Carliner, S. (2005). Course management systems versus learning management systems. Learning Circuits, Retrieved from http://www.astd.org/ LC/2005/1105 _carliner.htm on April 19, 2010.
Machado, M., & Tao, E. (2007). Blackboard vs. Moodle: Comparing user experience of learning management systems. Proceeding of the 37th ASEE/IEEE Frontiers in Education Conference, (pp. S4J-7 - S4J-12). Milwaukee, WI.
Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Giarratano, J. C., & Riley, G. D. (2004). Expert systems: Principles and programming (4th ed.). New York: Course Technology. Han, J., & Kamber, M. (2001). Data mining: Concepts and techniques. Boston, MA: Morgan Kaufmann. Hofmann, M., & Tierney, B. (2003). The involvement of human resources in large scale data mining projects. Proceedings of the 1st International Symposium on Information and Communication Technologies (pp.103-109). Dublin, Ireland. Kass, G. (1980). An exploratory technique for investigating large quantities of categorical data. Applied Statistics, 29, 119–127. doi:10.2307/2986296 Kumar, D. A., & Ravi, V. (2008). Predicting credit card customer churn in banks using data mining. International Journal of Data Analysis Techniques and Strategies, 1(1), 4–28. doi:10.1504/ IJDATS.2008.020020 Lane, T., & Brodley, C. E. (2003). An empirical study of two approaches to sequence learning for anomaly detection. Machine Learning, 51(1), 73–107. doi:10.1023/A:1021830128811
112
Magidson, J. (1988). Improved statistical techniques for response modeling. Journal of Direct Marketing, 2(4), 6–18. doi:10.1002/ dir.4000020404 Magidson, J. (1994). The CHAID approach to segmentation modeling: Chi-squared automatic interaction detection. In Bagozzi, R. P. (Ed.), Advanced Methods of Marketing Research. Cambridge, MA: Basil Blackwell. Memon, N., & Qureshi, A. R. (2005). Investigative data mining and its application in counterterrorism. Proceedings of the 5th WSEAS International Conference on Applied Informatics and Communications (pp. 397-403). Malta. Paulsen, M. F. (2003). Experiences with learning management systems in 113 european institutions. Journal of Educational Technology & Society, 6(4), 134–148. Phelps, C., & Michea, Y. (2003). Learning management systems’ evaluation focuses on technology not learning. AMIA Annual Symposium Proceedings, (pp. 969). Ramu, K., & Ravi, V. (2008). Privacy preservation in data mining using hybrid perturbation methods: An application to bankruptcy prediction in banks. International Journal of Data Analysis Techniques and Strategies, 1(4), 313–331. doi:10.1504/IJDATS.2009.027509
ANGEL Mining
Robinson, S. M. (2008). Moodle vs. WebCT: How the design of features affect the efficiency of e-learning by influencing interaction and collaboration. Retrieved from http://web.njit.edu/~sr89/ MoodlevsWebct.doc on April 19, 2010. Roiger, R., & Geatz, M. W. (2003). Data mining: A tutorial-based primer. New York, NY: Addison Wesley. Sclater, N. (2008). Web 2.0, personal learning environments, and the future of LMS. Research Bulletin EDUCAUSE Center for Applied Research, 2008(13), 1-13. Su, X., Khoshgoftaar, T. M., & Greiner, R. (2009). Making an accurate classifier ensemble by voting on classifications from imputed learning sets. International Journal of Information and Decision Sciences, 1(3), 301–322. doi:10.1504/ IJIDS.2009.027657 Tozicka, J., Rovatsos, M., & Pechoucek, M. (2007). A framework for agent-based distributed machine learning and data mining. Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems (art 96). Honolulu, Hawaii. Turban, E., McLean, E., & Wetherbe, J. (2004). Information Technology for Management (3rd ed.). New York, NY: Wiley. Vatsavai, R. R., & Bhaduri, B. (2007). A hybrid classification scheme for mining multisource geospatial data, Proceedings of the 7th IEEE International Conference on Data mining Workshops (ICDMW 2007) (pp. 673-678). Omaha, NE. Wagstaff, K., Cardie, C., Rogers, S., & Schrödl, S. (2001). Constrained k-means clustering with background knowledge. Proceedings of the 18th International Conference on Machine Learning (pp.577-584). Williamstown, MA.
Wang, G., Zhang, C., & Huang, L. (2008). ‘A study of classification algorithm for data mining based on hybrid intelligent systems. Proceedings of the 9th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing (pp. 371-375). Phuket Thailand. Wang, Y., Zhang, Y., Xia, J., & Wang, Z. (2008). Segmenting the mature travel market by motivation. International Journal of Data Analysis Techniques and Strategies, 1(2), 193–209. doi:10.1504/ IJDATS.2008.021118 Webb, G. I., & Zheng, Z. (2004). Multistrategy ensemble learning: Reducing error by combining ensemble learning techniques. IEEE Transactions on Knowledge and Data Engineering, 16(8), 980–991. doi:10.1109/TKDE.2004.29
KEY TERMS AND DEFINITIONS Association Mining: A data mining method used to find patterns of data that show conditions where sets of attribute-value pairs occur frequently in the data set. Association Rule: A rule found by association mining. Classification Mining: A data mining method used to find models of data for categorizing instances; typically used for predicting future events from historical data. Classification Rule: A rule found by classification mining. Clustering: A data mining method used to group similar records together based on a measurable distance between the records. Clustering Rule: A rule found by the analysis of clusters. Data Mining: A collection of processes that inductively analyze data to assess known relationships as well as to find interesting patterns and unknown relationships.
113
ANGEL Mining
APPENDIX A: ASSOCIATION MINING RULES
countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6545 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6502 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 6496 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 6493 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 6463 conf:(0.97) countFolder=(-inf-0.5] 6649 ==> countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6463 conf:(0.97) countDropBoxes=(-inf-0.5] 8366 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7573 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countTurnItIn=(-inf-0.5] 8306 conf:(0.92) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countPage=(-inf-0.5] 8184 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countTurnItIn=(inf-0.5] 8161 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] 8137 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 8130 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 8032 conf:(0.91) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countDropBoxes=(-inf-0.5] countQuiz=(inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6713 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countLink=(inf-0.5] countTurnItIn=(-inf-0.5] 6704 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countForm=(inf-0.5] countLink=(-inf-0.5] 6694 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countQuiz=(-inf-0.5] countForm=(inf-0.5] countLink=(-inf-0.5] countTurnItIn=(-inf-0.5] 6684 conf:(0.9)
114
ANGEL Mining
countLink=(-inf-0.5] 7955 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7312 conf:(0.92) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] 7206 conf:(0.91) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7198 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7197 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countPage=(-inf-0.5] 7170 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7164 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.93) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7744 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7714 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] 7682 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countTurnItIn=(-inf-0.5] 7668 conf:(0.9) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7963 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 8306 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 8278 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 8202 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countDropBoxes=(-inf-0.5] countForm=(inf-0.5] 8161 conf:(0.9)
115
116
Chapter 6
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems Kamla Ali Al-Busaidi Sultan Qaboos University, Oman Hafedh Al-Shihi Sultan Qaboos University, Oman
ABSTRACT Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS. DOI: 10.4018/978-1-60960-884-2.ch006
Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
INTRODUCTION Information and communication technologies (ICT) and the Internet have become major enablers of growth in business. The geographical outreach of the Internet and the wide global adoption of Web 2.0 technologies provide educational institutions with unprecedented opportunities to enhance their offerings. These technologies have transformed students’ perception of information and what they think about Web content and how to use it (Burnett & Marshall, 2003). Schools and universities are forced to investigate new means to revamp the educational process utilizing these technologies. Learning Management Systems (LMS) and e-learning have become the hype lately among stakeholders in education and training. The elearning market was worth more than US $18 billion worldwide in 2004 (Saady, 2005). Several leading universities around the world have adopted LMS for teachers and students to enhance the educational process (Hawkins & Rudy 2007; Browne, Jenkins & Walker, 2006; National Center for Educational Statistics, 2003). About 95 percent of participating institutions in the UK have adopted LMS (Browne et al., 2006); likewise, more than 90 percent of all participating universities and colleges in the US are adopting LMS (Hawkins & Rudy, 2007). Users’ acceptance and actual use of information systems is critical to its success. Likewise for a learning management system, its success to a great extent depends on users’ acceptance and use. Evaluating individual users’ acceptance and use of the e-learning systems is a “basic marketing element” (Kelly & Bauer, 2004). Although a learning management system survives through instructor and student use, the adoption of LMS is initiated by instructors’ acceptance and use. Even when LMS are well in place, instructors may not fully utilize all the features. For example, a survey of more than 800 instructors at over 35 institutions using Blackboard learning management system found that very few teachers use LMS tools for assessing
students or promoting community (Woods, Baker & Hopper., 2004). In addition, research indicates that fear of technology and lack of time may limit instructors’ adoption of LMS (Yueh & Hsu, 2008). Instructors should embrace and prepare for LMS use before preparing students for online learning (Chan, 2008). Even when designing LMS applications, teachers’ needs and capabilities should thoroughly be investigated (Yueh & Hsu, 2008). Therefore, teachers’ perspective within the context of LMS is crucial and should be studied carefully to ensure comprehensive uptake of LMS. Thus, the objective of this study was to investigate the critical factors influencing instructors’ acceptance and use of LMS, which may be influenced by technical and non-technical issues such as the instructors’ personal characteristics and the organization’s characteristics. It is important to analyze non-technical factors that promote the adoption and diffusion of LMS initiatives (Albirini, 2006; ElTartoussi, 2009). Consequently, this study specifically aimed to examine the impact of instructor’s individual characteristics (computer anxiety, technology experience and self efficacy), LMS characteristics (system quality, information quality and service quality), and organizational characteristics (management support, training and incentives) on the instructors’ acceptance (perceived ease of use and perceived usefulness) and use of LMS as a supplementary tool. The study also assessed the impact of the instructors’ acceptance and use of LMS on their intention for continuous supplementary use of LMS and intention for pure use of LMS for distance education. Many organizations start their LMS adoption as a supplementary tool to traditional teaching, hoping that this supplementary adoption eventually will promote the pure use of LMS for distance education. The following sections discuss the background literature, research framework and methodology, analysis and results, and the conclusion.
117
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
BACKGROUND Learning Management Systems A learning management system is “software that automates the administration of training” (World Bank, 2010). It is the use of a Web-based communication, collaboration, learning, knowledge transfer, and training to add value to learners and businesses (Kelly & Bauer, 2004). Specifically, a learning management system is an Internet application that aims to support education and training activities (Cavus & Momani, 2009) and provides a platform to support e-learning activities (Yueh & Hsu, 2008). Course Management Systems (CMS) and Learning Content Management Systems (LCMS) are sometimes used to indicate LMS (Yueh & Hsu, 2008); other related terms are Computer-assisted Learning (CAL), Computer-based Learning (CBL), and Online Learning (Chan, 2008). It should be noted, however, that LMS applications are not unique to educational institutions; even public and private organizations use such systems for training purposes. Reiser and Dempsey (2002) stated that in large US corporations, 20 percent of all training is delivered via LMS.
LMS Applications and Benefits to Instructors Many LMS applications are available. The most popular LMS used at colleges and universities in the US is Blackboard followed by WebCT, which was acquired by Blackboard, Inc. in 2006 (Falvo & Johnson, 2007). WebCT was used by millions of students from more than 2,500 universities in more than 80 countries (Chan, 2008). Other LMS solutions are Moodle, ATutor, Learn.com, Joomla, and Krawler. Blackboard has course management features that support integration with student databases (Kneght & Reid 2009; Blackboard, 2009). WebCT supports electronic communications such as email, bulletin boards, and chat rooms (Chan,
118
2008). Moodle learning management system, on the other hand, is sometimes preferred over the previous popular LMS packages (see, for example, Beatty and Ulasewicz, 2006). It is scalable open-source software used mainly by North American and European universities that supports group forum participation and features interactive tools between students and instructors (Beatty & Ulasewicz, 2006). LMS applications provide several features to instructors, especially to those who see the real benefits of the Internet in the teaching process (Burniske & Monke, 2001). Yueh and Hsu (2008) described course management tools, group chat and discussion, assignment submission, and course assessment as the primary tools in LMS. In addition, LMS help teachers provide students with educational materials and track their participation and assessments (Falvo & Johnson, 2007). Yildirim, Temur, Kocaman and Goktas (2004) described more technically sophisticated LMS features such as maintaining office hours online, creating student groups, and assigning online projects to groups. Ceraulo (2005) mentioned ePortfolios as a key feature in some LMS applications, which enable instructors to maintain student submissions throughout the course (i.e., tests, assignments, projects). LMS solutions tend also to increase interest in learning and teaching among students and teachers, respectively (Mahdizadeh, Biemans & Mulder, 2008). Aczel, Peake and Hardy (2008), and Naidu (2006) stated that LMS enhance teaching process efficiency and result in cost-savings.
Learning Management Systems Acceptance and Use User Technology Acceptance and Use LMS have been adopted by academic institutions to support their distance education and/or supplement their traditional way of teaching (Rainer, Turban & Potter 2007). User acceptance and use
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
of information systems is critical to their success. The same is true for a learning management system: its success, to a great extent, depends on users’ acceptance and use. The assessment of technology success has been conducted by utilizing various acceptance and use dimensions. Several studies assessed the determinants of technology acceptance (Bailey & Pearson, 1983; Davis, 1989; DeLone & McLean, 2003; Doll & Torkzadeh, 1988; Venkatesh & Davis, 2000). For instance, Davis (1989) assessed technology acceptance by perceived usefulness and intention to use; Venkatesh and Davis (2000) assessed technology acceptance by perceived usefulness, intention to use, and usage behavior. Alternatively, DeLone and McLean (2003) assessed technology success by user satisfaction and usage. In the LMS context, researchers also assessed instructors’ acceptance and use in various ways. Liaw, Huang and Chen (2007) assessed LMS acceptance by learners’ and instructors’ behavioral intention to use e-learning, which is influenced by perceived usefulness, perceived self-efficacy, and perceived enjoyment. Ball and Levy (2008) assessed LMS acceptance by instructors’ intention to use. Teo (2009) assessed LMS acceptance by teachers’ perceived usefulness and perceived ease of use. None of these studies, however, investigated the direct impact of instructors’ characteristics, LMS’ characteristics, and/or an organization’s characteristics on actual system use. Perceived usefulness, perceived ease of use, users’ satisfaction, and intention to use are important measures for technology acceptance and may eventually correlate with actual use behavior. Some researchers have found direct effects between such external factors and technology use (Igbaria, Guimaraes & Davis, 1995). Nevertheless, these acceptance measures do not explain all the variance of actual usage behavior. In addition, measuring attitudes and their link to actual usage behavior is extremely difficult; therefore, many researchers may choose
to stay with actual use behavior (DeLone & McLean, 2003). Thus, it is important to evaluate the direct impact of these critical determinants on actual system use. System use is and will continue to be an important indication of IS success (DeLone & McLean, 2003). In addition, the actual benefits of any technology may be realized only from actual technology use. Thus, technology use is the main aim of organizations to promote LMS and realize some of its expected benefits. Perceived usefulness and ease of use may be important factors for continuous intention to use. Consequently, As indicated earlier this study aimed to examine the direct impact of instructors’ characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ actual usage behavior, instructors’ perceived ease of use of LMS, and instructors’ perceived usefulness of LMS. Furthermore, the study examined the impacts of these acceptance (perceived ease of use, perceived usefulness), and actual use measures on continuous intention for supplementary use and intention for pure use of the LMS for distance education. Perceived usefulness and perceived ease of use may be important to ensure continuous use of the system and future intention to adopt it for distance education.
Instructor Characteristics The acceptance and use of LMS may, to a great extent, be determined by the characteristics of its users. Several dimensions of users’ characteristics have been proposed and investigated as determinants of technology acceptance. Some of these are users’ experience of technology, users’ self-efficacy, and users’ computer anxiety. Users’ experience of technology has been highly utilized to determine users’ technology acceptance (Venkatesh & Davis, 2000; Thompson, Compeau, Deborah & Higgins, 2006). Individual technology experience is the individual’s exposure to the technology as well as the skills and abilities that
119
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
s/he gains through using a technology (Thompson et al., 2006); users’ self-efficacy pays a major role in users’ acceptance and use of technology. Self-efficacy is defined as “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (Bandura, 1977, p.391). Thus, computer self-efficacy means individual’s selfassessment of their ability to apply computer skills to accomplish their tasks (Compeau & Higgins, 1995). Furthermore, a user’s computer anxiety is considered an important factor for technology acceptance and use. Computer anxiety is defined as “the fear or apprehension felt by individuals when they used computers, or when they considered the possibility of computer utilization” (Simonson, Maurer, Montag-Torardi & Whitaker, 1987, p. 238). Fear of computers may negatively impact technology acceptance and use. An individual’s computer anxiety, self efficacy and experience with technology may share some correlations, but they are not exactly similar individual traits (Ball & Levy, 2008). Computer anxiety is not simply a negative, short-term attitude toward computers that can be reduced by increasing technology experience; it is the users’ individual fear associated with computer use with or without experience (Ball & Levy, 2008). Likewise, individuals’ self efficacy is an individual trait that some individuals might have with or without technology experience. Some individuals are capable to accept and use LMS without a prior experience with technology because they have high level of self efficacy. In the context of e-learning, few studies have investigated the impact of instructors’ dimensions on LMS acceptance. Ball and Levy (2008) investigated the impact of self-efficacy, computer anxiety, and technology experience on instructors’ intention to use emerging learning experience in a small private university in the US and found that self-efficacy was the only major determinant of instructors’ intention. Teo (2009) found that computer self-efficacy directly impacts pre-service
120
teachers’ perceived usefulness, perceived ease of use, and behavioral intention in Singapore. Liaw et al. (2007) found that perceived self-efficacy determines instructors’ behavioral intention to use e-learning in Taiwan. Albirini (2006) investigated the perception of school teachers of the use of ICT in education in Syria, and the results highlighted the importance of teachers’ vision of technology, their experiences with it, and the cultural conditions on their attitudes toward technology. Mahdizadeh and his colleagues (2008) found that teachers’ previous experience with e-learning environments and ease of use explain teachers’ perception of the usefulness of e-learning environments and their actual use of these environments.
LMS Characteristics The characteristics of LMS may have a great impact on the instructor’s acceptance and use of LMS. Characteristics of any information system, including LMS, may be related to system, information, and service support quality as classified by DeLone and McLean (2003). E-learning systems’ quality was found to be significant on the instructors’ perceived usefulness, perceived enjoyment, and perceived self-efficacy, which consequently affect their intention to use the system in the classroom (Liaw et al., 2007). System quality, which refers to the characteristics of a system, is a key in users’ acceptance and use of any technology, including LMS. Researchers, such as Bailey and Pearson (1983), DeLone and McLean (2003), and Seddon (1997) have highlighted the impact of system quality on technology acceptance and introduced several ways to measure system quality. Information quality, which refers to the perceived output produced by the system, is also an important factor in instructors’ acceptance and use of LMS. Information quality refers to the accuracy, relevance, timeliness, sufficiency, completeness, understandability, format, and accessibility of the information (Bailey
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
& Pearson, 1983; Seddon, 1997). In addition, service quality may be a factor on the instructors’ acceptance and use of LMS; it refers to the quality of support services provided to the system’s endusers. DeLone and McLean (2003) indicated that ignoring service quality endangers IS effectiveness measurements. Common measurements of service quality are tangibles, reliability, responsiveness, assurance, and empathy (Parasuraman, Zeithaml, & Berry, 1988; Kettinger & Lee, 1994). In the e-learning context, few studies have examined the general quality of technology or specific dimension. For instance, from instructors’ and learners’ perspective, Liaw et al. (2007) investigated the impact of e-learning systems’ general quality on perceived usefulness, perceived enjoyment, and perceived self-efficacy, which consequently affect their intention to use the system in the classroom, and found it significant. Albirini (2006) indicates that instructors’ vision of technology impacts their attitudes toward the use of ICT in education. Two significant studies on the impact of technology on users’ acceptance of LMS are Pituch and Lee’s (2006) and Roca, Chiu and Martinez’s (2006), but they are from the learners’ perspective. Roca et al. (2006) investigated learners’ perceived system quality from three dimensions (system quality, information quality, and service quality). They found that learners’ perceived system quality factors (system quality, information quality, and service quality) directly affect their e-learning satisfaction and intention to use and indirectly their perceived usefulness. Pituch and Lee (2006) examined the impact of system quality from three dimensions: the system’s functionality, interactivity, and response. As indicated, limited studies provide a detailed examination of the influence of the three dimensions (system quality, information quality, service quality) of LMS on instructors’ acceptance. This study integrates these three dimensions of LMS on the instructors’ acceptance.
Organization Characteristics An organization’s characteristics play a major role in the behaviors of its employees, including the acceptance and use of any technology such as LMS. Corporate culture plays a key role in the success of any project. Schein defines culture as “the way we do things around here” (1985, p. 12). Cultural values shape an organization’s norms and practices, which consequently influence employees’ behaviors such as LMS utilization. Some of an organization’s characteristics that might be relevant to the utilization of LMS are management support, incentives, and training. Organizational support, represented by senior managers’ support, is also important for instructors to accept and use LMS in their teaching. Management’s support of end-users significantly improves computer usage (Igbaria, 1990). In the e-learning context, senior management support and the alignment of e-learning with the department and university curriculum are important for its adoption (Sumner & Hostetler, 1999). Motivators are also an important factor for instructors’ acceptance to integrate the technology in teaching. Motivators or incentives for instructors can be enforced by having the use of the technology as a factor in a nomination for teaching award, promotion, and tenure (Sumner & Hostetler, 1999). Finally, training end-users is important, and can be in form of workshops, online tutorials, courses, and seminars. There is a lack of empirical studies that capture the influence of organization factors on the acceptance and use of LMS generally. One of these is a qualitative study by Sumner and Hostetler (1999), who categorize the organizational factors that may influence the use of technology in teaching in terms of motivators/demotivators, training, technology alignment, and organizational and technical support. In addition, Teo (2009) found that facilitating conditions, measured by technical support, training, and administrative support, indirectly affect teachers’ acceptance of technology in education.
121
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
INSTRUCTORS’ LMS ACCEPTANCE AND USAGE FRAMEWORK Framework Development This study aimed to examine the impact of instructor’s individual characteristics, LMS’ characteristics, and an organization’s characteristics on instructors’ acceptance ad usage of LMS as a supplementary tool and, consequently, on continuous use and pure use intention for distance learning. As indicated, few studies have examined this integrated investigation of instructors’ LMS acceptance and usage. This study assessed the individual characteristics based on instructors’ computer anxiety and technology experience, LMS characteristics based on system, information, and service quality; and organizational characteristics based on management support, incentives and training. The study assessed the impact of these factors on the instructors’ acceptance (perceived ease of use and perceived usefulness) Figure 1. Instructors LMS acceptance and use model
122
and usage of LMS and, consequently, continuous supplementary use and future pure use intention. Figure 1 illustrates this study model.
Instructor Individual Characteristics Hypotheses Computer Anxiety Hypotheses Computer anxiety is “the fear or apprehension felt by individuals when they used computers, or when they considered the possibility of computer utilization” (Simonson, et al., 1987, p. 238). Computer anxiety is an important factor for the acceptance of the technology (Ball & Levy, 2008; Piccoli, Ahmad & Ives, 2001; Raaij & Schepers, 2008; Sun, Tsai, Finger, Chen & Yeh, 2008). Fear of computers may negatively affect the acceptance of LMS and the user’s perceived satisfaction (Piccoli et al., 2001). Empirical evidence of the impact of computer anxiety was mixed. Ball and Levy (2008) did not detect a significant link between
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
computer anxiety and instructors’ intention to use the e-learning; however, Sun et al.(2008) found that computer anxiety significantly impacts the learners’ perceived satisfaction of e-learning, and Raaij and Schepers (2008) found the computer anxiety impacts the learner’s perceived ease of use of e-learning. Therefore we hypothesized that: Hypothesis 1a: Instructors’ computer anxiety is negatively associated with perceived ease of use of LMS. Hypothesis 1b: Instructors’ computer anxiety is negatively associated with perceived usefulness of LMS. Hypothesis 1c: Instructors’ computer anxiety is negatively associated with use of LMS.
Technology Experience Hypotheses Users’ experience with the technology (EUT) also plays a major role in the acceptance of technology (Venkatesh & Davis, 2000; Thompson et al., 2006). An individual’s EUT is his/her exposure to the technology as well as the skills and abilities that are gained through using a technology (Thompson et al., 2006). Therefore, EUT may impact instructors’ acceptance of LMS for their classes. Although empirical quantitative research, such as that of Ball and Levy (2008), found no significant impact of EUT on instructors’ intention to use LMS, researchers Sumner and Hostetler (1999) indicate that current level of computer skills and extent of use of computing skills in teaching are important for instructors’ acceptance of ICT in education. Likewise, Wan, Fang and Neufeld (2007) highlight the importance of technology experience on the learning processes and, consequently, learning outcomes. Mahdizadeh et al. (2008) reveal that instructors’ prior experience with e-learning may explain their perception of the usefulness of e-learning environments and their actual use. Therefore we hypothesized:
Hypothesis 2a: The instructor’s experience with the use of technology is positively associated with perceived ease of use LMS. Hypothesis 2b: The instructor’s experience with the use of technology is positively associated with perceived usefulness of LMS. Hypothesis 2c: The instructor’s experience with the use of technology is positively associated with use of LMS.
Self-Efficacy Hypotheses Self-efficacy is “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (Bandura, 1977, p.391). Thus, computer self-efficacy means individuals’ selfassessment of their ability to apply computer skills to accomplish their tasks (Compeau & Higgins, 1995). Several empirical studies found significant effects of computer self-efficacy on the perceived usefulness on information systems (Venkatesh & Davis, 2000; Chau, 2001). The more ability the instructor has to apply a computer application, the most likely s/he perceives it easy to use and useful, and will eventually use it. In the context of e-learning systems, Ball and Levy (2008) found a significant effect of self-efficacy on instructors’ intention to use. In addition, computer self-efficacy was found to be significant on learners’ perceived ease of use (Lee, 2006; Pituch & Lee, 2006; Roca et al., 2006) and learners’ perceived satisfaction (Sun et al., 2008). Therefore, we hypothesized: Hypothesis 3a: An instructor’s computer selfefficacy is positively associated with perceived ease of use of LMS. Hypothesis 3b: An instructor’s computer selfefficacy is positively associated with perceived usefulness of LMS. Hypothesis 3c: An instructor’s computer selfefficacy is positively associated with use of LMS.
123
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
LMS Characteristics Hypotheses System Quality Hypotheses System quality is essential for the acceptance of any technology, including LMS. Researchers, such as Bailey and Pearson (1983), DeLone and McLean (1992), and Seddon (1997) highlight the impact of system quality on technology acceptance and have introduced several ways to measure it. Instructors’ acceptance of LMS may be determined to a great extent by system quality. The more functionalities, interactivity, and response of LMS, the better is its acceptance and utilization (Pituch & Lee, 2006). Quantitative empirical studies found a significant impact of system characteristics on e-learning acceptance: reliability (Wan et al., 2007; Webster & Hackley, 1997); accessibility (Wan et al., 2007); and system functionality, interactivity, and response (Pituch & Lee, 2006). Albirini (2006) indicates that instructors’ vision of technology impacts their attitudes toward the use of ICT in education. Therefore, we hypothesized that: Hypothesis 4a: LMS system quality is positively associated with the instructor’s perceived ease of use LMS. Hypothesis 4b: LMS system quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 4c: LMS system quality is positively associated with the instructor’s use of LMS.
Information Quality Hypotheses Information quality is also important for instructors’ acceptance of LMS, and refers to the perceived output produced by the system. Information quality with great accuracy, relevance, timeliness, sufficiency, completeness, understandability, format, and accessibility are important for the acceptance of an information technology (Bailey & Pearson, 1983; Seddon, 1997). There is a lack
124
of research on the impact of information quality on instructors’ acceptance of LMS. Some research was conducted from the learners’ perspective. Roca et al. (2006) measured information quality of LMS by indicators related to relevance, timeliness, sufficiency, accuracy, clarity, and format, and proved that information quality was directly significant for learners’ satisfaction and indirectly for perceived usefulness. Likewise, Lee (2006) found content quality was significant for learners’ perceived usefulness. In the Middle East, Al-Busaidi (2009), in an exploratory study in Oman, indicated that information quality (sufficiency, accuracy, relevance, timeliness, and understandability) was highlighted as a determinant of learners’ LMS use. Consequently, we hypothesize that: Hypothesis 5a: LMS information quality is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 5b: LMS information quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 5c: LMS information quality is positively associated with the instructor’s use of LMS.
Service Quality Hypotheses Service quality refers to the quality of support services provided to the system’s end-users. Instructors’ acceptance of LMS may, to some extent, be related to the quality of the support services. Common measurements of service quality are tangibles, reliability, responsiveness, assurance, and empathy (Parasuraman et al., 1988; Kettinger & Lee, 1994). Few studies have investigated the impact of service quality on LMS acceptance. For instance, Roca et al. (2006) assessed service quality by indicators related to responsiveness, reliability, and empathy, and confirmed its direct significance on learners’ satisfaction and indirect significance of perceived usefulness in the e-learning context. Thus, we hypothesized that:
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
Hypothesis 6a: LMS service quality is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 6b: LMS service quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 6c: LMS service quality is positively associated with the instructor’s use of LMS.
Hypothesis 7a: Management support is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 7b: Management support is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 7c: Management support is positively associated with the instructor’s use of LMS.
Organization Characteristics Hypotheses
Incentives Hypotheses
Management Support Hypotheses Management support is a key factor for the acceptance of any organizational initiative. Senior managers’ open approval and endorsement of LMS adoption promote instructors’ adoption and acceptance of LMS. Managers may support an LMS by encouraging instructors to adopt it and identify a clear vision of the objective of the LMS and how it is aligned with the university vision. Little research has investigated the impact of management support on instructors’ acceptance of LMS. However, in the e-learning context, senior managers should clearly identify the goal of LMS for the university curriculum (Sumner & Hostetler, 1999). This managers’ support assures instructors that using LMS is part of the organization’s culture and is useful and encourages them to adopt and use the system. Managers are recognized as a high authority (Ali, 1990); thus, instructors’ adoption and acceptance of LMS may be associated with the endorsement of their senior managers. Management support of end-users significantly improves computer usage (Igbaria, 1990). Facilitating conditions, including administrative support, indirectly affect teachers’ acceptance of technology in education (Teo, 2009). Consequently, we hypothesized that:
Motivators, in terms of incentives, are important factors for instructors’ acceptance to integrate LMS in their teaching. Incentives can be “nontrivial” monetary and non-monetary incentives. E-learning research lacks the assessment of incentives on LMS acceptance. Motivators or incentives for instructors can be enforced by using the technology as a factor in nomination for a teaching award, promotion, and tenure (Sumner & Hostetler, 1999). These incentives’ policies push instructors to adopt and utilize LMS for their teaching. Therefore, we hypothesized that: Hypothesis 8a: An incentive policy is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 8b: An incentive policy is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 8c: An incentive policy is positively associated with the instructor’s use of LMS.
Training Hypotheses Providing end-users with training is important, as training improves instructors’ adoption of LMS and enhances the perceived ease of use of LMS, illustrates its potential usefulness, and encourages its use in teaching. Limited research has investigated the impact of training on instructors’ acceptance of LMS, which can be in the form of workshops, online tutorials, courses, and seminars (Sumner &
125
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
Hostetler, 1999). Facilitating conditions, including training, indirectly affect teachers’ acceptance of technology in education (Teo, 2009). Thus, we hypothesized: Hypothesis 9a: Training is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 9b: Training is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 9c: Training is positively associated with the instructor’s use of LMS.
Usage and Future Intention Hypotheses Perception and Usage Hypotheses A technology’s perceived ease of use and perceived usefulness are found to be a significant determinant of the intention to use the technology (Venkatesh & Davis, 2000). The higher the instructors’ perceived ease of use and perceived usefulness of LMS, the higher the actual use. Perceived ease of use is also a significant determinant of users’ perceived usefulness of a technology (Venkatesh & Davis, 2000). Pituch and Lee (2006) found learners’ perceived ease of use of LMS to significantly affect perceived usefulness. Therefore, we hypothesized: Hypothesis 10a: Instructors’ perceived ease of use of LMS is positively associated with their use of LMS. Hypothesis 10b: Instructors’ perceived usefulness of LMS is positively associated with their use of LMS. Hypothesis 10c: Instructors’ perceived ease of use of LMS is positively associated with their perceived usefulness of LMS.
126
Continuous Supplementary Use Intention Hypotheses The intention to use the technology is significantly determined by users’ perceived ease of use and perceived usefulness (Venkatesh & Davis, 2000). The higher the instructors’ perceived ease of use of LMS, perceived usefulness of LMS, and actual use, the more likely it is that they will continue to use it. Continuous intention to e-learning use is determined by perceived usefulness and satisfaction (Hayashi, Chen, Ryan, Wu, 2004). Thus, we hypothesized: Hypothesis 11a: The instructors’ perceived ease of use of LMS is positively associated with their continuous supplementary use intention. Hypothesis 11b: The instructors’ perceived usefulness of LMS is positively associated with their continuous supplementary use intention. Hypothesis 11c: The instructors’ supplementary use of LMS is positively associated with their continuous supplementary use intention.
Future Pure Use Intention Hypotheses Many organizations begin their LMS adoption as a supplementary tool to traditional teaching, hoping that this supplementary adoption will eventually promote the pure use of LMS for distance education. Perceived ease of use, perceived usefulness, and actual use may have an important impact on continuous intention for supplementary use and intention for pure use of the LMS for education. When instructors believe that LMS is easy, useful, and can be utilized for supplementary purposes, they are more likely to adopt it purely for distance education. Technology perceived ease of use and perceived usefulness are found to be significant determinants of the intention to use the technology (Venkatesh & Davis, 2000). Perceived ease of use, perceived usefulness, and supplementary
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
use are significant determinants of learners’ use of e-learning for distance education (Pituch & Lee, 2006). Thus we hypothesized: Hypothesis 12a: The instructors’ perceived ease of use of LMS is positively associated with their pure use intention. Hypothesis 12b: The instructors’ perceived usefulness of LMS is positively associated with their pure use intention. Hypothesis 12c: The instructors’ supplementary use of LMS is positively associated with their pure use intention.
METHODOLOGY Participants’ Profile This study included 82 instructors from Sultan Qaboos University (SQU), the first and only public university in Oman. Since its launch, SQU has gone through many technology developments: it adopted WebCT and later switched to the open-source Moodle application. Instructors can voluntarily adopt LMS to supplement their traditional classes. The instructors were from different colleges in the university and with different demographics. About 62 percent were male and 38 percent were female. About 5 percent were assistant lecturers, 27 percent were lecturers, 50 percent were assistant professors, 13 percent were associate professors, and 5 percent were full professors. The instructors’ age varied from 20s to above 50s: about 8 percent were in their 20s, 26 percent were in their 30s, 16 percent in their 40s, and 32 percent were 50 or over. Almost 44 percent had less than six years of work experience, 30 percent had less than 11 years, 16 percent had less than 16 years, 7 percent had less than 21 years, and 2 percent had more than 20 years. Most indicated that their computer skills were above average. Almost 71 percent have above average computer skills; 23
percent, about average; and only 6 percent were below average. The majority, about 59 percent, has used the LMS for classes for three years or more; 30 percent have used it for one to two years; and 11 percent have used it for less than one year. The majority of the instructors, about 55 percent, have experience with only Moodle LMS; 9 percent have experience with only WebCT LMS; 31 percent have experience with both WebCT and Moodle; and 5 percent have experience with Blackboard.
Research Questionnaire The questionnaire was distributed to SQU instructors. An invitation email was sent to instructors to complete the study questionnaire either online or on an attached MS Word document. A reminder was sent two weeks after the initial invitation. Most of the instructors filled the questionnaire online (about 95 percent of them).Only five percent of instructors completed the questionnaire as a hard copy. The questionnaire included the constructs to be measured for quantitative analysis, along with demographic questions (e.g., gender, age, degree, LMS usage experience, work experience, and job title). Construct measurements items were phrased according to a five–point Likert scale (1= strongly disagree; 2=disagree; 3=Neutral; 4= agree and 5=strongly agree). To statistically evaluate the study framework, 28 indicators were used. Tables 1 and 2 show the total indicators used for each construct. The LMS characteristic constructs (system quality, information quality, and service quality) were adopted and modified from Roca et al. (2006) and Pituch and Lee (2006); the individual characteristics constructs (computer anxiety, self-efficacy, and technology experience) were adopted from Ball and Levy (2008); while the organizational characteristics’ constructs (management support, incentives, and training) were self-developed, based on Sumner and Hostetler (1999). The LMS acceptance construct (perceived ease of use and perceived
127
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
Table 1. Independent constructs measures and loadings Construct Measures
Loading
Computer Anxiety 1. I believe that working with computers is very difficult.
0.8913
2. Computers make me feel uncomfortable.
0.9560
3. I get a sinking feeling when I think of trying to use a computer.
0.8698
Technology Experience 1. I feel confident using the e-learning system
0.8465
2. I feel confident downloading/uploading necessary materials from the Internet.
0.8756
3. I feel confident using the chatting and discussion forums.
0.6275
Self Efficacy 1. I could use the e-learning system if I had never used a system like it before.
-0.7948
2. I could use the e-learning system if I had only the system manuals for reference.
0.0894
3. I could use the e-learning system if I had seen someone else using it before trying it myself.
0.8652
System Quality 1. The system offers flexibility in teaching as to time and place.
0.7694
2. The system offers multimedia (audio, video, and text) types of course content.
0.7962
3. The response time of the system is reasonable.
0.6025
4. The system enables interactive communication between instructor and students.
0.7737
Information Quality 1. The information provided by the system is relevant for my job.
0.8434
2. The information in the system is very good.
0.8919
3. The information from the e-learning system is up-to-date.
0.8407
4. The information provided by the system is complete.
0.8494
Service Quality 1. The system support services give me prompt service.
0.8425
2. The system support services have convenient operating hours.
0.8410
3. The system support services are reliable.
0.8761
4. The system support services are easy to communicate with.
0.8904
Management Support 1. Senior administrators strongly support the use of e-learning system.
0.9396
2. I get support by department chair or dean on my use of e-learning system.
0.8506
3. My mangers highlight the importance of e-learning system on my curriculum.
0.9188
Incentives 1. The use of e-learning is a factor in the nomination for teaching award.
0.9529
2. The use of e-learning system is a factor in determining promotion.
0.9558
3. The use of e-learning system is a factor in annual elevation of teaching.
0.9611
Training
128
1. I receive training workshops on how to use e-learning tools.
0.7277
2. I receive on-line manuals on how to use e-learning tools.
0.8759
3. I receive seminars on the use of e-learning tools.
0.8378
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
usefulness) were adopted and modified from Venkatesh and Davis (2000), and supplementary use, continuous supplementary use, and future pure use were adopted and modified according to Pituch and Lee (2006).
DATA ANALYSIS & RESULTS PLS Analysis Methodology Data was analyzed by PLS-Graph 3.0 software. PLS (partial least square) is a variance-based structural equation model (SEM) technique that allows path analysis of models with latent variables
(Chin, 1998; Chin, 2001). The PLS approach is a variance-based SEM that assists researchers in obtaining determinate values of latent variables for predictive purposes. The PLS does that by minimizing the variance of all dependent variables rather than using the model to explain the co-variation of all indicators (Chin, 1998; Chin and Newsted, 1999). Thus, the model paths are estimated based on the ability to minimize the residual variances of the dependent variables. The PLS algorithm uses an iterative process for the estimation of weights and latent variables scores. The process almost converges to a stable set of weight estimates. The evaluation of the model is based on (1) the assessment of the model mea-
Table 2. Dependant constructs measures and loadings Construct Measures
Loading
Perceived Ease of Use 1. Using e-learning tools is easy to me.
0.8896
2. E-learning tools are clear and understandable to me.
0.9280
3. I find it easy to get the e-learning system to do what I want it to do.
0.7843
Perceived Usefulness 1. Using e-learning system enables me to accomplish tasks more quickly.
0.8581
2. Using e-learning system improves my performance.
0.9251
3. Using e-learning system increases my productivity.
0.9255
4. Using e-learning system enhances the effectiveness on the job.
0.8958
5. Using e-learning system gives me greater control over my work.
0.8831
Supplementary Use 1. I use the e-learning system as many occasions as possible to supplement my teaching.
0.9165
2. I use the e-learning system on regular basis to supplement my teaching.
0.9047
3. I frequently use the e-learning system to supplement my teaching.
0.9051
4. I use the e-learning system to share/seek course information.
0.7357
5. I use the e-learning system to communicate with students
0.8259
Continuous supplementary Use Intention (CUI) 1. I will frequently use e-learning system to do a teaching task.
0.8871
2. I will use e-learning system on regular basis to supplement my classes in the future.
0.8523
3. I will always try to use the e-learning system to do a teaching task whenever it has a useful feature.
0.8915
Pure Use Intention (PUI) 1. I plan to teach purely online courses for distance learners.
0.9249
2. I will use e-learning system to teach purely online courses.
0.9627
3. I plan to teach purely online courses in as many occasions as possible.
0.9430
129
Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems
surements by assessing their validity, reliability, and discriminant validity, (2) the analysis of the paths of the structural model (Chin, 1998). Table 1 and Table 2 show the independent and dependant constructs’ measures and loading respectively.
Constructs Validity and Reliability The reliability and the validity are two criteria used by researchers to evaluate the applicability of their measurements to their investigated model. Reliability refers to the consistency of the measures (indicators) of a specific latent variable; whereas, validity refers to how well the concept is defined by the measures (Hair, Anderson, Tatham & Black, 1998; Crano & Brewer, 2002). With PLS, the reliability of the measurements was evaluated by internal consistency reliability, and the validity was measured by the average variance extracted (AVE), which refers to the amount of variance a latent variable captures from its indicators. AVE was developed by Fornell and Larcker (1981) to assess construct validity. The recommended level f
