Understanding Web Credibility: A Synthesis of the Research Literature (Foundations and Trends in Human-Computer Interaction)

Foundations and Trends® in Human–Computer Interaction 1:2 (2007)

Understanding Web Credibility: A Synthesis of the Research Literature Jonathan Lazar, Gabriele Meiselwitz, and Jinjuan Feng

now

the essence of knowledge

Understanding Web Credibility: A Synthesis of the Research Literature

Understanding Web Credibility: A Synthesis of the Research Literature Jonathan Lazar Towson University Towson, MD 21252 [email protected]

Gabriele Meiselwitz Towson University Towson, MD 21252 [email protected]

Jinjuan Feng Towson University Towson, MD 21252 [email protected]

Boston – Delft

R Foundations and Trends in Human–Computer Interaction

Published, sold and distributed by: now Publishers Inc. PO Box 1024 Hanover, MA 02339 USA Tel. +1-781-985-4510 www.nowpublishers.com [email protected] Outside North America: now Publishers Inc. PO Box 179 2600 AD Delft The Netherlands Tel. +31-6-51115274 The preferred citation for this publication is J. Lazar, G. Meiselwitz and J. Feng, Understanding Web Credibility: A Synthesis of the Research Literature, Foundation R in Human–Computer Interaction, vol 1, no 2, pp 139–202, 2007 and Trends ISBN: 978-1-60198-080-9 c 2007 J. Lazar, G. Meiselwitz and J. Feng  All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording or otherwise, without prior written permission of the publishers. Photocopying. In the USA: This journal is registered at the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923. Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by now Publishers Inc for users registered with the Copyright Clearance Center (CCC). The ‘services’ for users can be found on the internet at: www.copyright.com For those organizations that have been granted a photocopy license, a separate system of payment has been arranged. Authorization does not extend to other kinds of copying, such as that for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. In the rest of the world: Permission to photocopy must be obtained from the copyright owner. Please apply to now Publishers Inc., PO Box 1024, Hanover, MA 02339, USA; Tel. +1-781-871-0245; www.nowpublishers.com; [email protected] now Publishers Inc. has an exclusive license to publish this material worldwide. Permission to use this content must be obtained from the copyright license holder. Please apply to now Publishers, PO Box 179, 2600 AD Delft, The Netherlands, www.nowpublishers.com; e-mail: [email protected]

R Foundations and Trends in Human–Computer Interaction Volume 1 Issue 2, 2007 Editorial Board

Editor-in-Chief: Ben Bederson Human–Computer Interaction Lab University of Maryland 3171 A. V. Williams Bldg 20742, College Park, MD USA [email protected] Editors Gregory Abowd (Georgia Institute of Technology) Jonathan Grudin (Microsoft Research) Clayton Lewis (University of Colorado) Jakob Nielsen (Nielsen Norman Group) Don Norman (Nielsen Norman Group and Northwestern University) Dan Olsen (Brigham Young University) Gary Olson (University of Michigan) Sharon Oviatt (Oregon Health and Science University)

Editorial Scope R Foundations and Trends in Human–Computer Interaction will publish survey and tutorial articles in the following topics:

• History of the research Community

– Games

• Design and Evaluation

– Communication technologies

– Ergonomics/Human Factors

• Interdisciplinary influence

– Cognitive engineering and performance models

– The role of the social sciences in HCI

– Predictive models of interaction

– MIS and HCI

– User-centered design processes

– Graphic design

– Participatory design

– Artificial intelligence and the user interface

– Graphic design – Discount evaluation techniques – Design and interaction – Ethnography • Theory – Models of cognition – Empirical methods of evaluation – Qualitative methods of design and evaluation • Technology – Programming the graphical user interface

– Architecture and the role of the physical environment • Advanced topics and tends – Information visualization – Web design – Assistive technologies – Multimodal interaction – Perception and the user interface – Specific user groups (children, elders, etc.) – Sensor-based or tangible interaction

– Input technologies

– Ubiquitous computing

– Output technologies

– Virtual reality

• Computer supported cooperative work

– Augmented reality – Wearable computing

– History of CSCW in HCI

– Design and fashion

– Organizational issues

– Privacy and social implications

– Online communities

Information for Librarians R Foundations and Trends in Human–Computer Interaction, 2007, Volume 1, 4 issues. ISSN paper version 1551-3955. ISSN online version 1551-3963. Also available as a combined paper and online subscription.

R Foundations and Trends in Human–Computer Interaction Vol. 1, No. 2 (2007) 139–202 c 2007 J. Lazar, G. Meiselwitz and J. Feng  DOI: 10.1561/1100000007

Understanding Web Credibility: A Synthesis of the Research Literature∗ Jonathan Lazar1 , Gabriele Meiselwitz2 and Jinjuan Feng3 1

2

3

Department of Computer and Information Sciences, Universal Usability Laboratory, Towson University, Towson, MD 21252 USA, [email protected] Department of Computer and Information Sciences, Universal Usability Laboratory, Towson University, Towson, MD 21252 USA, [email protected] Department of Computer and Information Sciences, Universal Usability Laboratory, Towson University, Towson, MD 21252 USA, [email protected]

Abstract As more of our communication, commerce, and personal data goes online, credibility becomes an increasingly important issue. How do we determine if our e-commerce sites, our healthcare sites, or our online communication partners are credible? This paper examines the research literature in the area of web credibility. This review starts by examining the cognitive foundations of credibility. Other sections of the paper * Portions

of this paper were presented at the Symposium on Internet Credibility and the User, held in Seattle in April, 2005. No proceedings of the symposium were officially published.

examine not only the general credibility of web sites, but also online communication, such as e-mail, instant messaging, and online communities. Training and education, as well as future issues (such as CAPTCHAs and phishing), will be addressed. The implications for multiple populations (users, web developers, browser designers, and librarians) will be discussed.

Contents

1 Introduction

1

2 User Aspects of Credibility

5

2.1

6

Cognitive Research Background

3 Understanding Credibility in Different Application Areas 3.1 3.2 3.3

Credibility of Web Sites Credibility of One–One Communication (E-Mail and IM) Credibility of Many–Many Communication (Online Communities)

13 14 29 35

4 Education and Training

39

5 Future of Credibility

43

5.1 5.2 5.3

43 45

Phishing CAPTCHA WIKIPEDIA and Open Source Models of Information ix

49

6 Implications 6.1 6.2 6.3 6.4 6.5

Implications Implications Implications Implications Implications

53 for for for for for

Browser Designers Web Site Developers Users Librarians Researchers

53 54 55 56 56

7 Summary

59

References

61

1 Introduction

The diversity of Internet communication and content is celebrated as one of its strengths. We receive e-mails from friends, family, colleagues, and strangers. We read web pages from schools, non-profit organizations, corporations, and governmental agencies. We take part in online communities, where we share and communicate about our hobbies, our religion, our problems, and our health. When we receive all of this information, we rarely wonder about how credible this information is. However, the credibility of web-based information is a very important issue. When you receive an e-mail, how do you know that it’s actually from the person or organization listed as the sender? When you check out a web page, how do you know that the content on the web page is valid? How do you know that the organization is who they are presenting themselves to be? How do you know that the information being provided within the online community is accurate and from a recognized authority? While this might not make a difference if you are in an online community for fans of the new baseball team from Washington, DC, this will make a big difference if you are online to learn more about a rare form of cancer that you are facing. This paper will focus on the issues related to web credibility. 1

2

Introduction

While the main focus of this paper will be on the credibility of web sites, the credibility of e-mails and online communities will also be discussed. Information comes in many different forms on the Internet and Web. We get information from web pages, through e-mails, and from posted messages in online communities. While the specific interfaces for these information sources might be different, these are all forms of information. Credibility of this information is therefore an important concern. What is credibility? Credible information can be defined as believable information, information that can be relied upon as being accurate and correct [35]. Trust, a closely related concept, can be defined as the belief that a person (or information) is reliable and dependable [35]. Credibility is also closely related to concepts such as quality, authority, as well as persuasion. For instance, when making decisions, individuals may employ credibility as an additional filter to select items from a pool of information that have been judged as of being high quality [79]. How do people determine the credibility of information? There are many different judgments that are used. Some judgments are made consciously after much consideration, while other judgments are made intuitively by the user. For instance, if a user is very familiar with the subject content, he or she might be able to determine the level of credibility simply on how well the information matches up what they already know to be credible [35]. Other than that, users judge the credibility based on appearance. For instance, in the physical world, there are expectations regarding professional dress for certain professions. You would not interact with a mortgage loan officer who wears shorts and a t-shirt, as this person would not appear to be credible. In a similar vein, design features of interfaces can help project the idea of credibility (more information on this will be included in later sections of the paper). In the physical world, certifications can help project credibility. You only want to go to a board-certified doctor. You only want to have your taxes done by a certified public accountant. Web sites also have certifications that can help project credibility. Credibility of web-based information is an increasing concern. There are multiple reasons for this. The amount and scope of informa-

3 tion being delivered through the Internet and Web is expanding. Most business communication now takes place through e-mails. When political scandals break, e-mails are usually the form of documentation that are in the news. Our interaction with businesses or individuals through face-to-face contact or telephone calls is now limited. For instance, many companies are now forcing consumers to use web sites as their primary form of communication with the company, charging extra if the individual even wants to speak with a person. For example, Northwest Airlines charges extra if you want to speak to an individual and make a reservation through their call center. Southwest Airlines, and many other companies, offer special discounts available only on their web site. Some banks now charge extra if you want to use a teller. Transactions are only free if they are done over the web or at an ATM machine. Because we no longer have these face-to-face experiences, the interactions are either human–human (mediated by a computer), or human–computer. The ability to determine the credibility of these interactions is therefore paramount. This is especially true as the severity and importance of the task increases. As transactions increase from purchasing a t-shirt to purchasing a car, purchasing a home, or choosing which medical information to follow, the importance of establishing credibility also increases. Unfortunately, it is highly challenging to accurately and efficiently judge the credibility of web-based information. Due to the unique features of the web, it does not have a validated filtering mechanism to insure the quality of the information, such as the case of the peer review process in the academia field. In addition, compared to information presented in traditionally published materials, information online may not be traced back to a reliable source. The two factors combined make it quite difficult for the general public to judge the credibility of webbased information. This paper provides an overview of the topic of web credibility. First, the paper discusses the cognitive foundations of credibility. Then, user diversity (including older users, younger users, and users with impairments) will be discussed in the context of credibility. The main focus of the paper is on the topic of credibility within three different applications

4

Introduction

areas: web pages, one–one communication (such as e-mail and IM), and many–many communication (such as online communities). The impact of education and training on evaluating credibility is then discussed. Some of the future threats to credibility are addressed in the following sections. The end of the paper summarizes implications for various stakeholders. It is hoped that this paper will provide a thorough reading for individuals interested in the topic of credibility.

2 User Aspects of Credibility

Although there are some tools available to assist users in assessing credibility, much of the task of determining credibility of web content is the responsibility of the individual user. An important starting point in understanding credibility is that the skills needed to determine quality or credibility of web-based information are mainly the same as evaluation skills used in other channels of communication [64]. Several researchers have developed guidelines for web developers with the goal to improve web site credibility, but the web is a dynamic environment. Moreover, the web provides a platform for open source information models like Wikipedia, where information can be posted in a database by any user at any time (even anonymous users). There are no common standards for posting information on the web, and any information published on the web can be changed or modified at any moment, challenging users with the issue of evaluating this type of information and increasing the need for evaluation strategies [31]. At the end of the day, the final determination of credibility is still the responsibility of the user.

5

6

User Aspects of Credibility

2.1

Cognitive Research Background

Credibility evaluation of Internet-based information is a complex issue, and largely depends on the individual. It is therefore important to briefly explore cognitive research and the connection to evaluation to better understand these individual differences. Assessment of credibility of information is related to information evaluation, and information evaluation consists of several components, including critical thinking, epistemology, prior knowledge, and strategies [30]. Critical thinking is a higher order thinking skill with evaluation being one of its six major components [21]. When individuals evaluate information they typically study it for reliability, quality, credibility, and personal usefulness. Using critical thinking skills, individuals reason and make judgments. They deal with biases and develop strategies to develop clarity about issues [21]. Epistemology explains some of the individual differences in the evaluation process. Epistemology is a branch of psychology studying the nature and origin of knowledge and how it relates to concepts of truth, belief, and justification [27]. A user’s understanding of the origin of knowledge (including the identification of authority) can greatly influence his or her consideration and evaluation of credibility. Thus, identifying “true” or “false” information may vary from person to person. In general, users who assume that knowledge is unquestionable are least likely to consider new information and to criticize and override old information. However, users who assume that knowledge is fluid and changeable may be more likely to re-evaluate information [30]. Processing evaluation is further influenced by an individual’s prior knowledge and bias. Each individual conceives the external reality somewhat differently, based on unique sets of experiences with the world and the beliefs about them [54]. For example, a victim of identity theft may evaluate online banking and online shopping in a different manner than a person who never had this experience. Depending on prior knowledge, credibility may be more confidently expressed when new information confirms this prior knowledge, but it can also hinder the evaluation process when new information, for example, contradicts already acquired knowledge [30]. It should also be considered that each

2.1 Cognitive Research Background

7

person has a certain amount of bias, and that any acquired knowledge may contain errors. 2.1.1

Motivation and Context

In addition to differences highlighted by cognitive research, there are other factors influencing the process of credibility evaluation. User motivation and goals as well as context are significant elements in the credibility assessment and affect the evaluation results [31, 64]. Are users motivated to complete an information-seeking task? Is the task an important task to them? Users with high motivation usually follow the “central route” and employ a more systematic processing, whereas users with low motivation typically take the “peripheral route,” which relies more on heuristic evaluation [72]. For example, a user searching for financial information most likely will evaluate a variety of credibility factors based on checklists such as awards and seals of approval; while a user browsing the web for the latest entertainment information may have little interest in sophisticated evaluation strategies and may simply look for professional-appearing design and ease of access. Motivation and goals are also closely related to the required accuracy of the results. The more likely the result is required to be accurate, the more likely the individual is motivated to invest in the evaluation process. For example, a person researching information in a work environment is more likely to assess the credibility of the posted information than a person using the Internet recreationally. Accountability can strengthen the motivation to evaluate accurately even further. Research showed that, when motivation was increased through accountability, credibility assessment accuracy increased [84]. Environmental factors, such as the location of the workstation, social factors and user technical experience, can also have an effect on the credibility evaluation, especially when younger age groups are involved. For example, a teenage user in a public environment like a school or a library may invest less time into the assessment of credibility due to peer pressure than when obtaining and evaluating the same information using a computer at home. Users with little technical experience may feel overwhelmed using the browser’s configuration options

8

User Aspects of Credibility

to control cookies or pop-ups, and may end up with a significantly higher amount of material needing to be evaluated. 2.1.2

User Diversity

Internet users span a wide range of demographics, making web credibility research a complex task. In general, 70% of American adults currently use the Internet. A large percentage (91%) uses Internet search engines and email, and performs a variety of other tasks that involve judgment about the credibility of the web site that provided the information [73]. The largest percentage of Internet users (83%) are between 18 and 49 years old and represent the traditional, adult Internet user who is using the Internet in a school or work environment as well as for recreational use. However, there are some groups requiring special attention when investigating credibility evaluation, such as children, older users, and users with disabilities. Most of the research on credibility has been performed in the adult user community, although a large group of Internet users today are children, young adults, and a population of active seniors, also known as mature adults [30]. According to the U.S. Department of Commerce, up to 42% of children under the age of 10 are Internet users. In the age group between 10 and 17 years, up to 79% of children regularly use the Internet [69]. Children and young adults (2–17) are in a particularly difficult situation when it comes to credibility evaluation. Considering that constructing knowledge is a function of previous experiences, mental structures, and of beliefs that one uses to interpret objects and events, younger age groups have the disadvantage that they are possibly lacking domain knowledge. The knowledge construction process for domain knowledge requires time and the process of social negotiation to share multiple perspectives of reality [54]. Studies involving children and evaluation have shown that the ability to evaluate increases with age and education. Children are also inexperienced when it comes to setting and articulating goals and pursuing them, especially when multiple goals are present at the same time. Moreover, social factors are increasingly important in this group

2.1 Cognitive Research Background

9

of Internet users. Younger age groups usually trust and obey authority, whereas adolescents often are immensely influenced by their peers [30]. This age group also requires consideration of the social aspect of web usage, with more than 55% of all online adolescents (12–17) using social networking sites, such as myspace and facebook. Security awareness is prevalent in this age group; 66% of teens limit access to their profiles posted online. Almost half (48%) of the online teens visit their social network at least once a day, and often visit the sites from various locations, such as home or school [69]. Taking into account that users employ different credibility evaluation strategies depending on their motivation and context, and that younger age groups (2–17) are still building their knowledge based on experiences, this group has a unique set of requirements for evaluation strategies that should be customized to their special circumstances and usage of the web. The issue of social networking requires special attention, and the situation and interests of young Internet users emphasize the importance of considering individual differences when educating and training for credibility evaluation in this age group. A similar situation exists with the aging population. Many seniors live longer, are more active and are involved in the work environment long after reaching retirement age. Close to 45% in this group (age 50 and older) are frequent Internet users. Although Internet use is more prevalent with seniors who are still in the labor force, the segment of senior Internet users not in the labor force shows the highest growth rate of all age groups. The group of seniors not in the labor force shows an increase in Internet use of 24% since 2001 [69]. Wired seniors are the fastest-growing group of Internet users [69]. However, most of the growth in this group occurs in “aging” groups, where web users transition from work life into retirement, and carry their interest in the Internet with them [37]. There is little evidence that non-users in their 70’s and older suddenly decide to go online. Compared to younger age groups, seniors (age 65 or older) have a more limited range of online activities, most likely because they are utilizing other types of information than younger users [37]. More than 66% of wired seniors (age 65 or older) have looked for health information online. Although this represents an increase in this

10 User Aspects of Credibility age group of more than 25% since 2000, it is roughly the same number of general Internet users using online health information. Wired seniors surpass the general Internet population when researching information about Medicare and Medicaid; 19% of wired seniors compared to only 9% of users age 18–64 have visited sites connected to these issues. Surprisingly, seniors and younger Internet users both are highly interested in communication, especially email, but contrary to the younger age group many seniors decide not to participate in instant messaging (IM) technology [36]. Research has shown that older Internet users are less likely to engage in risky behavior (sharing personal information online, online dating, downloading software or music files), they often are shown to be “cautious clickers.” However, they are also less likely to take precautions or to make changes in their online behavior to avoid unwanted software or intrusions on their computer. Overall, compared to 45% of users age 18–28, only 27% of users age 51–59 have spyware installed on their computers [37]. It should also be noted that only 29% of users aged 65 or older use a computer at least on an occasional basis, although, this is likely to change as current users near age 65 enter this new age range [36]. Eight out of ten seniors in this age group think they will never go online. These numbers can directly be influenced by the younger generation. Eighty percent of younger Internet users (age 25–44) reported that they helped older relatives to use a computer and the Web [36]. Seniors are also more likely to suffer from vision problems or other disabilities. Unfortunately, users with a disability are among the populations with the lowest rates of Internet access. Studies have shown that many users aged 60 or older lose the ability to detect fine details, making it twice as difficult to navigate and use standard web sites, when compared to users between 18 and 55 years old [70]. It is hoped that, as younger and middle-aged users transition into becoming senior users of the Internet, they will continue their experienced and prudent practice, and bring with them the experience of several years working with web-based information [37]. Currently, the majority of wired older users are engaged in other types of information areas than younger users, and do not have the opportunity to build

2.1 Cognitive Research Background

11

and experience web-based information over time. Most of them are relying on family and friends to master basic web skills and have little experience with web security and credibility evaluation. It is a great challenge to determine the credibility of web-based information for users with disabilities. There are many resources on how to make a web site accessible for someone with an impairment, such as guidelines (e.g., the Web Content Accessibility Guidelines, http://www.w3.org/wai), tools (e.g., A-Prompt, InFocus, or Ramp), and even laws to encourage the development or accessible sites (Section 508 in the United States, http://www.section508.gov). Despite that, most web sites continue to be inaccessible [57, 60, 87]. An inaccessible web site is one that has not followed design guidelines, and does not have the back-end equivalents needed, so that it can be successfully used by individuals with disabilities. The issue of accessibility is closely related to credibility. In later portions of this paper, design features that highlight credibility will be presented. Many of these design features that establish credibility might not be available to someone with an impairment (e.g., someone who is visually impaired). In its place, accessibility, the ability to access content, the time spent making sites accessible, is what may lead a user with an impairment to judge credibility. For users with disabilities, the sites that are viewed as most credible are the sites that are most accessible. Sites that are inaccessible are viewed as the least credible (and since the information is not possible to ascertain, the credibility is irrelevant). Users with disabilities return again and again to the sites where they know that they will not experience problems accessing the information. These sites, in turn, become the sites that are viewed as credible. Similar to how graphical design influences the perception of credibility for visual users, accessibility influences the perception of credibility for users with impairments, although neither of these are necessarily the best ways to ascertain the credibility of web-based information.

3 Understanding Credibility in Different Application Areas

There are fundamental differences between the credibility of a web site as apposed to the credibility of a person in computer mediated communication. Credibility of a website is normally established through factors showcasing the physical nature of the organization, such as buildings or employees, as well as specific design features [33]. However, credibility of a communication partner is highly related to the trust toward a specific individual and tends to be affected by the type of communication and the relationship among the parties involved in the physical world. For instance, Bos et al. found that individuals who communicated via video and audio developed a more trusting relationship as compared to individuals who communicated via text chat [7]. Face to face meetings prior communicating online also helps to promote trust between communication partners [92]. For the purposes of this paper, interpersonal online communication will be divided into two categories: one–one (e-mail and IM), and many–many (online communities). One–one communication is when individuals or organizations are sending an e-mail or instant message to another individual or small group. The communication is personalized. It is only meant for specific recipients. Many–many communication is 13

14 Understanding Credibility in Different Application Areas when any number of individuals or organizations are both the source and the recipient of information. Online communities, which include tools such as listservers, chat rooms, blogs, and bulletin boards, are the most common form of many–many communication [74]. While this is a simple classification scheme for online communication information, it can help illuminate some of the differences in how people perceive the credibility of information, and how information can be presented in a way that is more credible. The following major sections of this paper will cover the credibility of web sites (Section 3.1), the credibility of one–one communication (Section 3.2), and the credibility of many–many communication (Section 3.3).

3.1

Credibility of Web Sites

There are literally billions of web pages. The problem is not a limited amount of information, but rather, too much information. Sifting through what’s on the web can be a challenge. Which information is credible, and which information is not? There are many implications: health information on the web can come from multiple sources. Which sources do we trust? Businesses want us to enter our data (for reservations, purchasing products, etc.) on web sites, rather than in person or over the phone. Can we reliably enter our data? Governmental agencies want citizens to pay taxes and register for government permits online. Are these the actual sites, or frauds, representing themselves as governmental agencies? If we know that information is credible, there are many benefits to all of these interactions. But how do we know if web sites are credible? The next sections will discuss the credibility of information on the web. There are really two separate, but related topics: (1) how do sites make themselves appear to be credible? and (2) how do users sort through and determine which sites are credible? 3.1.1

Design Features for Enhancing Web Site Credibility

Fogg describes four types of web credibility: presumed credibility, surface credibility, earned credibility, and reputed credibility [31]. Interestingly, three of the four types of web credibility are largely influenced by

3.1 Credibility of Web Sites

15

a user’s individual judgment, background and experiences. Presumed credibility, surface credibility, and earned credibility are all very much based on an individual user’s perception. Presumed credibility is based on general assumptions in the user’s mind; it relies on factors such as domain identifiers like .gov and .org, and how much competitor information is provided on the site. Surface credibility is based on a simple, first impression of the user, and is often influenced by professional design, ease of access, and the amount of advertising included on the site. Earned credibility is established over time by a site that consistently has proved to be accurate over a longer time period, for example, a time frame of a year or longer. Earned credibility is about building a long-term relationship with a user, and is mainly based on a user’s evaluation of how easy it is to interact with a site, the ability to provide personalized information, and how responsive the service is [31]. The one type of web credibility least influenced by an individual’s judgment is reputed web credibility, established by any awards the site may have won, seals of approval, or links from a credible source. Reputed credibility also connects online credibility with offline experiences. Sites that have been referred by a friend or colleague are perceived as more credible, and many sites offer easy options to forward articles or to invite friends to join [31]. There are a number of different approaches to making a web site appear credible. Good interface design may lead to an appearance of credibility. However, while users may primarily evaluate credibility based on the graphical appeal of a web site, in reality, users evaluate the credibility of information based on multiple dimensions [31]. However, it’s important to note that design features do not automatically mean that a site is authentic and credible, rather that it has the APPEARANCE of credibility. There are many design features that help support the perception of credibility. Fogg et al. have found that five design features help increase the perceived credibility of a web site: (1) conveying the real world aspect of the web site sponsor, (2) making sure that the web site is easy to use, (3) including markers of expertise, (4) including markers of trustworthiness, and (5) providing a personalized user experience [33].

16 Understanding Credibility in Different Application Areas Two design features seem to detract from the perception of credibility: overly commercial elements, and design features that make the site appear amateurish [33]. In a follow-up study, Fogg et al. determined that the design look of the site has the greatest impact on credibility [34]. The “design look,” in that study, included topics such as visual layout, white space, and use of appropriate color schemes [34]. In addition, banner ads tend to decrease the perception of credibility, while information on the author of the content (both a biography and/or pictures) can help increase the perception of credibility [32]. Testimonials from well-known sources can also help increase the perception of credibility [14]. While not a part of the design look, an important interface issue related to credibility is the response time. A number of research studies have examined response time, that is, how long it takes a web site to respond to a user request. The findings generally state that the shorter the response time, the higher the perceived credibility of the site. An increased response time changes the user’s perception of the quality of the content [52], and whether the content is interesting [78]. In fact, a long response time can change the user’s perception of the organization that sponsors the web site, and even impact on the perception that a transaction is secure [8]. While the user’s local connection speed is outside of the control of the site designer, the designer can attempt to minimize response time by keeping web page file sizes low [56]. Users are likely to feel that the information provided by an organization is more credible if the organization providing information in cyberspace is an organization that the user is familiar within the physical world [59]. This is obviously something that develops over time. Experience with a physical organization is developed over a number of years, and cannot be added overnight with a software application or a development tool. For instance, many users trust the web sites of newspapers, when they are familiar with those newspapers in print over a long period of time [65]. A newspaper’s reputation and credibility does not just develop overnight, but in other cases, it can be helpful if a web site can be designed so that it highlights the long-term history of the physical organization. Similarly, the web sites of neutral

3.1 Credibility of Web Sites

17

government organizations, such as the National Institutes of Health, are perceived as having more credibility than web sites for hospitals such as Johns Hopkins Hospital, which may be more concerned with their rankings than accuracy, and Intelihealth.com, which is owned by Aetna, a for-profit HMO [61]. Another way to make a site appear credible is to use some sort of certification seal [65]. For instance, a number of e-commerce sites use certification seals to convince users that their computer networks are secure [24]. These certification seals include Verisign and TrustE. The Better Business Bureau offers a certification seal, known as BBBOnline (http://www.bbbonline.org/) that certifies that the company sponsoring the web site follows the BBB Code of Ethics. The BBB also offers a separate certification seal related to the handling of personal information. Sometimes, credit card seals (such as those from Visa or Mastercard) can also help create the appearance of credibility [24]. Users using these sites can be sure that the site is secure and that their information will be protected. A number of certification seals exist for determining the credibility of health information on the web. These programs, including URAC (http://www.urac.org/) and Hi-Ethics (http://www.hiethics.com/), help establish the credibility of health related web sites [61]. Other types of certification seals can relate to site accessibility. Some sites use the “BOBBY-approved” symbol (see http://bobby.watchfire.com). This symbol informs users that the site has passed Bobby certification for accessibility (although the tool is no longer called “Bobby”). Other sites may state that they are accessible, as certified by the National Federation of the Blind certification program (see http://www.nfb.org/seal/intro.htm). Another way to promote credibility is to include a statement of some type, detailing a policy of the web site. For instance, the most popular type of policy is a privacy policy, where the site will clearly state how information collected on the site is being used [66]. Another type of policy is the universal usability policy, which states how usable the site is for diverse user populations (such as users using assistive technology, old browsers, or slow connection speeds) [49].

18 Understanding Credibility in Different Application Areas 3.1.2

User Evaluation of Site Credibility

There are a number of ways that users evaluate credibility of the information that appears on web sites. Users tend to evaluate credibility by using the design appeal of the web site [34]. This is not necessarily the best methodology for evaluating credibility; however, it is one of the most frequent approaches used. Certification symbols, such as those discussed in previous sections, can help users establish credibility, however, in reality, this only transfers the need to determine credibility from the web site to the sponsor of the certification symbol. Users must determine if they feel that the certification symbol itself is credible. If the user has long-term experience with the sponsor of the web site, this might encourage the user to believe that the information provided on the web site is credible. The presence of privacy policies also appears to influence the perceptions of web site credibility. Although it was reported that only 54% of visitors read a site’s privacy policy when they first visit the site [26], users may perceive a higher level of credibility if a web site has a privacy policy [33]. Non-design related aspects of credibility that can be evaluated by users included accuracy, bias, and completeness of information [65]. Finally, users may feel that credibility is in question if information presented is either out-of-date or has misspellings. Health information is probably the most important and serious type of web information that users look for. The consequences for using noncredible information are the highest when looking for health information [61]. It is therefore disturbing to note that, in a 2002 study from the Pew Internet and American Life Project, only 25% of those who acquire health information online always check to determine the credibility of the information source [39]. This can be problematic, especially considering that one study found that less than 50% of the health-related information available on the web has been reviewed by medical doctors for accuracy [38]. When users actually do check to ascertain the credibility of information, they check the source, date, and privacy policy of a web site. Another study found that, as a health crisis increased in severity, users were actually more likely to evaluate health information to determine the credibility of that information [83]. Overly commercial

3.1 Credibility of Web Sites

19

elements of a web site tend to “turn off” the users and make them question the credibility of the health information. These users typically find the health information using a search engine, which, again, can be problematic (see later section for more info on search engines). 3.1.3

Tools to Assess the Credibility of Web Pages

There are a number of tools that can be used to help users assess the credibility of information on the web. One way for users to evaluate the credibility is to use a list of questions developed by content experts [40]. When users access a web site that they are not previously familiar with, they can use these series of questions, relating to topics such as accuracy and currency of information. Examples of credibility evaluation question lists are available at: http://www2.widener.edu/WolfgramMemorial-Library/webevaluation/webeval.htm and http://www.lib. purdue.edu/ugrl/inst/evaluationchart.pdf Of course, no set of questions would fit the needs of every web site, and the responsibility for determining the credibility falls on the user. Some more focused question lists have been developed for specific topics, such as the DISCERN questionnaire (http://www.discern.org.uk/), which focuses specifically on health information for consumers [40]. In addition, a questionnaire can be presented while the user is actually utilizing a site. A tool called the Information Quality tool (http://hitiweb.mitretek.org/iq/) is essentially a pop-up window that asks questions of the user, related to the credibility of the site, while the user is utilizing that site. After users have responded to all of the questions, giving details about the web site, a score is calculated by the IQ tool, rating the credibility of the web site examined [40]. Limited credibility evaluation tools are now being included in new versions of both security tools and web browsers. The placement of these tools, directly in mainline user applications, will most likely mean that they will be effective in getting user attention and actually being utilized. However, because it relieves the user from the responsibility for determining the credibility, the long-term effectiveness is questionable. Norton Internet Security 2007 installs a small add-on, located at

20 Understanding Credibility in Different Application Areas

Fig. 3.1 An add-on tool from Norton Internet Security that evaluates sites for known credibility, security, and fraud problems.

the top of the browser window, which provides information on whether a site is potentially fraudulent. This feature, known as “Phishing Protection” (see Figure 3.1) checks to see if this is a page that is either known to be non-credible or has features that make it non-credible, possibly unsafe, or insecure. If a site is deemed to be fraudulent, a fullscreen warning appears (see Figure 3.2). However, there is still a user responsibility. Certainly, this tool is imperfect, and users are asked to contact Symantec (the company that sells the Norton products) if they find that any web sites are incorrectly categorized. Microsoft Internet Explorer version 7 offers a similar type of tool to notify users if a site is a well-known phishing web site (see Figure 3.3). These tools might be able to check a site that is known to be fraudulent, or has insecure features, however, the effectiveness of parsing text (say, on healthcare suggestions) to determine if the text comes from a credible source would be limited or non-existent.

3.1 Credibility of Web Sites

21

Fig. 3.2 A warning from Norton Internet Security that a page may be fraudulent.

The limitation here is that all of these methods for determining credibility are based on information provided by either the site sponsor, or based on the previous experience or knowledge of the user. What if such information is lacking, or the user does not have the necessary experience or knowledge? One solution is for users to utilize a pre-screened list of sites. A series of sites might be pre-screened by librarians or other content experts to determine the credibility of the information [40]. One example of such a site is the Librarian’s Index to the Internet (http://www.lii.org). The limitation here is that this is only feasible if the type of content that the user is interested in has been included in the pre-approved list. While the LII is a very general resource list, more specific lists exist, where sites have been examined and pre-approved. For instance, http://www.ds-health.com/ is a web site run by Dr. Len Leshin. It provides articles on Down Syndrome and links to other sites which

22 Understanding Credibility in Different Application Areas

Fig. 3.3 The anti-phishing features on Microsoft Internet Explorer version 7.

have been pre-screened by Dr. Leshin. Since this is not a site like the National Institutes of Health, or the New York Times, which would already have a certain level of assumed credibility, this site (see Figure 3.4) provides a number of features to help establish credibility. For instance, it offers a certification seal. It notes the professional credentials of the site sponsor. It also provides a biography of the site sponsor (noting that he is both a pediatrician and a parent of a child with Down Syndrome). Reputation systems (also known as recommender systems) are another way that users, based on the experience and knowledge of others, can evaluate the credibility of information, products, services, or individuals on a web site. Reputation systems are areas of a web site where users can post their perceptions and experiences with information, products, individuals, or organizations [16, 53, 74]. The concepts of credibility and trust are increasingly being added to reputation systems

3.1 Credibility of Web Sites

23

Fig. 3.4 DS-Health, a site with a number of features to establish credibility.

[71]. The idea is that if other users provide their feedback, it can help establish the credibility of information, or let the user know that certain information or a certain individual is not credible. This information can be as simple as a numerical rating, or can be complex, rating multiple aspects of credibility or providing lengthy feedback. This type of reputation manager has been applied most often in e-commerce companies. This is especially important when there are various parties involved in a transaction (such as those on Ebay), where the parties will never meet, and the intentions and credibility of the strangers must be established [82]. An example of Ebay’s reputation manager is in Figure 3.5. However, this approachs can also be applied to establishing the credibility of information on a web site. For instance, users can comment on how credible information is on a health web site. In 2004, the Pew Internet and American Life Project found that 26% of adult Internet users in the United States had used a reputation system to rate a product, service, or person [76].

24 Understanding Credibility in Different Application Areas

Fig. 3.5 The reputation manager from EBay.

The reputation systems that collect inputs from multiple parties, although being used increasingly and proved highly useful, have limitations that need to be addressed. One major concern with reputation systems is the credibility of the feedback or user evaluations. It is possible that someone can fabricate either positive or negative feedback toward a web site or a seller to fraudulently influence the reputation of the site or the seller. Since the user evaluation is provided anonymously, it is very difficult, if at all possible, to trace individual feedback and evaluate the validity of the feedback. The potential damage of anonymous online posts on personal reputation is highlighted by a recent case in which two Yale law school students were forced to file a lawsuit against a law school discussion board. The two students claimed that a series of anonymous offensive postings on the discussion board damaged their reputations [55]. Before an effective solution is implemented to track the validity of the inputs of online reputation systems, the users of those systems should be warned of the possible fraudulent postings and suggested to use the information with caution.

3.1 Credibility of Web Sites

3.1.4

25

How Do Search Engines Impact Credibility?

As previously discussed, users often do not extensively evaluate the credibility of the information that they find on the web. This is especially true with the information that users find from search engines. Users perceive that the information that they receive from the results of a query on a search engine is non-biased and accurate. However, search engine companies are for-profit companies that make money by presenting results in a specific order. However, users do not perceive this bias. For instance, a Pew Internet and American Life Study found that only 38% of users actually are aware that there is a difference between search results that are sponsored (and paid for) and those that are not sponsored [28]. Among those 38% who are aware of the differences, only half can always tell which search results are sponsored and which are not. Figure 3.6 shows search results on Google, with two different areas of sponsored links (one with blue background, the

Fig. 3.6 Search engine results showing sponsored links.

26 Understanding Credibility in Different Application Areas other with a normal background). Small gray text notes that these are sponsored links, but not all users will notice this. Despite the existence of sponsored results on search engines, 64%–73% of users (depending on frequency of usage and experience) feel that search engines are a fair and unbiased source of information [28]. Aside from the difference between sponsored and unsponsored links, there are also techniques that are used by webmasters to “trick” the search engine into ranking a specific site higher than others [62]. By adding certain keywords in certain phrases, sometimes, the web site can appear higher in a list of responses to a query. In addition, if a search engine bases some of the ranking on which links users had followed in the past, or which links were most popular, there are software tools that web developers can use to somehow “trick” the search engine into a higher ranking for that site based on click popularity. Therefore, the results of a search engine are, at all times, questionable. 3.1.5

Credibility and E-Commerce

E-commerce, online banking, and online investing are integral components in the lives of most Internet users. Over 90% of all Internet users search for information on the Internet, over 70% use the web to buy a product or service, and 43% conduct their banking and investment activities online [73]. Credibility and trust are closely related and become increasingly important in e-commerce and e-finance. Fogg defines trustworthiness and expertise as the two dimensions that make up credibility [31]. Trustworthiness is represented by sites that the user perceives to be unbiased, fair and honest. Sites can increase perception of trust by users, for example, by giving a physical address and contact information, by limiting advertising, and by providing articles with citations and references [31]. Perceptions of expertise levels can be increased by providing quick responses to customer service questions, sending email confirmations, and/or by having sites that look professionally designed [31]. Users are more likely to use a site that has earned their trust. Trust mechanisms assist users in dealing with risks in specific types of relationships and determine the formation of specific relationships that are

3.1 Credibility of Web Sites

27

in turn influenced by specific types of trust mechanisms [22]. For example, experienced users have a higher level of trust in the sites that they have established consumer relations with. In addition to the level to which a user has an established relation to a web site, user expertise also influences trust. Many Internet users understand the dangers of credit card fraud and identity theft; roughly 80% of all Internet users are somewhat concerned that someone could steal and misuse their personal information. These high levels of concern have caused some behavior changes in online consumers in recent years. Many e-commerce Internet users have stopped giving out personal information and are more likely to read a site’s published policies and user agreements before they decide on any purchasing activity [18]. However, increasing trust and credibility of a site does not necessarily make the site more secure. Producing systems that are both secure and usable has been a challenge that many researchers have investigated [6, 20, 81]. Secure systems must be usable to accomplish the goal; they must consider human factors and policies, as well as security mechanisms [20]. A common problem designing secure systems is that as system security increases, system complexity also increases. Unfortunately, this often causes systems to be less secure — errors become more probable, configurations may be mismanaged, and users may try to circumvent security mechanisms because they are too difficult to use. Therefore, usability must be an important goal in designing security systems [6]. Acknowledging that security does not happen in a vacuum, considering the context in which security mechanisms are being used, and making reasonable demands on users can assist in making security work as it is intended [6, 81]. Education and training can further increase the dissemination and integration of security in organizations, enhancing the understanding and necessity for users to actually use (and to use correctly) the security mechanisms provided in their systems [81]. In addition to security, privacy is also a key concept in e-commerce and electronic financial transactions. Privacy can be understood as an individual’s capability to control their personal information in particular social situations. This capability highlights the fact that privacy is individually subjective and socially situated. Similarly to security,

28 Understanding Credibility in Different Application Areas privacy also includes risks and must consider the control of the individual user to manage his or her privacy. Again, context can change the perception of user privacy dramatically. Users in e-commerce and online banking may have completely different expectations regarding privacy than users visiting a social network site [1]. Often it is also a difficult situation for users to determine what is necessary information and what should be a protected private information. For example, most users realize that it is necessary to provide certain personal information to an e-commerce site. However, is it absolutely necessary to answer truthfully regarding details about mother’s maiden names and schools we have attended as a child? Privacy again highlights the largely situated nature of secure and usable systems, contextual and individual differences should be considered when investigating trust and credibility in e-commerce and e-finance [1]. These contextual and social differences are also evident when extending the evaluation of trust and credibility in e-commerce across cultural domains. A cross-cultural study, investigating the suitability of e-commerce systems for other domains than the one for which they were originally created, discovered that there were distinctly different perceptions and concerns about privacy and security. For example, 63% of United Kingdom users were concerned about security compared to only 9% of users in China [86]. Considering these observations regarding context, individual differences, and cross-domain development, DiGioia and Dourish suggest social navigation as a model for usable security [23]. Since precise requirements for security and evaluation of what is secure and insecure are mostly made by the end user in any given moment, this model suggests a more complex incorporation of the user into the evaluation. Implementing interactive design and making actions of other users available can assist individuals in the evaluation of security and privacy. Showing the history of user actions, uncovering usual and unusual patterns, and activities of others within a system can be used to increasingly integrate the social and situational context, and can support the inclusion of individual differences where trust, credibility, security and privacy are concerned [23].

3.2 Credibility of One–One Communication (E-Mail and IM)

3.2

29

Credibility of One–One Communication (E-Mail and IM)

E-mail is the most personal form of web-based communication. People accept an e-mail as your word. For many, e-mail is their primary form of business communication. Organizations communicate with their clients or members using e-mail. Family members keep in touch using e-mail. One study from 1985 says that e-mail increases the number of people whom you regularly communicate with by a factor of 10 [48]. Instant messaging (IM) has also become important, as a shorter, quicker way of communicating with others. While e-mails are asynchronous, IM is synchronous. The obvious downside of IM is that both users must be logged in and at their computer at the same time. In addition, communication partners must be using the same IM application. While there are a number of differences between e-mail and IM [68], they are outside of the scope of this paper. Simply put, some people rely on e-mail and IM as their main form of communication. However, the credibility of e-mail can sometimes be unclear. You sometimes receive e-mails from names that you know, but the e-mail somehow does not seem to be from the sender. In the physical world, people rely on face-to-face interactions, or interactions over the phone, to determine the credibility of their communication partner. There are many aspects of communication that are considered when determining face-to-face credibility, such as vocal intonations, facial expressions, body language, and proxemics. On the phone, we can still rely on vocal intonation, emotion in the voice, loudness, or whispering in the voice. If you say, “She’s a good friend,” depending on the vocal intonation or hints of sarcasm in the voice, there can be different intended messages. All of these aspects of communication (vocal intonations, facial expressions, proxemics) can help support the development of trust between communication partners [29]. And all of these aspects of communication are more effective than plain text, in developing trust among communication partners [7]. However, none of these aspects are present in e-mails or IM. Despite this fact, research has found that individuals tend to be more honest in e-mails than in phone calls or IM [47, 88]. However, if you do not know the sender of

30 Understanding Credibility in Different Application Areas the e-mail, how can you assess the credibility of what they are saying? How do you decide if you want to open and read the e-mail? Does the subject of the e-mail play a role? The next section discusses how users determine the credibility of incoming e-mail messages. 3.2.1

Determining the Validity of Senders

It is certainly important to determine the credibility of the sender of an e-mail, when it arrives in your inbox. Most e-mails are actually from the sender listed. However, there are a number of situations which could cause false sender information. For instance, if a virus takes over your e-mail program, it could send e-mails, replicating the virus, to all those in your address book. These e-mails are sent from the e-mail program of the sender listed, although the sender did not write those e-mails. Another example is when an individual is able to hack an e-mail account and essentially perform the same function as a virus. The e-mail is coming from the e-mail program of the sender listed, but in reality, the sender did not write the e-mail. If the recipient of the e-mail knows the sender personally, they will usually be able to tell if the e-mail is valid, due to the style and content of the message [65]. When unsure, a follow-up phone call can confirm the validity of the message. If the recipient of the e-mail message does not know the sender, it can be harder to determine the validity of the message. Another topic related to the credibility of e-mails is spam. Spam is an issue that regularly makes the headlines. According to the Messaging Anti-Abuse Working Group, spam has increased from approximately 10% of overall mail volume in 1998 to as much as 80% in the year 2006 [63]. Unwanted e-mails, encouraging you to refinance your mortgage, purchase medicine online, make parts of your body bigger, and recover $35 million from your relative in Nigeria, fill e-mail inboxes on a regular basis [75]. These are clearly non-credible e-mails, and users can determine this simply by the e-mail subject line. Frequently, the sender’s e-mail address also gives away the fact that an e-mail is not credible. However, what about e-mails that could possibly be credible, and could be spam? For instance, one commonly sent e-mail appears to be from Washington Mutual Bank, noting that there have been prob-

3.2 Credibility of One–One Communication (E-Mail and IM)

31

lems with the user’s bank account. If the user does not have an account at Washington Mutual, this is an easy fraud to identify. However, it is much harder for the Washington Mutual customer to determine the credibility of this message. How should the user act in a situation like this, where it is unclear whether the e-mail is credible or not? The consumer can actually call the individual or organization involved and identify the credibility of the e-mail. But with spam emails filling mailboxes on a daily basis, those phone calls will become painstaking and time-consuming. Effective e-mail filtering mechanism is urgently need to fight these types of fraudulent e-mails and prevent them from even reaching the user’s mailbox. 3.2.2

Tools for Assessing the Credibility of E-mail

A popular way to address e-mail credibility (especially spam) that is still under rapid development is e-mail-filtering (either organizational or personal). A group of e-mail-filtering techniques depend heavily on user inputs. For example, a Whitelist filter requires users to create a list of approved addresses from which they accept messages. Filters such as Whitelists provide a strong level of spam protection, blocking nearly 100% of unwanted e-mails. But they also raise major problems including increased work load of the users and inevitably high false positive rate (valid e-mails identified as spam e-mails and filtered to the folder with lower priorities) [75]. The second group of spam filtering solutions was built on top of various machine-learning techniques ranging from the very basic Na¨ıve Bayes algorithm [80] to the more advanced logistic regression, vector machine, or compression-based approaches [46]. Using the machine learning-based technique, the filter needs to be trained by a large number of e-mails, both valid and spam, to learn how to differentiate spam from valid mails. For example, early machine learning-based filters search for words such as “free” and “sex” that frequently appear in spam mails and analyze the frequency and location of those terms to make the judgment. The approach is called a static-filtering technique if the list of terms or features that were searched for is unchanged. However, the spammers quickly learned to adapt to the spam-filtering

32 Understanding Credibility in Different Application Areas techniques and successfully avoided or altered the use of those key terms or features in spam mails. The adaptation forced the spam-filters to frequently update the key terms and feathers based on which they make the judgments. It seems that the constant battle between the spam filters and spammers will be going on for a long time. Besides terms and presentation features, some recent machinelearning algorithms could examine more subtle features such as writing style. For instance, if e-mails regularly come from a friend, there will be a documented history of the friend’s writing style. An e-mail program could store general information about the writing style, and note to users when the writing style from that individual changes drastically. This could signify to the user that an e-mail was automatically generated by a virus, or was written by a hacker. With notification, it would be up to the user to examine the message more carefully to determine credibility. More recently, Stolfo et al. extended the traditional domain of email analysis by modeling user behavior [85]. An empirical study based on e-mails from 15 users suggested that the tool could capture 99% of viral e-mails with 0.38 false positive rate (38% of e-mails with attachments were mislabeled as viral). Social network analysis has emerged as a highly promising technique for email filtering in the past few years [43]. A social network is a social structure made of nodes that are tied by one or more specific types of relations. Social network analysis views social relationships in terms of nodes and ties. The nodes in a social network can be individuals or organizations. The ties are the relationship between the nodes, such as friendship, ratings, financial exchange, etc. A number of studies have demonstrated the success of applying social network to e-mail filtering. Among them, Golbeck and Hendler proposed an e-mail scoring prototype called TrustMail based on reputation ratings computed through social network technology [44]. Boykin and Roychowdhury developed a social network-based application that was able to classify approximately 50% of a user’s e-mail into valid or spam categories [9]. Other approaches have also been proposed to fight spam mails, typical examples including IP address-based and security identity-based spam detection [19]. All those techniques aim at preventing the spam e-mails from entering users’ e-mail box (or the folder with high prior-

3.2 Credibility of One–One Communication (E-Mail and IM)

33

ity in the users’ mail box) and considerable success has been achieved. But it should be noted that no spam-filtering techniques are capable of achieving 100% accuracy in the classification of spam and valid mails. All spam filters are subject to a trade-off between false alarms (valid mails identified as spam) and missed hits (spam identified as valid e-mails). In order to capture a higher percentage of spam mails, an increased number of valid e-mails will be classified as spam, and vice versa. This is why the majority of the filtering applications only provide recommendations as to the likelihood of an incoming mail being spam and let the users make the final judgment. Even though spam filtering software significantly lower the users’ work load on cleaning up e-mails, they cannot completely replace human work. Therefore, it is crucial to develop techniques that would help a human user effectively judge the credibility of an e-mail. A number of new features could be added to e-mail programs to help users judge the credibility of incoming e-mails. One feature helps the users trace the real source of the e-mail. Consider the e-mail message in Figure 3.7. It asks the user to verify their account information on ebay.com. The credibility is already questionable. However, there is one hint that this e-mail is not credible and is very likely spam. The e-mail address that the message was sent from indicates ebay.com. Yet, the link that the user is requested to click on (where it says “click here to verify your account”) is actually directing the user to a non-ebay domain in Germany. A user would only note this if they closely examine the link (by placing their mouse on top of the link to determine the target URL). This is a hint that this is spam. A feature could be added to an e-mail program that could check e-mails, such as this one, that come from unknown individuals, and look to see if the stated e-mail address matches the domain that is listed as a link. This type of spam is also known as phishing. There is a thorough description of some of the challenges of phishing in Section 5.1 In addition, digital signatures might be used to help establish that the sender of an e-mail is really who they say that they are. The concept of digital signatures has been well accepted in e-commerce. According to a survey on 470 Amazon.com merchants in 2005, approximately 60% of merchants thought that receipt sent by online merchants should be

34 Understanding Credibility in Different Application Areas

Fig. 3.7 An e-mail message with questionable credibility.

digitally-signed [42]. Some security enabling approaches such as encryption may also prove useful in making e-mails more credible. But when adopting those techniques, the developers should be aware of the possible negative impact on the usability of the applications and ensure that the e-mail systems with additional security features are still easy to use [41]. 3.2.3

Users with Disabilities and Spam

The validity of e-mail is an especially great problem for users with disabilities. For users with disabilities, spam and non-credible e-mail poses a greater problem than for typical users. Whereas typical users without disabilities can just see an e-mail in preview format, determine that it’s spam, and delete it, this is not possible for a user with a disability. For instance, a blind user might need to hear a good part of the e-mail (using a screen reader) to determine if it is credible or not.

3.3 Credibility of Many–Many Communication (Online Communities)

35

The cost of dealing with spam, in terms of time, is very high. Other disability populations face similar challenges, and spam just makes their interactions with the computer that much harder. To further complicate the situation, existing anti-spam solutions such as the Whitelist filters do not work effectively for users with disabilities, particularly blind users. To have a high level of spam protection will save time, but important e-mails may be deleted or overlooked. To have a lower level of spam protection, more e-mails may make it through, but time will be wasted dealing with the spam. For the majority of blind users who depend on screen access software to access e-mails, the time required to retrieve false alarms (valid e-mails filtered out as spam) or identify valid e-mails among spam mails is significant longer than that of traditional users without impairments. Currently, this domain of research is severely understudied. Users with disabilities are in urgent need of effective solutions that could provide both desirable security and high usability.

3.3

Credibility of Many–Many Communication (Online Communities)

Online communities consist of multiple individuals interested in a specific topic, sharing resources, and communicating using group computer-mediated communication tools [74]. These tools include listservers, chat rooms, bulletin boards, and blogs. Online communities vary on the amount of face-to-face contact among community members [58]. The amount of face-to-face contact may impact on the credibility of information. Some communities, such as those based on physical communities such as the Blacksburg Electronic Village or the Seattle Community Network, often have frequent face-to-face contact among community members. Some communities are based on periodic face-to-face contact, such as those related to religious communities, professional communities, and sports communities. Community members may meet face-to-face at annual events. Other communities, such as those related to healthcare, emotional support, or role playing, tend to have no face-to-face contact [58]. The amount of face-to-face contact can impact on credibility, since with frequent face-

36 Understanding Credibility in Different Application Areas to-face contact, users might be familiar with other community members and be able to judge the credibility of the information that is posted by those specific members. The major factors in an online community that can help users determine the credibility of information are registration, moderation, identification of expertise, and reputation feedback. These types of approaches fall under the umbrella of what is known as “sociability” [74]. Sociability, unlike usability, focuses on what is needed to help ensure that an online community is successful as a social place. Sociability details include moderation, registration rules, rules on posting, rules on flaming, and rules on lurking. 3.3.1

Tools for Determining the Credibility of Information in Online Communities

Registration can be the first tool used to assess credibility. In some online communities, registration is only allowed to people with certain credentials. For instance, in one online community for anesthesiologists, only credentialed, certified anesthesiologists were allowed to register, post messages, or even read posts [74]. Membership in the community itself established credibility. In another online community for folk music, an e-mail message, establishing the user’s identity, as well as reasons for wanting to join, were required before posting or reading was allowed. The amount of control on membership, and the thresholds required for registering, tend to lead to establishing a certain level of assumed credibility about communication within the online community. If anyone can join, and if anyone can join anonymously, there is a low level of assumed credibility on communication within the community. However, if membership in the community has been tightly controlled, and if all new members to the community have been evaluated and vetted, and if users are AWARE of this vetting process, this can lead to a high level of perceived credibility on communication within the community. Another tool that can influence credibility in an online community is strong moderation. If a communication tool (such as a listserver, newsgroup, or bulletin board) is strongly moderated, it means that an expert in the topic of study must approve all posts. This role could be

3.3 Credibility of Many–Many Communication (Online Communities)

37

considered similar to someone like a librarian who is a content expert, and reviews and approves content, to ensure the credibility of information. Moderators are especially important in online communities related to health care information, as they can help separate informed, credible advice, from other advice that is simply someone’s opinion. One step further to this approach is to make clear the source of specific information within the online community. On a bulletin board related to athlete’s foot, for instance, there should be a clear distinction between the informed advice of a podiatrist, the experiences of someone suffering with athlete’s foot, and a representative of a pharmaceutical company that sells a cream to treat athlete’s foot. Even noting, as some bulletin boards on http://www.webmd.com do, which information is coming from a licensed source, and which information is not coming from a licensed source, could be helpful. Figure 3.8 displays a screenshot from WebMD, where it is noted in the screen name that

Fig. 3.8 A screenshot from WebMD, where a message post is identified as coming from a WebMD staff member.

38 Understanding Credibility in Different Application Areas the person posting is a WebMD staff member. While this provides more credibility-related information to users, this is a trade-off, because, as in Section 3.1.1 of the paper, users have more information on credibility, but have shifted part of the burden for determining credibility to an outside source, and the source itself must be evaluated for credibility. When users let outside sources do the credibility evaluation for them, it might feel like the burden of credibility evaluation has been lifted from the user, in reality, users must still evaluate, only now, they are evaluating the third-party source for credibility, rather than the information itself. Another example is an online community/digital library known as CalFlora, which helps track plant growth and distribution in California [89]. Data can be added by any individual across the state, who can record their reports of where various plants and fauna have been found. The challenge is to determine the skill level of those reporting the plant growth, as many plants look similar. To address this issue of credibility, the CalFlora asks users, when they register, to report their bio credentials, their institutional affiliation, specific expertise, and experience level [89]. The idea is that based on that data, other users and scientists can make judgments about the credibility of the information, based on the skill level of the individual that added the information. Reputation systems are also useful to help establish what information is credible within an online community. Reputation systems can help establish what information other users found credible or useful. More information on reputation systems is available in previous sections. It should be noted that the nature of most online communities are to be places of free and open speech. If community members want to keep it that way, then it should be clear from the start that the credibility of information within the community (especially in a health-related community) is questionable. As long as it is clear that the credibility of information is questionable, users can make informed decisions on the use of that information.

4 Education and Training

While many tools and approaches exist for users to evaluate credibility of web sites and online communication, they will only do so if they understand that credibility is an important issue. Therefore, education and training on this topic is essential. While younger age groups are more comfortable than older age groups when it comes to using technology, they are also less able to detect misinformation, understand underlying commercial messages, or evaluate the strength of an argument [30]. Older internet users (60 years and older) are much less adventurous than the younger generation when using the internet and can be considered “cautious clickers.” Unfortunately they are also much less likely to make changes in their online behavior to avoid unwanted software on their computer [37]. As much as individuals differ in their cognitive approach, background, and experience, teaching about information evaluation must consider these individual differences [11]. Most educational professionals now favor a constructivist approach in education, which focuses on what a person knows and the interpretation and construction of individual knowledge representation, and the promotion of higher order thinking. In a constructivist learning environment knowledge acquisition is situated, being in part a product of the activity, context, and culture in which it is developed and used [10]. 39

40 Education and Training To make training and education on credibility evaluation effective, and to build this situated knowledge, credibility evaluation should become embedded into the curriculum and must become part of the educational strategy of institutions [21]. One of the largest efforts geared toward information literacy is a project known as the Big6 (see http://www.big6.com/ for more info). It is a curriculum model geared at introducing information literacy into K-12 schools. The idea is for students to get better at problem solving, by evaluating the quality and usefulness of the information that they find. Research indicates that critical evaluation skills are based on the development of strategies that users apply in information settings. Strategies for successful evaluation consist of several sub-processes, with the origin of these strategies most likely being prior knowledge. Once the user has learned these evaluation mechanisms, they can be triggered through associations [30]. Training users in developing these evaluation skills could, for example, include the development of checklists. Several researchers have developed checklists, most often including evaluation criteria like accuracy, authority, objectivity, currency, and scope [64]. However, considering that knowledge is largely developed in situated context, an integrated approach to teaching and learning about credibility evaluation seems to emerge as a very effective approach and can help to speed up the process of evaluation. Moreover, it is important to make credibility evaluation an integral element in all aspects of the curriculum, unfortunately often students are given their assignment and are on their own when it comes to researching and evaluating information on the Internet [11]. Integration requires not only the development of strategies and skills, but requires also the development of learning outcomes, instructor support, and assessment across the grades as early as possible [11]. Availability of online information also changes the landscape of education from an isolated classroom environment to a more social, international and collaborative learning environment. When educating for credibility evaluation, cooperation on a broader national and international scale could assist users to understand environments outside of their domain.

41 Further, it is vital to educate and train teachers about credibility evaluation, pre-service teachers as well as in-service teachers. Educating the educators would reach the broadest possible audience, especially teachers who may have had very little exposure to formal information technology education. It would be desirable to reach a point where most educators are pro-active and not reactive in technology use. Often, students inform their teachers of new sites and technologies, and it should be a goal to have most educators in a position where they know more than their students [12].

5 Future of Credibility

During the past decade, several newly emerged techniques pose new challenges to both the credibility of both web sites and webbased information. The next sections will discuss phishing, captchas., and wikis.

5.1

Phishing

One of the most widely spread threats that has a significant impact on web credibility is phishing attacks. In case of a typical phishing attack, a perpetrator sends an official looking e-mail that attempts to obtain personal and financial information. Most phishing e-mails direct the recipient to a phony website designed to look like they come from well-known, legitimate, and trusted business, financial institutions, or government agencies [25]. The primary purpose of phishing is to obtain the consumer’s personal information such as usernames and passwords to engage in various fraud schemes. Although often being used interchangeably with the term “identify theft,” phishing is actually just one of a number of distinct methods that identity thieves use to steal information through deception [5]. 43

44 Future of Credibility

Fig. 5.1 Number of unique phishing reports received by Anti Phishing Work Group during 2004 to 2006. (Figure constructed based on data from APWG website.)

Phishing is a type of security threat with relatively short history but has experienced substantial growth in the past several years. Between December 2003 and June 2004, the number of unique phishing attacks rocketed from less than 100 cases to over 1400 cases. Figure 5.1 demonstrates the number of phishing reports received by Anti-Phishing Work Group [4], a leading multinational industry coalition that focuses on phishing, According to APWG, the number of unique phishing reports received each quarter has grown over 200%, from around 20,000 in the last quarter of 2004 to more than 76,000 in the last quarter of 2006. The existence of a large number of active phishing websites posts an enormous threat to the credibility of e-commerce websites and, in the long run, may undermine public trust in the use of the Internet for online banking and e-commerce. It was reported that some phishing schemes were able to convince up to 5% of the recipients to reveal their personal data, resulting in a significant number of consumers suffering from credit card fraud, identify fraud, and financial loss [3]. To further complicate the task of anti-phishing, a lot of companies or organizations that are victimized by phishing tend not to report those instances to law enforcement or the media, for the fear that their customers would lose trust to them if the true volume of the phishing attacks was made public.

5.2 CAPTCHA

45

The concern of the negative impact of phishing on web credibility is confirmed by a recent consumer survey that showed declining confidence in Internet security. More specifically, the survey found that nine out of ten adult Internet users in the United States had changed their Internet usage habit due to the threat of identify theft. Approximately 30% adult users hade reduced their overall Internet usage and 25% had stopped shopping online [18]. In order to combat phishing attacks, education of the general public to increase the awareness of phishing scams is highly critical. Multiple organizations have compiled lists of advice to the public for preventing and reporting phishing attacks [5]. Among them, important rules of thumb include: •

•

•

• •

• •

•

Protect your computer from viruses, spyware, and other security threats. Be suspicious of any e-mail requesting personal or financial information. Ensure that you are using a secured website when submitting credit card information. Regularly log into online accounts. Do not assume the legitimacy of a website just by its appearance. Avoid using the links in an e-mail to get to any web page. Avoid providing personal information by filing out forms in e-mail messages or pop-up windows. Always report a phishing attack to an authorized group such as FTC.

Besides public education, more effective authentication techniques, legislation, and law enforcement are also essential for limiting the detrimental impact of phishing schemes on web credibility.

5.2

CAPTCHA

CAPTCHA (also known as “Human Interaction Proofs,” or HIPs) stands for “Completely Automated Public Turing test to tell Computers and Humans Apart.” A CAPTCHA is a program that can generate

46 Future of Credibility and grade tests that most humans can pass, but current computer programs cannot pass [2]. CAPTCHAs are developed to prevent automated software from performing actions that will downgrade system performance. CAPTCHAs have been widely used by websites such as Yahoo, Hotmail, and PayPal to prevent e-mail spams, automated registrations, and automated postings to blogs and online communities. All these applications have great implications for both web credibility and online communication partner credibility. For example, during the 1999 online poll hosted by Slashdot.com asking votes for the best graduate school in computer science, the students at both Carnegie Mellon University (CMU) and Massachusetts Institute of Technology (MIT) wrote computer automated programs to vote for their own school. Eventually the online poll became a contest between the two automated programs, and both MIT and CMU received over 21,000 votes while all other schools received 1000 less. This raised a serious question as to whether we can trust any online polls and be confident that the votes are only from humans, not machines [2]. Another domain where automated programs post a direct threat to credibility is multiplayer online games. Online games have become an increasingly important social, cultural, and economic phenomena. Millions of players routinely gather online to play games for various reasons. One group of games that are particularly popular require human players to play against one another, in which case the inherent limitations of human physical and cognitive capabilities are essential factors for performance and the true enjoyment comes from the ability to transcend those limitations. However, some players use automated programs to gain unfair advantages when playing those games. Once “fair” players found out that they are playing against a machine, not a human, their trust toward online communication partners can easily be shattered [45]. CAPTCHAs provide an effective approach to distinguish automated programs from humans and therefore, play an important role in support a credible online environment. There are four major forms of CAPTCHA. The most commonly used CAPTCHA is based on the human capability to read distorted and corrupted text and the inability of existing computer technology to do the same. For instance, Figure 5.2

5.2 CAPTCHA

47

Fig. 5.2 Demonstration of CAPTCHA using distorted text. (Figure adopted from www.wikipedia.org, 2007.)

illustrates four distorted letters “SMWM” [90]. In a real use case, a website may require the user to read and enter the four letters before granting the user access to the system. Another form of CAPTCHA requires the user to solve a visual pattern recognition problem that is extremely difficult for computers to solve. The third form of CAPTCHA requires the user to view a number of pictures and identify the common features of those pictures. The fourth form of CAPTCHA employs audio cues and renders words or numbers into a sound clip, which the user must interpret. For more detailed descriptions and examples, please see to the CAPTCHA project website [13]. CAPTCHAs have been widely used in various domains such as online banking, e-commerce, as well as online games [45] and are becoming increasingly important in preventing automated abuses. However, there is very limited research on the impact of CAPTCHAs on individual user web credibility assessment. Chellapilla et al. evaluated the impact of the text distortion rate on humans [15]. It was found that as distortion rates increase, it becomes extremely difficult for humans to decode a CAPTCHA. No result was reported on whether the difficulty in solving a CAPTCHA affects the user’s perceived credibility of a website. Also, no studies have been reported on how users respond to different types of CAPTCHAs (e.g., text-based CAPTCHAs vs. imageor audio-based CAPTCHAs). Recent studies show that CAPTCHAs are not as robust and effective as previously believed. Instead, they can be quite vulnerable to attacks. For instance, object-recognition techniques have been employed to break commercial text-based CAPTCHAs and the accuracy can be as high as 99% [67]. Chellapilla et al. found that as

48 Future of Credibility distortion rates increase, it becomes extremely difficult for humans to decode the CAPTCHA while computer performance is barely affected [15]. These findings are highly disturbing since they indicate that computers have the potential to solve CAPTCHAs and the existing key barrier against mass automation abuses may be quite vulnerable. Another major challenge with CAPTCHAs is accessibility. The wide adoption of CAPTCHAs in online banking and e-commerce comes at a huge price to the large population of individuals with disabilities. CAPTCHAs are particularly problematic for individuals who are blind, visually impaired, or dyslexic [91]. Blind users depend on text equivalents readable by screen access software to access images or graphics on the Internet. However, CAPTCHAs are not accompanied with text equivalents since the text could be easily exploited by computerized programs. Therefore, CAPTCHAs make it impossible for thousands of blind screen reader users to create online accounts, complete transactions, or post comments. A small number of sites do offer audio versions of CAPTCHAs, which are accessible for individuals with visual or reading impairments. Users click on the link, listen to the audio clip, and write the text that they hear in the audio clip. However, these audio CAPTCHAs are rare on web sites, and it is possible that they are more prone to hacking than visual CAPTCHAs, so it is unknown if they will appear in greater numbers. World Wide Web Consortium has expressed deep concerns over the wide spread of visual CAPTCHAs on the Internet and doubts whether the CAPTCHA technique can provide the desired protection. Even with those concerns, the trend to adopt CAPTCHAs on the Internet will continue in the foreseeable future. For CAPTCHAs to be effective and accessible, considerable research need to be conducted to come up with innovative or improved solutions. Holman et al. made the first step forward and proposed an interesting new form of CAPTCHA that integrates both visual images and sounds, but uses concepts rather than text [50]. For instance, one of their CAPTCHAs provides the user with a picture of a cat and the sound a cat makes, and a user can utilize either one or both. The advantage of this new application is that it can be used by both visual users and blind users. Visual users can solve the CAPTCHA via either the picture or the sound. Blind

5.3 WIKIPEDIA and Open Source Models of Information

49

users can solve the CAPTCHA by listening to the matching sound of the picture. It also would work for users who are hearing impaired. A preliminary evaluation involving both visual users and blind users provided highly promising results. Both user groups could solve the new form of CAPTCHA easily with high accuracy and were quite satisfied with the interaction experience.

5.3

WIKIPEDIA and Open Source Models of Information

The web provides a platform for open source information models like Wikipedia, where information can be posted in a database by any user, even anonymous users, at any time. Although wikis support collaboration on a large scale across the globe, this type of information sharing is criticized by many for having no accountability and for publishing incorrect information [17]. Despite the concerns about the reliability of information on Wikipedia, it is a highly used online encyclopedia and is most popular among well-educated users. More than 50% of users with at least a college degree use Wikipedia on a regular basis, compared to only 22% of users with a high school diploma. Young adults (between 18 and 29 years of age) have especially shown to be early adopters of Wikipedia and use it on a regular basis. For example, 44% in this age group use Wikipedia as a resource for information [77]. Wikipedia is one of the most visited sites on the internet. The site’s popularity can be attributed to a number of factors. The database covers a vast amount of information; and information can be updated and added conveniently in a timely manner by any user. In addition, Wikipedia draws much traffic from search engines, which often rank articles according to the number of links found in them. Since Wikipedia articles usually contain numerous links, they often appear on top of the search list [77]. In addition, information seekers are attracted to Wikipedia by a convenient interface, convenient search procedures and result presentations. Interestingly, for users searching for science information, convenience matters more than accuracy; 71% of users seeking science information preferred sites that are convenient compared to 13%

50 Future of Credibility of users preferring accurate information [51]. However, it should be mentioned that users preferring convenience also performed additional checking using other online and offline information, suggesting that Wikipedia may be used as a filtering tool or a source for preliminary information gathering and evaluation [51]. These numbers are surprising and raise the issue of credibility of open source databases. How can users evaluate credibility of the information published in these environments? Applying the four types of credibility (presumed, surface, earned, and reputed credibility) described by Fogg to Wikipedia can provide some insight on how users may perform credibility evaluation in this open source environment [31]. Wikipedia credibility can easily be established in three of the four categories. All three categories are mainly based on the user’s perception of credibility. Presumed credibility in this case is established, for example, by using a .org identifier leading users to believe that the site is a non-profit site and thus more likely to be trustworthy. Surface credibility in this case, for example, is established by a professional looking design, simple and easy navigation, and little advertising. Earned credibility is established by a long-term relationship with the user. A typical example here is the group of science information seekers, who evaluate the credibility of Wikipedia by using other online and offline information sources. If it proves to be credible more often than not over a long period of time, this would establish earned credibility. Interestingly, the one category least influenced by a user’s perception is reputed credibility, which would be established by third party seals of approval or awards the site has won. Unfortunately, Wikipedia does not show any such third party approvals for users on their site. However, Wikipedia is often linked to by search engines, which users may perceive to be a trustworthy and believable source, and which in turn could establish some degree of reputed credibility. Open source information has many unique characteristics, but unfortunately some may be benefits as well as disadvantages. Information can be changed instantly by any person in the world and provides access to latest news and developments, but also leaves much room for errors and inconsistencies. Individual, social, and cultural differences further add to the complexity of credibility evaluation in these

5.3 WIKIPEDIA and Open Source Models of Information

51

Fig. 5.3 Wikipedia information about missing citations and/or footnotes.

environments. Credibility is difficult to evaluate for the information seeker, and users carry most of the burden to evaluate credibility. Social navigation systems may be used to increase credibility of open source databases and allow a more complex incorporation of the user into the evaluation process [23]. The popularity of social networking especially with the younger age group and the high number of young Wikipedia users makes an even more compelling argument for the integration of the user into the evaluation process. Figure 5.3 shows how Wikipedia integrates user evaluation into the process of information seeking. Articles are marked if they lack sources or if there are only few links pointing to the article. Users are encouraged to add and modify information to increase the accuracy of the open source database, and links are provided for easy user access.

6 Implications

Given the previous discussion, possible next steps are presented for various stakeholders. These stakeholders include browser designers, web site developers, users, librarians, and researchers.

6.1

Implications for Browser Designers

More features need to be added to web browsers, which can address information credibility. Currently, web browsers include many features to highlight issues related to security. For instance, there is usually a padlock or other icon that indicates that you are using a secure site. A dialog box pops up, noting that you are about to enter a secure site (or that you are about to leave a secure site). There is no reason that similar features, related to web credibility, could not be developed. For instance, rather than a stand-alone tool (the Information Quality tool described earlier in the paper), a short pop-up, which allows users to evaluate the credibility of information, could be integrated into the browser. Other software tools (as listed in Section 3.1.3) have already started to appear in browsers, and while they do evaluate whether sites might be misleading, these tools only present the computer’s guess as 53

54 Implications to the credibility of a web site. These tools integrated into browsers do not support user decision making. Since there are a number of known certification seals for information credibility, the browser could alert the user when they are going to a certified credible site, and when they are leaving a certified credible site. An icon could be added, similar to the padlock icon that indicates a secure site, and the icon could indicate when the user is visiting a credible site. Equivalents for the icons (such as audio icons) should also be made available for users with visual impairment.

6.2

Implications for Web Site Developers

In this paper, suggestions for highlighting the credibility of a web site were presented. Like most usability guidelines, the guidelines are complex, and will not be read by large numbers of people who are actually developing web sites. To help with this effort, those leading the web credibility effort should take a page from those focusing on web accessibility. Knowing that most people would never read their 50+ page guidelines document on web accessibility, the Web Accessibility Initiative (http://www.w3.org/wai) created a set of 10 “quicktips” for making a web site accessible. These quicktips fit on a business card, and they are short (even shorter than typical usability heuristics), to the point, and can be understood by a wide range of information technology professionals. These cards are given away for free, and up to 500 cards can be ordered at no charge. This has been one of the most useful documents ever created related to web accessibility. Therefore, a similar approach might be useful. A short list of “quicktips,” which address design features that highlight whether a site is credible, could be distributed to web developers. As part of these quicktips, developers should be encouraged to incorporate a number of features into their web sites. Certainly, privacy policies and noting the source and date of all information can help establish the credibility of information. In addition, where possible, reputation systems should be incorporated into web sites. When users can add information to the web site, there should be some feature where they can note their expertise and experience, which can be used to help evaluate the credibility

6.3 Implications for Users

55

of the information that they post. However, as discussed previously in the paper, when providing reputation systems, the web site should remind the user of the possible fraudulent manipulation of the reputation systems and suggest the user to use the information with caution.

6.3

Implications for Users

Users need to have clear tools that help them evaluate the credibility of information on the web. In earlier sections, we proposed adding design features into web browsers, that help users evaluate the credibility of web sites, such as pop-up tools, and new dialog boxes and interface widgets. We also propose that, similar to the quicktips described for web developers, that a similar set of quicktips be developed for users. Since users will have to evaluate multiple web sites for credibility on a regular basis, we suggest limiting the number of quicktips to five. Again, these quicktips for evaluating web sites can be distributed via business cards. We propose the following five heuristics for assisting users in determining the credibility of a web site: 1. Is the site sponsor clear, and would it be considered unbiased? 2. Is the source of the specific information clear, and would it be considered a knowledgeable source? 3. Is there a certification seal? 4. Is the date that the information was created and/or posted clear? 5. Have outside parties (librarians, knowledgeable experts, recommender communities, etc.) established the credibility of information on the site? Another issue for users is determining the credibility of incoming e-mails. More users are aware of the challenges of evaluating incoming e-mails (which may have viruses) than are aware of the challenge of evaluating web sites. Many users already know not to open file attachments from those that they do not know. The press frequently reports events related to new viruses and/or worms, and tells the public what

56 Implications keywords, phrases, or hints to watch out for. Users are more savvy about assessing the credibility of incoming e-mails. However, as the complexity of these threats becomes more of a problem, users will need increased education on topics of spyware, bots, and updating virus protection software.

6.4

Implications for Librarians

Based on the research described in the paper, it is clear that users DO need assistance in determining the credibility of web-based information. Efforts such as the Librarian’s Index to the Internet are helpful. The number of similar efforts needs to be expanded. This is not to be confused with censorship. Censorship occurs when users are limited in what they can see. Users can still visit whichever sites they prefer, however, when librarians help sort sites by information quality, users then have the option of visiting sites that are more credible. However, the web is a dynamic place, and it is nearly impossible to sort through and evaluate all web sites in advance. Therefore, the better and more realistic goal is for librarians to continue with their Herculean efforts to educate users about the topic of web credibility. As experts in the field of evaluating information quality, librarians are the group poised to take the lead in educating all groups about the problem of web site and communication partner credibility.

6.5

Implications for Researchers

There are many potential directions for researchers in the area of credibility. These research areas include a wide range of both human and technology-related topics. Certainly, new tools need to be developed that can help users evaluate the credibility of web sites and online communication partners. Different approaches to establishing credibility online must be developed, and the security and robustness of these approaches must be evaluated. Similarly, more research needs to be done on understanding the strategies that users take when evaluating the credibility of online information. This is especially true of diverse

6.5 Implications for Researchers

57

user populations. How do older users evaluate if the banking site that they are using is credible? How do users with disabilities determine if a news site is credible? When you receive an e-mail from your friend, what aspects of the e-mail do you use to determine if the e-mail is real or is spoofed? These are the types of research questions that need more attention.

7 Summary

Web credibility is a multi-dimensional problem, involving tools and technologies, users, institutions, and policies. Only when attention is paid to all four aspects of this problem can solutions truly be found. If users want to determine the credibility of a web site, the web site sponsor will also need to be interested in proving their site credibility. If a web site sponsor wants to make clear that their web site is credible, the user must spend the time to evaluate the credibility. Institutions, such as libraries and schools, must help to educate users about web credibility. Some tools do exist now to help users evaluate credibility, such as spam filters, evaluation lists, and certification seals. In the future, new approaches for evaluating credibility, such as built-in browser features, and language parsers, will need to be developed. Those hoping to deceive users on the web have taken the first steps. It’s time for those interested in establishing credibility on the web to catch up.

59

References

[1] M. Ackerman and S. Mainwaring, “Privacy issues and human-computer interaction,” in Security and Usability, (L. Cranor and S. Garfinkel, eds.), Sebastopol, CA: O’Reilly Media, 2005. [2] L. Ahn, M. Blum, and J. Langford, “Telling humans and computers apart automatically,” Communications of the ACM, vol. 47, no. 2, pp. 57–60, February 2004. [3] Anti-Phishing Working Group, Phishing Activity Trends Report, January 2005. Retrieved on March 10, 2007 from http://www.antiphishing.org/ reports/APWG Phishing Activity Report-January2005.pdf, 2005. [4] Anti-Phishing Working Group, Phishing Attach Trends Reports, Retrieved on March 10, 2007 from http://www.antiphishing.org/reports/ APWG Phishing Activity Report-January2005.pdf, 2006. [5] Binational Working Group on Cross-Border Mass Marketing Fraud Report on phishing. Retrieved on March 10, 2007 from http://www.usdoj.gov/ opa/report on phishing.pdf, 2006. [6] M. Bishop, “Psychological acceptability revisited,” in Security and Usability, (L. Cranor and S. Garfinkel, eds.), Sebastopol, CA: O’Reilly Media, 2005. [7] N. Bos, J. Olson, D. Gergle, G. Olson, and Z. Wright, “Effects of four computermediated communications channels on trust development,” in Proceedings of ACM Conference on Human Factors in Computing Systems (CHI), pp. 135– 140, 2002. [8] A. Bouch, A. Kuchinsky, and N. Bhatti, “Quality is in the eye of the beholder: Meeting users’ requirements for Internet quality of service,” in Proceedings of ACM Conference on Human Factors in Computing Systems (CHI), pp. 297– 304, 2000. 61

62 References [9] P. Boykin and V. Roychowdhury, “Personal email networks: An effective antispam tool,” IEEE Computer, vol. 38, no. 4, pp. 61–68, 2005. [10] J. Brown, A. Collins, and P. Duguid, “Situated cognition and the culture of learning,” in Classic Writings on Instructional Technology, (D. Ely and T. Plomp, eds.), Englewood, Colorado: Libraries Unlimited, Inc., 2001. [11] A. Bruckman, “Student research and the Internet,” Communications of the ACM, vol. 48, no. 3, pp. 35–37, 2005. [12] G. Byerly and C. Brodie, “Internet (and/or Institutional) Credibility and the User,” Symposium on Internet Credibility and the User, Available at: http:// projects.ischool.washington.edu/credibility/symposium papers.htm, 2005. [13] CAPTCHA Project The CAPTCHA Project website, Retrieved on March 10, 2007 from http://www.captcha.net/, 2007. [14] I. Ceaparu, D. Demner, E. Hung, H. Zhao, and B. Shneiderman, “Web we trust: Establishing strategic trust among online customers,” in E-Service, (R. Rust and P. Kannan, eds.), pp. 90–107, Armonk, NY: E. Sharpe Publishers, 2002. [15] K. Chellapilla, P. Simard, and M. Czerwinski, “Computers beat humans at single character recognition in reading-based human interaction proofs (HIPs),” in Proceedings of the Second Conference on E-mail and Anti-Spam (CEAS), 2005. [16] M. Chen and J. Singh, “Computing and using reputations for Internet ratings,” in Proceedings of ACM Conference on E-Commerce, pp. 154–162, 2001. [17] CNET News, Growing pains for wikipedia. Available at: http://news.com.com/ Growing+pains+for+Wikipedia/2100-1025 3-5981119.html, 2005. [18] Consumer Reports WebWatch, Leap of faith: Using the Internet despite the dangers. 2005. Retrieved on March 10, 2007 from: http://www.Consumer WebWatch.org. [19] D. Cook, J. Hartnett, K. Manderson, and J. Scanlan, “Catching spam before it arrives: Domain specific dynamic blacklists,” in Proceedings of the 2006 Australasian Workshops on Grid Computing and E-research, vol. 54, 2006. [20] L. Cranor and S. Garfinkel, eds., Security and Usability. Sebastopol, CA: O’Reilly Media, 2005. [21] S. Curzon, “Internet credibility: Seven teaching challenges,” Symposium on Internet Credibility and the User, Available at: http://projects.ischool. washington.edu/credibility/symposium papers.htm, 2005. [22] K. Daisay, “Trust in E-commerce,” Communications of the ACM, vol. 48, no. 2, pp. 73–77, 2005. [23] P. DiGioia and P. Dourish, “Social navigation as a model for usable security,” in Proceedings of Symposium on Usable Privacy and Security (SOUPS), pp. 101– 108, 2005. [24] S. Diller, L. Lin, and V. Tashjian, “The evolving role of security, privacy, and trust in a digitized world,” in The Handbook of Human-Computer Interaction, (J. Jacko and A. Sears, eds.), pp. 1213–1225, Mahwah, NJ: Lawrence Erlbaum Associates, 2003. [25] J. Downs, M. Holbrook, and L. Cranor, “Decision strategies and susceptibility to phishing,” in Proceedings of Symposium on Usable Privacy and Security (SOUPS), pp. 79–90, 2006.

References

63

[26] J. Earp and P. Baumer, “Innovative web use to learn about consumer behavior and online privacy,” CACM, vol. 46, no. 4, pp. 81–83, 2003. R [27] Epistemology, The American Heritage Dictionary of the English Language. fourth ed. Retrieved March 07, 2007. Available at: http://dictionary. reference.com/browse/epistemology, no date. [28] D. Fallows, “Search engine users: Internet searchers are confident, satisfied and trusting — but they are also unaware and naive,” Pew Internet and American Life Project, Available at: http://www.pewinternet.org/ report display.asp?r=146, 2005. [29] J. Feng, J. Lazar, and J. Preece, “Empathy and online interpersonal trust: A fragile relationship,” Behaviour and Information Technology, vol. 23, no. 2, pp. 97–106, 2004. [30] M. Fitzgerald, “Skills for evaluating web-based information,” Symposium on Internet Credibility and the User, Available at: http://projects. ischool.washington.edu/credibility/symposium papers.htm, 2005. [31] B. Fogg, Persuasive Technology. New York, NY: Morgan Kaufmann Publishers, 2003. [32] B. Fogg, J. Marshall, T. Kameda, J. Solomon, A. Rangnekar, J. Boyd, and B. Brown, “Web credibility research: A method for online experiments and early study results,” in Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHi), pp. 295–296, 2001. [33] B. Fogg, J. Marshall, O. Laraki, A. Osipovich, C. Varma, N. Fang, J. Paul, A. Rangnekar, J. Shon, P. Swani, and M. Treinen, “What makes web sites credible? A report on a large quantitative study,” in Proceedings of CHI 2001: Human Factors in Computing, pp. 61–68, 2001. [34] B. Fogg, C. Soohoo, D. Danielson, L. Marable, J. Stanford, and E. Tauber, “How do users evaluate the credibility of web sites?: A study with over 2500 participants,” in Proceedings of the 2003 Conference on Designing for User Experiences, pp. 1–15, 2003. [35] B. Fogg and H. Tseng, “The elements of computer credibility,” in Proceedings of ACM Conference on Human Factors in Computing Systems (CHI), pp. 80–86, 1999. [36] S. Fox, “Older Americans and the Internet,” Pew Internet and American Life Project, Available at: http://www.pewinternet.org/reports.asp, 2004. [37] S. Fox, “Are “Wired Seniors” sitting ducks?,” Pew Internet and American Life Project, Available at: http://www.pewinternet.org/reports.asp, 2006. [38] S. Fox and L. Rainie, “The online health care revolution: How the Web helps Americans take better care of themselves,” Pew Internet and American Life Project, Available at: http://www.pewinternet.org/report display.asp?r=26, 2000. [39] S. Fox and L. Rainie, “Vital decisions: How internet users decide what information to trust when they or their loved ones are sick,” Pew Internet and American Life Project, Available at: http://www.pewinternet.org/ report display.asp?r=59, 2002.

64 References [40] J. Fritch, “Heuristics, tools, and systems for evaluating Internet information: helping users assess a tangled Web,” Online Information Review, vol. 27, no. 5, pp. 321–327, 2003. [41] S. Garfinkel, J. Schiller, E. Nordlander, D. Margrave, and R. Miller, “How to make secure email easier to use,” in Proceeding of ACM CHI 2005, pp. 701–710, Portland, Oregon, April 2–7 2005. [42] S. Garfinkel, J. Schiller, E. Nordlander, D. Margrave, and R. Miller, “Views, reactions and impact of digitally-signed mail in e-commerce,” in Proceedings of Financial Cryptography and Data Security Ninth International Conference, Roseau, The Commonwealth of Dominica, 2005. [43] J. Golbeck, “The science of trust on the web,” Foundations and Trends in Web Science, (To appear), 2007. [44] J. Golbeck and J. Hendler, “Reputation network analysis for email filtering,” in Proceedings of the First Conference on E-mail and Anti-Spam, Mountain View, California, 2004. [45] P. Golle and N. Ducheneaut, “Keeping bots out of online games,” in Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 262–265, Spain, 2005. [46] J. Goodman, G. Cormack, and D. Heckerman, “Spam and the ongoing Battle for the inbox,” Communications of the ACM, vol. 50, no. 2, pp. 25–33, 2007. [47] J. Hancock, J.Thom-Santelli, and T. Ritchie, “Deception and design: The impact of communication technology on lying behavior,” in Proceedings of ACM Conference on Human Factors in Computing Systems, pp. 129–134, 2004. [48] R. Hiltz and M. Turoff, “Structuring computer-mediated communication systems to avoid information overload,” Communications of the ACM, vol. 28, no. 7, pp. 680–689, 1985. [49] H. Hochheiser and B. Shneiderman, “Universal usability statements: Marking the trail for all users,” Interactions, pp. 16–18, 2001. [50] J. Holman, J. Lazar, J. Feng, and J. D’Arcy, “Developing usable CAPTCHAs for blind users,” in Proceedings of ASSETS 2007, 2007. [51] J. Horrigan, The Internet as a Resource for News and Information About Science, Available at: http://www.pewinternet.org/PPF/r/191/report display.asp, 2006. [52] J. Jacko, A. Sears, and M. Borella, “The effect of network delay and media on user perceptions of web resources,” Behaviour and Information Technology, vol. 19, no. 6, pp. 427–439, 2000. [53] C. Jensen, J. Davis, and S. Farnham, “Finding others online: Repuation systems for social spaces online,” in Proceedings of ACM Conference on Human Factors in Computing Systems, pp. 447–454, 2002. [54] D. Jonassen, “Objectivism versus constructivism: Do we need a new philosophical paradigm?,” in Classic Writings on Instructional Technology, (D. Ely and T. Plomp, eds.), Englewood, Colorado: Libraries Unlimited, Inc. (Original Work published 1991), 2001. [55] F. Langfitt, “Women file suit to defend reputation,” National Public Radio, Retrieved July 26, 2007 from http://www.npr.org/templates/story/ story.php?storyId=11159611&ft=1&f=1070, 2007.

References

65

[56] J. Lazar, Web Usability: A User-Centered Design Approach. Boston: AddisonWesley, 2006. [57] J. Lazar, P. Beere, K. Greenidge, and Y. Nagappa, “Web accessibility in the Mid-Atlantic United States: A study of 50 web sites,” Universal Access in the Information Society, vol. 2, no. 4, pp. 331–341, 2003. [58] J. Lazar and J. Preece, “Classification schema for online communities,” in Proceedings of the 1998 Association for Information Systems Americas Conference, pp. 84–86, Baltimore, MD, 1998. [59] J. Lazar and A. Sears, “Design of E-business web sites,” in Handbook of Human Factors and Ergonomics, (G. Salvendy, ed.), pp. 1344–1363, Hoboken, NJ: John Wiley & Sons, 2005. [60] E. Loiacono and S. McCoy, “Web site accessibility: An online sector analysis,” Information Technology and People, vol. 17, no. 1, pp. 87–101, 2004. [61] W. Luo and M. Najdawi, “Trust building measures: A review of consumer health portals,” Communications of the ACM, vol. 47, no. 1, pp. 109–113, 2004. [62] C. Lynch, “When documents deceive: Trust and provenance as the new factors for information retrieval into tangled web,” Journal of the American Society for Information Science and Technology, vol. 52, no. 1, pp. 12–17, 2001. [63] MAAWG (Messaging Anti-Abuse Working Group), MAAWG Email metrics program. first quarter 2006 reprts. June 2006. Available at www.maawg.org/ about/FINAL 1Q2006 metrics report.pdf, 2006. [64] M. Metzger, “Understanding how Internet users make sense of credibility: A review of the state of our knowledge and recommendations for theory, policy, and practice,” Symposium on Internet Credibility and the User, Available at: http://projects.ischool.washington.edu/credibility/symposium papers.htm, 2005. [65] M. Metzger, A. Flanagin, K. Eyal, D. Lemus, and R. McCann, “Credibility for the 21st Century: Integrating perspectives on source, message, and media credibility in the contemporary media environment,” in Communication Yearbook 27, (P. Kalbfleisch, ed.), pp. 293–335, Mahwah, NJ: Lawrence Erlbaum Associates, 2003. [66] G. Milne and M. Culnan, “Using the content of online privacy notices to inform public policy: A longitudinal analysis of the 1998–2001 U.S. web surveys,” The Information Society, vol. 18, no. 5, pp. 345–360, 2002. [67] G. Mori and J. Malik, “Recognizing objects in adversarial clutter: Breaking a visual CAPTCHA,” in Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 134–141, 2003. [68] B. Nardi, S. Whittaker, and E. Bradner, “Interaction and outeraction: Instant messaging in action,” in Proceedings of ACM Conference on ComputerSupported Cooperative Work (CSCW), pp. 79–88, 2000. [69] National Telecommunications and Information Administration [NTIA], A Nation Online: Entering the Broadband Age. U.S. Department of Commerce, Available at: http://www.ntia.doc.gov/reports/anol/ NationOnlineBroadband04.htm, 2004.

66 References [70] Nielsen Norman Group, Web Usability for Senior Citizens — Summary, Available at: http://www.nngroup.com/reports/seniors/, 2002. [71] J. O’Donovan and B. Smyth, “Trust in recommender systems,” in Proceedings of ACM 2005 Conference on Intelligent User Interfaces, pp. 167–174, 2005. [72] R. Petty and J. Cacioppo, Communication and Persuasion: Central and Peripheral Routes to Attitude Change. New York, NY: Springer-Verlag, 1986. [73] Pew Internet, “Internet activities latest trends,” Pew Internet and American Life Project, Available at: http://www.pewinternet.org/trends/ Internet Activities 1.11.07.htm, 2007. [74] J. Preece, Online Communities: Designing Usability, Supporting Sociability. New York: John Wiley & Sons, 2000. [75] J. Preece, J. Lazar, E. Churchill, H. DeGraaff, B. Friedman, and J. Konstan, “Spam, spam, spam, spam: How can we stop it?,” Extended Abstracts of the ACM Conference on Human Factors in Computing Systems, pp. 706–707, 2003. [76] L. Rainie and P. Hitlin, “The use of online reputation and rating systems,” Pew Internet and American Life Project, Available at: http:// www.pewinternet.org/report display.asp?r=140, 2004. [77] L. Rainie and B. Tancer, “Wikipedia users,” Pew Internet and American Life Project, Available at: http://www.pewinternet.org/PPF/r/212/ report display.asp, 2007. [78] J. Ramsay, A. Barbesi, and J. Preece, “A psychological investigation of long retrieval times on the World Wide Web,” Interacting with Computers, vol. 10, no. 1, pp. 77–86, 1998. [79] S. Y. Rieh and D. Danielson, “Credibility: A multidisciplinary framework,” in Annual Review of Information Science and Technology, (B. Cronin, ed.), pp. 307–364, 2007. [80] M. Sahami, S. Dumais, D. Heckerman, and E. Horvitz, “A Bayesian approach to filtering junk e-mail,” in Learning for Text Categorization Papers from the AAAI Workshop, AAAI Technical Report WS-98-5, 1998. [81] M. Sasse and I. Flechais, “Usable security,” in Security and Usability, (L. Cranor and S. Garfinkel, eds.), Sebastopol, CA: O’Reilly Media, 2005. [82] B. Shneiderman, “Designing trust into online experiences,” Communications of the ACM, vol. 43, no. 12, pp. 57–59, 2000. [83] E. Sillence, P. Briggs, L. Fishwick, and P. Harris, “Trust and mistrust of online health sites,” in Proceedings of ACM Conference on Human Factors in Computing Systems (CHI), pp. 663–670, 2004. [84] I. Simonson and P. Nye, “The effect of accountability on susceptibility to decision errors,” Organizational Behavior and Human Decision Processes, vol. 51, no. 3, pp. 416–446, 1992. [85] S. Stolfo, S. Hershkop, K. Wang, O. Nimeskern, and C. Hu, “A behaviorbased approach to securing email systems: Mathematical methods, models and architectures for computer networks security,” in Proceedings, Springer Verlag, September 2003. [86] Q. Su and C. Adams, “Will B2C E-commerce developed in one cultural environment be suitable for another culture: A cross-cultural study between

References

[87]

[88]

[89]

[90] [91] [92]

67

amazon.co.uk and dangdang.com,” in Proceedings of International Conference of E-commerce (ICEC), pp. 236–243, 2005. T. Sullivan and R. Matson, “Barriers to use: Usability and content accessibility on the web’s most popular sites,” in Proceedings of ACM Conference on Universal Usability, pp. 139–144, 2000. S. Sussman and L. Sproull, “Straight talk: Delivering bad news through electronic communication,” Information Systems Research, vol. 10, no. 2, pp. 150– 166, 1999. N. VanHouse, “Trust and epistemic communities in biodiversity data sharing,” in Proceedings of ACM 2002 Joint Conference on Digital Libraries, pp. 231–239, 2002. Wikipedia, CAPTCHA, Retrieved on March 10, 2007, from http://en. wikipedia.org/wiki/Image:Captcha.jpg, 2007. World Wide Web Consortium (W3C), Inaccessibility of CAPTCHA. Retrieved on March 10, 2007 from http://www.w3.org/TR/turingtest/, 2007. J. Zheng, E. Veinott, N. Bos, J. S. Olson, and G. M. Olson, “Trust without touch: Jumpstarting long-distance trust with initial social activities,” in Proceedings of Conference on Human Factors in Computing Systems (CHI), pp. 131–146, 2002.