Credibility and the World Wide Web

Overview

If you look at the most frequently visited Web sites, you’ll see that many sites seek to persuade users in some way. MSN and other leading portal sites such as AOL and Yahoo try to convince users to do their Web searching, shopping, and chatting with friends on their sites or those of their affiliates. They also hope that users will register with them—create a personalized homepage, such as My MSN, My AOL, or My Yahoo—giving the site operators information about users and providing a way to contact them directly in the future. These portals succeed only when they are successful at persuasion.

Credibility is a key factor in a Web site’s ability to persuade.

Even sites that focus mainly on providing information and content, such as about.com or cnet.com, attempt to persuade. Their goal is to convince users that visiting their site is the best way to get what they need, be it news, an mp3 file, or the latest games. Technical support sites try to persuade users to solve their problems online rather than calling the company. Even personal Web sites have persuasion as part of their objective. People want those visiting their site to think of them in the best light, as competent professionals or interesting human beings. As I see it, if someone didn’t want to influence others in some way, he or she would not take the time or energy to set up aWeb site.

The Importance of Web Credibility

While many factors contribute to a Web site’s power to influence, one key factor is credibility. Without credibility, sites are not likely to persuade users to change their attitudes or behaviors—to embrace the site’s cause, register personal information, make purchases, click on ads, complete surveys, or bookmark the site for return visits. For this reason, it’s important to understand and design for Web credibility.

What makes a Web site credible? What elements cause people to believe what they find on the site? Web credibility—what it is, how it is won, and how to understand it better—is the focus of this chapter.

Variability of Web Credibility

The Web can be a highly credible source of information, depending on the person or organization behind a given Web site (the site “operator”). The Web also can be one of the least credible information sources. Many pages on the Web reflect incompetence, and some are pure hoaxes. You’ve probably been to Web sites that not only lacked expertise, but seem designed to deceive. Because few barriers prevent people from publishing on the Web, you’ll find deceptive coverage of current events, health information that is factually incorrect, and ads that promise the impossible.

One notable example of bogus information online was designed to teach investors about online fraud and persuade people to be more skeptical about what they find online. At mcwhortle.com, Web surfers read that “McWhortle Enterprises is an established and well-known manufacturer of biological defense mechanisms.” As surfers browse this site, they see an “About Us” page, along with a doctored photograph of the company headquarters in Washington, D.C., contact information, and testimonials. The company site claims to have technology to produce a portable biohazard alert detector and seeks outside funding. As Web surfers investigate more and click on the link that says, “When you are ready to invest, click here,” the ruse is made known, with the heading “Watch out! If you responded to an investment idea like this . . . You could get scammed!” and a message below indicating that the site was posted by various government agencies, including the Securities and Exchange Commission and the Federal Trade Commission. [1 ]The SEC has said it will create hundreds of hoax Web sites of this type to increase investor awareness.

Some deceptive Web sites have been created with malicious intent, such as the fraudulent sites paypai.com, created to fool users into thinking it was the PayPal site, and wwwbankofamerica.com, which posed as the giant bank. Both sites attempted to elicit personal information from visitors for fraudulent purposes. [2 ]

Other deceptive sites are set up simply to amuse browsers who happen by, [3 ]such as the site for the Oklahoma Association of Wine Producers, [4 ]the Christian Women’s Wrestling site, [5 ]and the site for the End of the Internet (“Thank you for visiting the End of the Internet. There are no more links. You must now turn off your computer and go do something productive.”). [6 ]

Especially if they are new to the Web, people’s general perception of this medium’s credibility can plummet after they are deceived a few times or find factual errors. [7 ]If you spend a reasonable amount of time surfing the Web, you’ll get a sense of the broad range of credibility online. With time and experience, most users learn how to separate the good from the bad, the believable from the unbelievable.

[1 ]For more about the fake Web site whortle.com, see an article online by the Associated Press at http://www.businesstoday.com/business/technology/ap_fake01302002.htm or the SEC’s press release at http://www.sec.gov/news/headlines/scamsites.htm.

[2 ]For more information about these and other Internet scams, visit the Identity Theft Resource Center at http://www.idtheftcenter.org/html/scam-alerts.htm.

[3 ]One site has gathered a few dozen examples ofWeb site hoaxes. See http://www.museumofhoaxes.com/hoaxsites.html.

[4 ]The hoax Web site for the Oklahoma Association of Wine Producers is at http://Web.fvdes.com/Web_Eval_TL/OKWine2/okawp.html.

[5 ]The farcical Christian Women’s Wrestling (CWW) site can be found at http://www.jesus21.com/poppydixon/sports/cww.html.

[6 ]The humor page claiming to be last page of the Internet is at http://www.shibumi.org/eoti/index.htm.

[7 ]For one of the early editorials pointing out that the Web is facing a credibility crisis, see R.Kilgore, Publishers must set rules to preserve credibility, Advertising Age, 69 (48): 31 (1998).

Two Sides of Web Credibility

Web credibility has two sides, one that relates mostly to Web surfers and one that relates to Web designers (Figure 7.1). On the one side, credibility is important to those who use the Web. People need to evaluate if Web sources are credible or not. This issue of “information quality” has been embraced by librarians and teachers. [8 ]

Figure 7.1: Web credibility from two points of view.

To better assess the credibility of an online source, librarians and others advocate that Web surfers examine who the author is, how timely the content is, and how the content compares with similar content from trusted sources, such as experts in the field. Fortunately, many good guidelines already exist to help students and researchers evaluate the information they find online. [9 ]

The other aspect of Web credibility relates primarily to Web site designers. The main issue for designers is how to create Web sites that convey appropriate10 levels of credibility, as discussed in Chapter 6.

Although designing for Web credibility is an important issue for designers, in 1998 there was almost no information or research related to the subject. [11 ]To help fill the gap, my colleagues and I began conducting research projects to gain a better understanding of the factors that go into Web credibility. Our hope was that our research, described in this chapter, would be a key resource for people interested in Web credibility issues. [12 ]The chapter lays a foundation for deeper understanding by providing ways to think about Web credibility, sharing results from quantitative research, and offering general principles about designing for credibility.

[8 ]For a librarian’s view of evaluating Web information, see the site from Hope Tillman, director of Libraries at Babson College: http://www.tiac.net/users/hope/findqual.html.

[9 ]Some excellent guidelines exist for evaluating Web information quality. For example, see http://wwwlibrary.csustan.edu/lboyer/webeval/webeval.htm. Also see the work of Kaaren Struthers at http://leo.stcloudstate.edu/research/credibility1.html. Finally, visit the Librarians’ Index to the Internet at http://www.lii.org/.

[11 ]While most Web designers seek to design sites with maximum levels of credibility, a more admirable goal would be to design sites that convey appropriate levels of credibility— that is, sites that accurately convey their performance levels, the quality of their content, and so on.

[12 ]As my lab team was researching Web credibility in 1998, Cheskin Research was researching a related, but not identical, area: trust in ecommerce. See eCommerce Trust Study, a joint research project by Cheskin Research and Studio Archetype/Sapient. (January 1999). Available online at http://cheskin.com/think/trust/assets/images/etrust.pdf.

They followed up this study with a study with an international scope: Trust in the Wired Americas (July 2000). Online at http://cheskin.com/think/studies/trustIIrpt.pdf.

The Stanford Web Credibility Studies

Over the past four years, my Stanford Persuasive Technology Lab has conducted a number of research studies related to Web credibility, involving over 6,000 participants. After conducting a few smaller experiments and surveys, in the fall of 1999, 10 graduate students[13] and I launched a large-scale research project to investigate Web credibility. [14 ]

We were intrigued by the results of the 1999 study[15] and decided to take another “snapshot” in 2002,[16] to see how perceptions had changed in the intervening years. We plan to repeat the study every two or three years, to give us not only a snapshot of Web credibility at single points in time but a better understanding of how the Web and user perceptions of Web credibility are changing over time.

The 2002 study was nearly identical in content and methodology. However, because the Web is a dynamic medium, some changes were necessary for the newer study. While trying to keep changes to a minimum, we discarded a few questions that seemed outdated, reworded a few questions, and added some questions to probe new issues in the shifting landscape of Web use and technology. In 1999we asked people about their credibility perceptions of 51 items; in 2002 we asked about 55 items.

Developing the questions for each survey was a rather involved and careful process. In 1999, after the graduate students and I reviewed the few existing pieces of literature on Web credibility and surfed many Web sites, taking notes, our team identified more than 300 elements that could affect the way users perceived the credibility of a given Web site. Because this set contained too many elements to manage in a single study, through an iterative process involving lots of discussion, we selected a subset—the items we found most important or interesting. [17 ]

We ran three pilot studies to refine our methodology and questions. Only after refining the questions and methodology did we launch our first large scale online survey, in 1999.[18]

We made sure the experience of participating in the research was similar in both studies. After reading an introductory passage, the participants read through roughly 50 randomized items describing a Web site element. They were asked to rate each element on a scale of –3 to +3 to show how that element would affect their perception of the credibility of the site’s information. We asked participants to draw upon their cumulative experience with the Web to make their evaluations. Figure 7.2 lists some of the survey questions used in both studies.

Figure 7.2: Excerpt from online Web credibility survey. In 1999, 1,410 participants completed the survey; in 2002, we had 1,481 participants.

In 1999, more than 1,400 people participated in the survey; roughly the same number participated in 2002.19We used similar recruiting methods both times, and we were fortunate in how things shaped up: as you’ll see in Table 7.1, the demographics for the studies are similar.

A Few Words about Our Findings

The overall results from our two large studies are summarized on the continuum in Figure 7.3. This graphic illustrates the results of over 3,000 people responding to more than 50 items. As you examine the figure, you’ll see how each item fared overall in 1999 (the scores in black boxes). Possible scores range from+3 to –3, matching the response scale we used in the study; in other words, if the average response to an item in 1999 was +1.5, Figure 7.3 lists the item at +1.5 on the continuum.

Figure 7.3: Summary of Stanford Web Credibility Survey results: 1999 and 2002.

The graphic also shows how the data changed in the 2002 study. For each item that was worded identically in both studies, you’ll find a small arrow and a value indicating how much the mean score for that particular item increased or decreased in the 2002 study. (As noted at the bottom of Figure 7.3, items that were reworded or dropped in 2002 are indicated with a * or “NA,” respectively). As Figure 7.3 shows, the highest mean score in the studies was +2.2; the lowest was –2.1. All the other credibility items in the study fall between these points; they are placed along the continuum, top to bottom.

In reviewing the continuum, I find it interesting to see what items landed at the top (those that had the greatest positive impact on credibility), at the bottom (those that had the greatest negative impact), and which fell in the middle (items that had no impact on Web credibility).

It’s also worth noting that items on the continuum must be at least 0.3 apart to be considered truly different in terms of their practical (versus statistical) significance. For example, looking at the 1999 data, there’s no practical significant difference between listing “the organization’s physical address” (+2.0) and listing a “contact phone number” (+1.9). However, there is a practical significant difference between a site that lists a physical address and a site that recognizes that you have been there before (+0.7). [20 ]

Next, it’s important to note that the items listed in Figure 7.3 represent the exact wording used in our surveys. To be sure, people will differ in how they interpret each item; this is an inevitable weakness of any survey. This was one reason for conducting our pilot tests before launching the 1999 study—to weed out bad questions and refine other questions in order to reduce ambiguity. (For the most part we succeeded, as evidenced by the fact that the averages on each item for U.S. and Finnish respondents closely matched, despite the differences in native language. If the questions were ambiguous, you would expect to see more evidence of interpretation differences.)

My research team and I believe the results of these studies are robust, since the findings of the pilot studies[21] and our two large-scale studies were similar. The convergence in results we’ve seen, despite the differences in methodology, sample, and time, give us confidence in our findings.

Even though we’ve seen similar findings appear in our various studies, our work so far represents early investigations into the elements of Web credibility.

This is not the final word. Our intention was to cover a lot of territory and, along the way, raise new issues and questions to explore in future credibility research. In other words, the results in Figure 7.3 and in the rest of this chapter provide a starting point for discussion, speculation, confirmation, and critique.

We consider these studies successful if they do any of the following:

• Contradict one of your hunches about what makes Web sites credible
• Generate a discussion (or argument) about Web credibility
• Lead you to form new hypotheses about Web credibility
• Motivate you to investigate Web credibility
• Change the way you perceive Web sites
• Change how you design Web sites

The remainder of this chapter will describe and integrate the findings of two Web credibility studies with the credibility frameworks outlined in Chapter 6. After laying out the results, I will describe a new framework for Web credibility, one that divides Web elements into three categories: Web site operator, content, and design.

By using the survey method of research to take snapshots of Web credibility perceptions in 1999 and in 2002, we hoped to assess how people felt about the Web at those times and, perhaps, identify trends that were starting to form. Frankly, it’s too early to identify trends with confidence, but as you examine the data and combine it with your own experience, you can begin to form your own hypotheses about what makes Web sites credible and what is changing as the Web evolves along with the people who use it.

Interpreting the Data

In this chapter I present high-level findings from both the 1999 and the 2002 Web credibility surveys. When a study has a large number of responses, as these surveys have, even small differences in means (average scores) can end up being statistically significant. But these small differences may not be large enough to have much practical significance. One of our challenges in interpreting the data, therefore, was to determine which data have practical as well as statistical significance. These are the items I will highlight in this chapter.

(Readers who would like a fuller explanation of how we decided which items have practical significance may consult the endnotes.[22])

[13]You can find the latest Stanford research onWeb credibility at www.webcredibility.org.

[14 ]The Stanford students who joined me in researching Web credibility in 1999 were Preeti Swani, Akshay Rangnekar, Marissa Treinen, Nicholas Fang, John Shon, M.D., Othman Laraki, Chris Varma, Alex Osipovich, JonathanMarshall, and Jyoti Paul.

[15]Conducted during December 1999, this Web credibility research project was supported by Timo Saari and MauriMattsson of Alma Media and Peter Dodd of Nortel Networks. For more information on this and other research we’ve done in this area, visit www.webcredibility.org.

[16]My lab published a different analysis of our 1999 data previously. See B. J. Fogg, J. Marshall, O. Laraki, A. Osipovich, C. Varma, N. Fang, J. Paul, A. Rangnekar, J. Shon, P. Swani, and M. Treinen, What makes a web site credible? A report on a large quantitative study, Proceedings of ACM CHI 2001 Conference on Human Factors in Computing Systems (New York: ACM Press, 2001). http://www.acm.org/pubs/articles/proceedings/chi/365024/p61-fogg/p61-fogg.pdf.

[17 ]We performed the 2002 research with the collaboration of Makovsky & Company, a New York–based communications agency. Mike Sockol was the person at Makovsky who worked with us to implement the study.

[18]This process was based on our Web credibility teams’ opinions. We went through the variables one by one and decided if having data about that item would be important or interesting. For example, to us it seemed the effects of advertising on the Web would be an important area to investigate and document. In contrast, the effects of animated gifs—simple eye candy—did not seem worthy of our time and effort. And we ruled out some elements because a survey as a research method would not be able to shed much light. For example, we ruled out asking people what they thought about Web sites with black backgrounds.

[20 ]Each research method has its own set of weaknesses, including surveys. Surveys don’t always capture how people actually feel or behave. However, we chose the survey method because it was the most efficient means of gathering a significant amount of data to begin building a foundation for future research, including controlled laboratory and contextual field studies with behavioral measures.

[21]Although we have a large sample, we cannot say this sample is representative of all Web users in 1999 or in 2002. We found no defensible way to obtain a true random sample of Web users. However, the fact that we found similar results in the pilots and among participants from both the United States and Finland gave us more confidence in our results. As with any research, readers must interpret findings with care, determining how well the results apply to their own situations.

[22]For a description of how our team arrived at the 0.3 figure, see endnote 22.

Trustworthiness and Expertise on the Web

Web credibility, like the general concept of credibility outlined in Chapter 6, is made up of two dimensions: trustworthiness and expertise (Figure 7.4). When a Web site conveys both qualities, people will find it credible. When it lacks one of these qualities, credibility will suffer. In essence, when people perceive a site to be unbiased (or biased in a way that fits the user’s worldview) and knowledgeable—factors underlying trustworthiness and expertise—they also will view it as credible. This is the same general formula introduced in Chapter 6.

Figure 7.4: The two dimensions of Web credibility.

Principle of “Real-World Feel”

A Web site will have more credibility if it highlights the people or organization behind the content and services it provides.

Trustworthiness and Web Credibility

People rely on perceptions of trustworthiness in evaluatingWeb credibility. Table 7.2 shows the trustworthiness-related elements from the Stanford studies that were reported to boost credibility the most. As Table 7.2 shows, elements that allow people to contact a Web site source increase perceptions of Web credibility. And the more direct the contact, the better. Listing an organization’s physical address boosts credibility dramatically. Listing a phone number also has a major impact. These two elements rank significantly higher than listing an email address. [23 ]Why? The study data do not provide an exact answer, but it seems these elements may show more clearly that real people are behind the Web site—people who can be reached for questions, comments, or complaints. A site that opens itself to direct contact from Web users shows confidence that its information and services are fair, unbiased, and honest.

Table 7.2 also illustrates that Web credibility increases when a site allows people to verify the information presented, including articles with citations and references, as well as links to outside sources. Providing a means for outside verification also shows confidence, as people can easily determine if the information is biased, by checking references or clicking to see what other sources have to say.

Principle of Easy Verifiability

Credibility perceptions will be enhanced if a Web site makes it easy for users to check outside sources to verify the accuracy of site content.

Elements that Increase Credibility: Significant Changes in 2002 Results

When comparing the results of the 1999 and 2002 studies, three of the items in Table 7.2, when taken together, may have practical significance. These items (in boldface text) have to do with contact information: physical and email addresses and phone number. These items had a greater impact on credibility in 1999 than in 2002. It may be that by the time of the 2002 study, people came to view these contact elements as expected; they were less impressed than in 1999 if a site posted this information.

The score of one item in Table 7.2, regarding articles containing citations and references, dropped significantly between 1999 and 2002. A partial explanation is that country of origin made a large difference. Between 1999 and 2002, the average scores of Finnish participants on this item declined much more significantly than those of U.S. participants (these details are not shown in the table). It’s important to note that in both studies, citations and references made a significant positive impact on credibility evaluations. What’s not yet clear is why the Finns became much less impressed with citations and references in the years between 1999 and 2002, compared to the U.S. participants. Future research (or a deeper exploration of differences in culture, including Internet culture) may provide a clearer answer.

Table 7.2 shows elements related to trustworthiness that boost Web credibility. Our research also uncovered a number of elements related to trustworthiness that decrease credibility perceptions. These are listed in Table 7.3.

As Table 7.3 shows, credibility suffers most when Web sites make it difficult to distinguish between ads and content. This takes a toll on perceived trustworthiness, probably because people feel the site is designed to fool them into clicking on the ads or at least viewing ads as part of the site’s information.

People know that advertisements are prone to bias. When there is no clear distinction between the two, people may view the entire site as biased.

Table 7.3 also illustrates that Web sites are judged by the company they keep. A Web site’s credibility suffers when it links to another site that is perceived to lack credibility, as a link may imply an endorsement.

The next trustworthiness element has to do with advertisements. Our studies showed that ads that pop up in new browser windows hurt a site’s credibility. This makes sense; people usually go to Web sites with goals in mind, and they expect the site to help them accomplish those goals. Pop-up ads are a distraction and a clear sign the site is not designed to help users as much as possible. Pop-up ads make people feel used, perhaps even betrayed. (This is not to say that all ads hurt the credibility of a Web site. In some cases they can increase credibility perceptions, if they match the content of the page and come from reputable sources.[24])

The last element that hurts credibility is using a domain name that doesn’t match the company name. Last year I set out on the Web to reserve a campsite at Sunset State Beach near Santa Cruz. This is a park I’ve enjoyed before, supported by tax dollars and run by camp rangers. I was somewhat puzzled when I found out the Web site that handles these reservations for the state beach is “ReserveAmerica.com,” not a park Web site or .gov site. It made me wonder if I was really working with the right people and if the information I was getting on camp site availability and price was accurate.

When it comes to companies, the same concept applies. If you’re seeking information about ocean kayaks from a company called Wave Rider International and its Web site is PaddleAwayNow.com, you’re likely to wonder why it doesn’t have a URL similar to its company name. Has the company been acquired? Is it going out of business?

It’s likely that respondents in our studies sensed something suspicious about a company that didn’t operate under its own name. There may be other explanations as well. But one thing seems clear: to sustain credibility, a company should use a domain name that matches its company name as closely as possible. Credibility increases when people can identify the site operator, its values, and its motives. [25 ]Hidden identities or motivations make people wary.

Elements that Decrease Credibility: Significant Changes in 2002 Results

The items in Table 7.3 show some significant differences between the 1999 and the 2002 studies (highlighted in boldface). The most notable change is the increased damage that pop-up ads have on perceived credibility. It’s likely that, as these types of ads have become more common, people are becoming more annoyed with them, viewing Web site operators that allow pop-up ads on their site as unconcerned about what attracted people to their site in the first place. Pop-up ads are a distraction, and the data suggest that they are doing significantly more damage to credibility in 2002 than they were in 1999.

Another significant difference between the years has to do with sites linking to other sites that are not credible. This is another change in results for which we have no satisfying explanation. Perhaps people are beginning to judge Web site credibility more by the merits of the particular site than the company the site keeps. Despite the change in scores between the two studies, the overall message for designers remains the same: to create a highly credible Web site, do not link to sites that are not credible.

Expertise and Web Site Credibility

Trustworthiness is only part of the Web credibility equation. In the Stanford Web Credibility studies, about half the elements that ranked highest in promoting Web credibility dealt with issues of expertise, including expertise in running a Web site efficiently and reliably, as indicated by the elements shown in Table 7.4.

The top two elements of expertise—responding to customer service questions and confirming transactions via email—imply that the Web technology functions properly and, perhaps more important, that the site operator is expert at being responsive to customers. Listing the authors’ credentials for each article is another form of expertise in the form of knowledge. Although not always done on Web sites, the simple act of listing credential information can boost credibility considerably. [26 ]

Principle of Fulfillment

A Web site will have increased credibility when it fulfills users’ positive expectations.

Technical and design expertise also emerge as elements that strongly enhance Web credibility. Having a search feature suggests the site operator is adept with the technical elements of Web design. This demonstration of technical expertise may create a halo effect, making the entire Web site seem more credible.

The visual design of the site is equally important; a professional-looking design boosts perceptions of Web credibility. Finally, updating content demonstrates that the site operator has been able to coordinate the many things that must happen to keep the site updated—another indication of expertise.

Elements that Increase Credibility: Significant Changes in 2002 Results

Table 7.4 shows significant practical differences in average scores of three items (shown in boldface) between 1999 and 2002.27 The first two items—quick response in customer service and sending email confirmations—seem to go hand in hand. While these attributes still boost credibility perceptions significantly in 2002, they don’t have as great an impact as they did in 1999. It seems reasonable to speculate that by 2002, many more site operators had figured out how to run effective and responsive sites. It became common, even expected, to receive confirmation emails when buying products and services online. Also, by 2002 people may have been less impressed by companies that responded quickly to customer questions, as this, too, has become a standard expectation. Again, in pointing out these slight drops in credibility scores between the two years, it must be noted that providing responsive service is one of the best ways to boost the credibility of a Web site.

The final item in Table 7.4 that shows a significant difference between years deals with how people viewed sites that listed author credentials. Here, too, the scores of Finnish participants declined far more than those of U.S. participants between 1999 and 2002 (these detailed findings are not shown in the table). The fact that major differences in scores between the United States and Finland occurred in two items suggests an opportunity for further research into cultural differences.

Web sites also can demonstrate a lack of expertise, causing people to doubt their credibility. Table 7.5 shows four items from the Stanford studies that relate to lapses in expertise and that result in reduced credibility. According to our studies, people viewed sites with outdated content as lacking in credibility. Ensuring that a site has the latest information takes dedication and skill. Sources that don’t update their sites are probably viewed as not serious about the sites or simply negligent, both of which hurt expertise perceptions.

Some forms of Web content are more time-sensitive than others. In an online experiment my lab is currently running, we’re investigating how sensitive different types of Web content are to being out of date. In planning a vacation to the Philippines (the task in our study), we hypothesize that a Web site will lose credibility if the travel advisory posted is from 1997 compared to a travel advisory from five days ago. In contrast, we are hypothesizing that information about the cuisine of the Philippines dated 1997 will have significantly less impact on the perceived credibility of the site. [28 ]

Even minor problems with a Web site can have a major impact on users’ perceptions of the site’s expertise and, therefore, its credibility. Participants in our studies reported that Web sites lose a great deal of credibility when a link doesn’t work or when the site has a typographical error. [29 ]

Technical difficulties also affect the perceived credibility of a Web site. People expect the Web to be available around the clock except, in some cases, for scheduled and announced maintenance periods. When a site goes down (as some notable sites have in the past few years), people view this as a credibility issue, even though the fault may have nothing to do with the site operator. [30 ]

Elements that Decrease Credibility: No Significant Changes in 2002

None of the items in Table 7.5 differed from 1999 to 2002 to a degree that is practically significant. Why? Perhaps these items have been minimum expectations for Web sites since 1999, and these expectations had not changed by 2002. People still perceive sites to have less credibility when they are not updated, when a link doesn’t work, and so on.

In summary, these study results show that when it comes toWeb credibility, expertise matters. Demonstrations of expertise win credibility points, while a lack of expertise has the opposite effect.

[23 ]We had more questions in the pilot studies, sometimes as many as 90 questions, and we recruited a different sample: in one pilot, the sample was HCI specialists; in another, it was our friends and family whom we could convince to test the survey.

The purpose of the pilot studies was to refine the questions (finding confusing items and making them clearer, narrowing the question set down to a manageable number, which we decided was about 50), and make sure the backend part of the Web site was capturing data correctly. Even though the pilot studies weren’t designed to give us data we’d publish, we still did the statistics to see what we would find.

The items that affected credibility a great deal were roughly the same in all the studies, notwithstanding the different samples and size of the surveys.

[24]Because we have such a large number of respondents, even small differences in means (such as 1.1 versus 1.0) are statistically significant. However, as I point out in the text, statistical significance is one thing; practical significance is another.

Using statistics and rational thinking as our guides, in analyzing the data for practical significance, my team and I decided to drawn the line at 0.3. This means that, within a given study, if any two means are more than 0.3 points apart (when including the entire data set for the year), those two items do indeed have practical impacts on perceived credibility.

Comparing items between the 1999 and 2002 studies, we chose to be more conservative, since the samples—although close—are not identical. Also, because the survey took place at a different time in history, there may be other factors, such as a news story about corruption in the United Way, that had a temporary impact on how people think about charity groups, including their presence on the Web. In the end, we chose the more conservative figure of 0.4 (when using the entire data set) to be the necessary difference between mean scores to be an interesting (practical) difference.

Therefore, in the discussion of the data in this chapter, I point out the differences between data sets that are at 0.4 or higher and suggest reasons for this change. Of course, there is no way to prove that my explanations are accurate, but I think you’ll find my comments to be reasonable. There are a couple items for which I cannot explain the difference. As a lab, we’ve looked over these items from various angles, and we’re still scratching our heads. But that’s how research usually proceeds: questions remain unanswered— some of which may be resolved in future studies.

[25 ]As mentioned in the previous endnote, we determined that any two means that are at least 0.3 apart have a significantly different (in practical terms, not just statistical terms) impact on people’s perceptions of Web credibility.

[26 ]In 1999, I advised a Stanford honors thesis that showed, contrary to our expectations, that ads on Web pages increased the credibility of the Web page. We didn’t expect this. But after reexamining the data and the pages, we realized that the banner ads from reputable companies—in this case, including Yahoo and Visa—could endow Web sites with credibility, making an impression on users: if Visa cares enough about this Web site to advertise, it must be a credible site. Thesis information: N. Kim, World Wide Web Credibility: What Effect Do Advertisements and Typos Have on the Perceived Credibility of Web Page Information? Senior honors thesis, Stanford University (1999).

[28 ]Making the motives and policies of Web sites transparent is a major focus for Consumers Union Consumer Webwatch Project, for which I serve as an adviser. See www.consumerwebwatch.org for more.

[29 ]Almost all guidelines for evaluating the credibility of Web information advocate checking on the author’s credentials. However, I can find no other quantitative evidence, besides my lab’s research, that shows that listing credentials will boost credibility. Perhaps other researchers have found this question too obvious to merit a controlled study.

[30 ]In 2002, we dropped the item regarding searching archives because it seemed this feature was no longer prominent on the Web. Instead, we replaced this item with the following: “The site has search capabilities” (mean in 2002: +1.2).

The Four Types of Web Credibility

The four types of computer credibility described in Chapter 6—presumed, reputed, surface, and earned—also apply to users’ experiences on the Web. Table 7.6 reviews each type of credibility and lists a Web-related example for each.

Presumed Credibility on the Web

Presumed credibility describes the extent to which a person believes something because of general assumptions. These assumptions help people evaluate—rightly or wrongly—the credibility of Web sites.

Many elements from the Stanford Web Credibility Studies relate to presumed credibility. The four key elements that boost presumed credibility are shown in Table 7.7. (Our studies had no items that decreased presumed credibility to any practically significant extent.) First, the results show that sites are seen as more credible if they “represent a nonprofit organization” or have a URL that ends with “.org,” which many people associate with nonprofit organizations. [31 ]

Many people assume that, because nonprofit organizations are not seeking commercial gain, they are more likely to be trustworthy. This assumption is not always based in fact, but it does have an effect when it comes to perceived Web credibility.

Why the significant difference in scores for these two elements between 1999 and 2002? Clearly, nonprofits have lost some of their luster on the Web. The reason may have to do with people realizing that most anyone can set up a site that appears to be a nonprofit organization. As people gain more experience with the Web, they also are developing a healthy skepticism—things are not always what they appear to be, even with nonprofits. (Despite this decline in scores, it’s important to note that Web sites for nonprofits and those ending with “.org” still are evaluated positively in regard to credibility.)

Another key finding: A site is more credible if it “links to its competitors’ sites.” The likely assumption behind this finding is that companies that give users all the facts, including those not under their control, are being totally honest. By helping people access information—even information that may not be in the source’s best interest—the source will be viewed as highly trustworthy. [32 ]

The last key element that contributes to presumed credibility is having a site that “says it is the official site for a specific topic.” [33 ]The assumption is that official sites have more expertise on a given topic than a site that isn’t official.

Official sites also may be assumed to be more trustworthy because if they were not fair or honest, they would lose their official status. (Of course, Web users still need to determine if the site that declares itself to be the “official” site is telling the truth.)

Reputed Credibility on the Web

People’s perceptions of Web credibility often center on reputed credibility, a form of credibility that’s based on the endorsement or recommendation of a third party. On the Web, reputed credibility shows up in the form of awards, seals of approval, links, and endorsements from friends, among others. The key elements that affect reputed credibility are shown in Table 7.8. (As with presumed credibility, no item in our studies decreased reputed credibility to any practically significant extent.)

Awards

The Web has spawned many endorsements in the form of awards (Figure 7.5). Some sites prominently list the awards they have won—site of the day, a top Web site, teacher approved. Posting awards is a good strategy to play up reputed credibility. Our study results show that they are a key element in enhancing perceptions of Web credibility, but perhaps not as much as we expected. It may be that the people in our sample were experienced enough to know that some awards on the Web are meaningless. This explanation seems especially true when you look at the change from 1999 to 2002. In 2002, participants reported that having received an award boosts perceived credibility of a Web site very little, if at all. Research in future years will help show if this decline was a fleeting result or something more permanent.

Figure 7.5: Awards posted on Web sites can boost credibility, even though some people recognize that Web awards can be “pointless.”

Seals of Approval

Seals of approval (Figure 7.6) are similar to awards. A handful of companies have set out to tame the lawless World Wide Web—or at least to offer users a sense of security. The lack of regulation on the Web has created a market for third-party endorsements. Similar to UL Lab endorsements or Good Housekeeping seals of approval, Web-related endorsements can convey credibility.

Figure 7.6: Various organizations have leveraged the fact that users seek third party endorsements on the Web.

Consider TRUSTe, a nonprofit consumer advocacy group. The trademarked motto for TRUSTe cuts to the heart of credibility: “Building a Web you can believe in.” VeriSign—“the sign of trust on the net”—also aims to give a sense of security to people who make Web transactions. In the health arena, the Health on the Net Foundation has created a set of guidelines for health sites. Sites that display the HON code logo are supposed to follow the policies of the foundation.

Organizations such as TRUSTe, VeriSign, and the Health on the Net Foundation have found they can fill a void. Consumer Reports is currently moving into this space with its Consumer Web watch Project, for which I serve as an adviser. [34 ]Web site operators seek these third-party endorsements, and Web users rely on them to determine the trustworthiness of Web sites.

Links from Credible Sources

Links from other Web sites to a given site also can convey credibility. The results from our studies confirm the impact of these incoming links. The studies showed that Web sites gain a great deal of credibility when “linked to by a site you think is believable.” This effect is intuitive: If you were running a Web site on health information, a single link from a prestigious site such as the Mayo Clinic could dramatically boost your site’s credibility. Through the link from their site, you received an implied endorsement from a prestigious organization.

Word-of-Mouth Referrals

Finally, our study data show that the classic word-of-mouth strategy boosts perceptions of Web credibility. Sites that are “recommended to you by a friend” are perceived as more credible. This is not surprising; in general, you would expect your friends to be trustworthy, looking out for your best interest.

The power of word-of-mouth referrals is not new, but it can take on new forms online. Some Web sites make it easy for you to send an article to a friend. Other sites, such as the community site Yahoo Groups, ask you to compose a personal message to a friend when you invite him or her into your virtual group. These are all referrals, which boost perceptions of site credibility.

The Web offers other types of reputed credibility, such as experts who endorse a site, magazines that favorably review a site’s functionality or content, and search engines that list the site early in the list of matches. Although not part of our studies, these all likely have an impact on the perceived credibility of Web sites.

Surface Credibility on the Web

Presumed and reputed credibility can exist without people experiencing Web sites firsthand; however, the other types of credibility—surface and earned— require direct experience. Of the two, surface credibility is the most common, while earned credibility is the most robust.

Often people use the Web in ways that don’t allow for earned credibility to develop. They surf around, hopping from page to page and site to site, making quick evaluations of Web credibility by browsing through sites. For casual Web surfing, surface credibility matters most and earned credibility matters little, since people are not processing information deeply and are not engaging with a site over an extended period of time.

Design Matters

What conveys the surface experience of a Web site, and how does this affect perceptions of credibility? Tables 7.9 and 7.10 show the key elements.

One key element in surface credibility is visual design. People can quickly take in the design of a site—the colors, the layout, the images, and other design elements. The Stanford Web Credibility Studies show that site design matters a great deal in assessing surface credibility. Sites that look “professionally designed” boost credibility substantially. People apparently use these limited impressions to make an initial assessment of a site’s credibility.

When evaluating credibility, Web surfers also consider how easy a site is to access. Sites that require “a long time to download” take a credibility hit. Also, according to the data in our studies, sites that require “a paid subscription to gain access” lose some credibility. [35 ]

Advertising is another element that affects surface credibility perceptions. People surfing the Web may not read a Web site’s content in detail, but they are likely to get a sense of the advertising on the site. As noted earlier, a site loses a great deal of credibility by making it “hard to distinguish ads from content” or by “automatically pop[ping] up new windows with ads.” As stated earlier, popup ads are more common in 2002 and, therefore, more annoying. It only takes a quick glance to sense the ad density on a page, making this a surface credibility issue. Participants in our research studies reported that sites having “one or more ads on each page” lose credibility.

Enhancing Surface Credibility

Because people can’t absorb all Web site elements at once, Web designers must emphasize those elements that boost surface credibility most. Which elements to emphasize depends on the Web site’s purpose. For example, if a site deals with news, it’s important to quickly show that the site’s information is current. Our studies show that sites gain credibility when they have been “updated since your last visit.” To boost surface credibility, a news Web site could highlight the frequency of its updates.

Another example: If a site deals with health, it’s important to convey the expert sources behind the information. In a recent pilot study asking people to compare the credibility of health Web sites, researchers in my Stanford lab found that participants responded quite positively to InteliHealth.com (Figure 7.7). In reading their comments, we found that one element made InteliHealth seem highly credible: the use of the Harvard Medical School name. Every page of the InteliHealth site contains in the upper left corner a prominent blue oval that says “Featuring Harvard Medical School’s Consumer Health Information” along with an image of the Harvard crest. This is great example of bringing to the surface elements that are likely to boost credibility. The designers at InteliHealth did not hide the Harvard affiliation in a footnote or an “about us” page; the message is featured on each page, taking up valuable screen space but doing an important thing for a health site: clearly establishing credibility.

Figure 7.7: By making the Harvard name prominent, InteliHealth gains surface credibility.

Designing for surface credibility is a balancing act. On the one hand, a site must fill users’ needs for information or services quickly—or at least make a quick promise to fill those needs. A portion of the homepage must be devoted to this. On the other hand, the site must use the homepage to convey surface credibility—by showing a photo of the organization’s headquarters building; listing clients, partners, or experts associated with the site; or including other content that instantly conveys expertise or trustworthiness. Both of these requirements must be met within the limitations of a browser window.

Earned Credibility on the Web

The last type of Web credibility is the gold standard: earned credibility. Especially on the Web, where people surf quickly from site to site, earned credibility is the most difficult type to gain, but it is also the type of credibility that is most likely to lead to attitude and behavior changes. Conveying the three other types ofWeb credibility is useful primarily so that the site will eventually gain earned credibility. When earned credibility is high, people are likely to spend more time at the site, visit it more often, make repeated purchases (if it is an e-commerce site), tell others about it, and be open to persuasion techniques the site uses. Earned credibility is all about establishing an ongoing relationship between a Web user and the Web site or operator. According to our research, a solid, ongoing Web relationship is based on three key attributes of a site: interactions are easy, the site’s information is personalized, and service is responsive (Tables 7.11 and 7.12).

Earned credibility is the most difficult to achieve but the most likely to change attitudes or behaviors.

The Interaction Is Easy

Earned credibility grows from Web interactions that are easy. Our survey results show that Web sites gain credibility when “the site is arranged in a way that makes sense to you.” At the opposite end of the spectrum, a site loses credibility when it “is difficult to navigate.”

Principle of Ease-of-Use

A Web site wins credibility points by being easy to use.

Likability is a stepping stone to credibility. A person who likes someone tends to think the other person is credible as well. [36 ]The same is true for human-computer relationships. [37 ]If a person finds a Web site easy to use, he or she likely will think the site is also credible.

The Information Is Personalized

In our studies, we found that a Web site was perceived to be more credible when it offers personalized information—specifically, when it “recognizes you have been there” and when it selects “news stories according to your preferences.” Even ads on the Web become slight bonuses when they are personalized; the study data show that a site gains credibility when the “ads match the topic you are reading about.”

Principle of Personalization

Web sites that offer personalized content and services get a boost in credibility.

Personalization seems to enhance credibility in two ways. First, when a Web site has the capability to tailor content, people will view the site as smarter, boosting expertise perceptions. Next, tailored experiences can make people think the Web site understands their preferences and is working to help them achieve their goals. Unless the tailoring is done poorly (e.g., raising concerns about privacy by clumsily showing how much the Web site really knows about you38), users are likely to perceive the site—and the people behind it—as trustworthy. For instance, people who use sites such as my.yahoo.com or myschwab.com may view the personalized versions of these Web sites as more trustworthy.

The Service Is Responsive to Customer Issues

Finally, Web sites can earn credibility when people find them responsive. Our studies show significant increases in credibility for a Web site that “provides a quick response to your customer service questions.” Participants also evaluated a Web site to be much more credible when it “sends emails confirming transactions you make.” It may be that, in the period between the 1999 study and the 2002 study, people had become more comfortable with online transactions and that the email confirmation was less needed—or more expected.

Principle of Responsiveness

The more responsive to users, the greater the perceived credibility of a Web site.

Earned credibility should be a key goal for creators of persuasive Web sites. The study data highlight some specific design issues for achieving this: make Web sites easy to use, personalized, and responsive. These three factors will do more than perhaps anything else to boost credibility perceptions that make a difference.

[31 ]When this study is complete, the results will be posted online at www.webcredibility.org.

[32 ]In 1999, I advised a Stanford honors thesis investigating Web credibility. The results from this controlled laboratory study also showed that simple typos have clear negative effects on Web credibility. The more important the information (e.g., life-depending information), the stronger the negative effect. Thesis information: N. Kim, World Wide Web Credibility: What Effect Do Advertisements and Typos Have on the Perceived Credibility of Web Page Information? Senior honors thesis, Stanford University (1999).

[33 ]In recent years hackers have brought down Web sites for titans like Yahoo, the New York Times, and eBay. This can be costly, in terms of lost dollars and lost credibility. For example, when eBaywent down for 22 hours in 1999, not only did it cost the company $3.9 million in credits given to users and a 9% drop in share price, media reports suggested that the downtime also cost eBay credibility among its users. See the following:

http://special.northernlight.com/ecommerce/always_there.htm

http://www.gilian.com/webassets.pdf

http://www.webtechniques.com/archives/2000/03/plat/

[34 ]You don’t need to be a nonprofit organization to register a domain name with an “.org” ending. Anyone can purchase these domain names, but many people don’t know this.

[35 ]E. Walster, E. Aronson, and D. Abrahams, On increasing the persuasiveness of a low prestige communicator, Journal of Experimental Social Psychology, 2: 325–342 (1966).

See also S. Chaiken and D. Maheswaran, Heuristic processing can bias systematic processing: Effects of source credibility, argument ambiguity, and task importance on attitude judgment, Journal of Personality and Social Psychology, 66: 460–473 (1994).

[36 ]We deleted this topic in the 2002 survey because we felt it applied to a small number of sites.

[37 ]See www.consumerwebwatch.org.

The Web Credibility Framework

What exactly is the Web? Is it an information system, a place for community gatherings, a space for accessing and using software applications, a huge shopping mall, or a next-generation broadcast medium? The answer seems to change over time. This dynamism has made the Web difficult to study rigorously because what exists today may not exist—or may not be relevant— tomorrow. By the time you plan, execute, and document a careful study, the Web playing field can shift, making your study a historical artifact rather than a useful research step forward.

To promote research and robust understanding of Web credibility, I’ve created a framework that outlines categories of Web site variables. Within this “Web Credibility Framework” (Table 7.13), a Web site can be described by three categories: the operator of the site, the content of the site, and the site’s design. These categories are helpful in sorting out the many issues relating to Web credibility. For example, if you’re a designer, you’ll find that only certain issues are under your control, while other issues, such as content, may not be.

The main purpose of the framework is to provide an orderly way of thinking about the many elements of a Web site, which in turn provides a systematic way to think about researching or designing for Web credibility. When my Stanford team set out to study Web credibility, we used this framework to ensure that we were covering a wide variety of elements, since our goal was to create a foundation for focused research and design efforts.

The Web Credibility Grid

The Web Credibility Framework can be extended to another level when it is combined with the four types of credibility described in Chapter 6—presumed, reputed, surface, and earned. These two perspectives can be integrated to create a grid (Table 7.14) that captures many elements of a Web experience.

The Web Credibility Grid illustrates that the study of Web credibility has many facets, from brand perception of a company to the technical details of a Web site. Some elements that contribute to Web credibility are under the direct control of Web designers; others are not. As a result, increasing the credibility of a company’s Web site requires a concerted effort from many parts of the organization.

Designers can use the grid to identify the cells they have control over, then focus on designing to boost credibility in those areas, such as making the site easy to navigate. Marketing and legal departments could use it to ensure credibility of content. Public relations personnel may focus on enhancing perceptions of the company’s overall brand. Fulfillment or customer service personnel could concentrate on strengthening earned credibility by delivering fast responses. The grid provides a way for the entire organization to gain a better understanding of how its parts must work together to create and maintain a credible Web site.

The Future of Web Credibility Research and Design

Because academia and industry lack a deep understanding of Web credibility, many research and design explorations are waiting to be performed, with rich insights waiting to emerge. Like the study of persuasive technologies, the study of Web credibility is mostly uncharted territory, offering opportunities for those interested in creating new knowledge.

Web credibility can be difficult to study, not only because there are so many facets to a Web site but because of a host of factors external to the Web. One external factor is users—how they process information, how they value individual components of credibility, and their differing goals when using the Web, as well as contexts in which they do so. All of these factors can vary significantly from one person to the next, and all influence credibility perceptions.

The study of Web credibility also faces the “moving target” problem. [39 ]For Web credibility researchers, there are three significant moving targets: the Web user base, user experience levels, and Web technology itself. As these variables change and evolve, research done in the past may no longer apply or may not be as useful.

In addition to the challenge of individual differences, researchers must be aware that an individual user may have different goals at different times when using the Web, and these goals impact credibility perceptions. At one time, the user may seek specific information on the Web, such as mutual fund performance figures. At other times, they may be using the Web to catch up on the news. In both cases, site credibility matters a great deal. At other times a user might go to the Web just to pass the time, in which case credibility is not likely to matter much.

All of these factors complicate research efforts. Nonetheless, in assessing credibility, users will focus on the same basic factors: trustworthiness and expertise. In addition, the different types of credibility—presumed, reputed, surface, and earned—are likely to remain constant. For researchers, these constants will help to guide future research efforts. For designers, the need to create credible computing products won’t change; in fact, it is likely to grow as people become more experienced users.

People will spend more time on a site they find credible; they’ll subscribe to its services, buy its products, use its chat rooms, tell their friends about it. When a site is designed with credibility in mind, it will be well positioned to change users’ attitudes or behaviors. Credibility makes persuasion possible.

For updates on the topics presented in this chapter, visit www.persuasivetech.info.

[39 ]This finding surprised us in 1999. We expected people to think more highly of paid sites. Perhaps participants simply responded negatively to the idea of paying for anything.