XI User Satisfactionwith Web Portals An Empirical Study

Li Xiao

The George Washington University, USA

Subhasish Dasgupta

The George Washington University, USA

Abstract

A Web portal is a site that aggregates information from multiple sources on the World Wide Web and organizes this material in an easy user-friendly manner. Portals usually consist of a search engine, e-mail, news, and interactive chat facilities. The two main types of portals are horizontal and vertical portals. A horizontal portal is a Web site that provides consumers access to a number of different sites in terms of content and functionality. A vertical portal focuses on a specific community of users who share a common interest. In this chapter we investigate the impact of user characteristics such as gender, age, experience, and Web use on user satisfaction with Web portals. In our study we are unable to detect any differences in satisfaction based on gender. We find that the users with seven to 10 years of work experience are most satisfied with Web portals, while users with more than 10 years work experience report least satisfaction with Web portals. We also find that users that use Web portals the least (30 to 60 minutes per week) are least satisfied with them. Users of vertical (or customer community) portals are always more satisfied than users of horizontal (or mega) portals.

Introduction

Advances in new information technologies and development of the Internet and World Wide Web in the 1990s have changed the computing environment for both individuals and businesses. Since its birth in the 1990s we have been witness to widespread use of the World Wide Web for commercial purposes. Companies have moved to the Internet arena to connect with their customers, provide support, and sell products and services. The number of Web sites increased from a paltry 130 in 1993 (Coopee, 2000) to well over 34 million in 2003[1], and this number is growing by the day.

This explosion of Web sites and information available on the Web gave birth to a new type of Web application or site: a search engine. Web search engines are sites that can search the Web and provide valuable links that meet certain criteria. These search engines, such as, Yahoo!, Lycos and Excite, to name a few, over a period of time morphed into Web portals. They are not restricted to search engines anymore; they provide e-mail, calendar, games and collaborative (chat) facilities. Web portals now constitute some of the most visited sites on the Web.

A portal is defined in different ways. Isaacs (1999) defines a portal to be “a single, browser-based approach to finding useful information — independently of where the information comes from and how it is created.” A portal is also described as a Web “supersite” with a collection of links to popular Web services on the Internet[2]. Most portals provide an aggregation of information from a number of different sources, such as, e-mail, calendar, stock information, and news. They also allow users the ability to personalize and customize the portal. Personalization is the process of tailoring pages to individual users’ characteristics or preferences[3]. Customization is the ability to allow the user to receive personal information from the site. So, to summarize, portals provide individual users personalized and customized access to a variety of information on the Web.

The phenomenal increase of Web sites on the Internet has resulted in increasing pressure on them to succeed. In order to survive and prosper in a highly competitive environment, companies continue to allocate significant resources for the development and maintenance of Web-based systems in the hope of attracting more online customers. User satisfaction is an important predictor of online consumer behavior and the success of a Web-based system. If customers are satisfied with the services received through the online system, it is likely they will keep using the system. On the other hand, if customers get frustrated and dissatisfied with the online system, they would be very unlikely to come back for a visit.

The purpose of this research is to evaluate the success of Web portals. To evaluate its success, our approach is similar to what other researchers have done in the area of IS success — we use user satisfaction as a surrogate measure for success of the system. This evaluation will be a significant contribution to research in the area, and provide us with a better understanding of online consumer behavior.

The organization of the chapter is as follows. In the next section we provide a theoretical framework in which we also include review of the literature. Then, we present our research model and hypotheses. In the next section, we report on our research methodology. We then provide results, followed by a discussion and conclusion section.

[1]Who is Source http://www.whois.sc/internet-statistics/

[2]What is a portal anyway? http://www.acs.utah.edu/acs/news/portals.html

[3]Whatis.com http://whatis.com

Theoretical Framework

The objective of this chapter is to evaluate user satisfaction with Web portals. To achieve this goal, we scan the information systems literature for research on portals and other factors that influence satisfaction. Few studies have investigated this issue. So, in this section we review information systems literature on three separate topics that are of interest in this research: (1) Web portals, (2) user satisfaction with information systems, and (3) factors that influence user interaction with information systems. While we have discussed the importance of Web portals and user satisfaction in this study, we have to understand that this interaction does not take place in a vacuum. A number of other factors, such as user age, education, and experience can influence satisfaction with Web portals. In the third sub-section (factors that influence user interaction with information systems), we will identify other factors that are important to our study. In this sub-section we also propose our research hypotheses. We now consider each of these topics separately.

Web Portals

Web portals have been defined in a number of different ways. A portal is described as a Web “supersite” with a collection of links to popular Web services on the Internet[4]. A portal is also defined as “a term, generally synonymous with gateway, for a World Wide Web site that is or proposes to be a major starting site for users when they get connected to the Web or that users tend to visit as an anchor site.”[5]

Portals can be classified into three types based on the functions they perform: horizontal portals, vertical portals and enterprise portals. A Horizontal portal is a Web site that provides consumers with access to a number of different sites in terms of content and functionality. This portal is generally a consumer portal that can be personalized and customized by the user. Horizontal portals are also called general public portals or mega portals. My Yahoo! and My Excite are examples of horizontal portals. A Vertical portal or vortal focuses on a specific community of users (Isaacs, 1999). This portal is geared towards a narrow audience or a community with specific interests, such as, consumer goods, computers, retail, brokerage services, and banking. An enterprise portal (also called a corporate portal) provides access to proprietary internal information within a company via a company intranet and access to selected Internet sites. For example, employees can have access to their pay stubs or retirement contributions through an enterprise portal. There are other specialized portals based on technology or location that have been identified in information systems research. Examples of such portals are: WAP portals that use mobile technology, and embedded or appliance portals where the portal is embedded in an appliance such as WebTV and OnStar in vehicles.

In this research we restrict our discussion to the two major types of Web portals: horizontal portals and vertical portals. Although we find a number of papers in the information systems literature that classified Web portals, we fail to identify research that investigated use of Web portals. In the next section we review the literature on user satisfaction with information systems.

User Satisfaction

User satisfaction has received considerable attention of researchers since the 1980s as an important surrogate measure of information systems success (Aladwani and Palvia, 2002; Aladwani, 2002; Bailey and Pearson, 1983; Goodhue and Thompson, 1995). While most user satisfaction instruments were developed for transaction processing and traditional systems that were not Web-based at the time of development, some instruments have been successfully validated in an Internet-based environment. This validation is important if we assume that the Internet provides a unique environment that makes a user’s experience considerably different from that of a traditional information system. In the paragraphs that follow we provide a review of literature on user satisfaction primarily from a measurement perspective.

The user satisfaction measurement literature has been dominated by two scale development studies, one by Bailey and Pearson (1983) and the other by Doll and Torkzadeh (1988). Bailey and Pearson first identified 36 factors that could influence satisfaction from a survey of 22 studies done earlier. They put special emphasis on the dimensionality and intensity of every individual’s reaction to each of the factors identified. They used a semantic differential technique with four bipolar adjective pairs and a 7-interval scale. They explained this technique by an example, “… the meaning of “format of output” (a factor) could be measured between the pairs: good vs. bad, simple vs. complex, readable vs. unreadable, and useful vs. useless (Bailey and Pearson, 1983; Aladwani, 2002). Two additional scales were added for each factor: (1) satisfactory vs. unsatisfactory and (2) important vs. unimportant. Validity and reliability of this instrument was checked using a sample of 32 middle managers (these were managers who had previously been interviewed in the factor-identification phase).

A review of existing literature shows that applications of the Bailey and Pearson user satisfaction and the revised User Information Satisfaction scale have been limited. One of the reasons cited in other research is that the scale was a semantic differential rather than a Likert-type scale.

Doll and Torkzadeh (1988) designed and validated a 12-item instrument to measure enduser satisfaction called the End User Computing Satisfaction, or EUCS, instrument. They proposed a second order factor model of end-user computing satisfaction, which consisted of five first-order factors: content, accuracy, format, ease of use and timeliness. These factors were measured using 12 items. The second-order factor was interpreted as end-user satisfaction. In their instrument, Doll and Torkzadeh added two global measures of perceived overall satisfaction and success to serve as a criterion. Their instrument was regarded as comprehensive because they reviewed previous work on user satisfaction in their search for a comprehensive list of items. The end-user satisfaction construct was developed with a five point Likert-type scale (1 = almost never; 2 = some of the time; 3 = about half of the time; 4 = most of the time; and 5 = almost always). In their original study, Doll and Torkzadeh, validated this instrument using end user data from 44 organizations. A number of studies have confirmed the validity of this instrument (Doll and Torkzadeh, 1991; Doll et al., 1994; McHaney and Cronan, 1998; McHaney et al., 1999; Chen et al., 2000; McHaney et al., 2002).

We use Doll and Torkzadeh’s satisfaction instrument and the overall satisfaction criteria for our research. We choose this measure for our study because it is a validated instrument that has been widely used in information systems research.

Factors Influencing User Interaction

Based on our review of literature in the area of Web portals, user satisfaction and factors influencing user interaction with information system, we identify that demographic factors such as gender, age, experience and use of Web portals will influence an individual’s satisfaction with the system. This essentially means that gender, age, experience and use of Web portals are independent variables in our research model, and satisfaction with the Web portal is our single dependent variable. In the paragraphs that follow we provide evidence from the information systems literature to support our choice of variables and propose hypotheses for each combination of independent and dependent variables.

A number of factors influence user interaction with computer systems. In a study of knowledge and information workers in a university setting, Harrison and Rainer (1992) investigated the role of individual differences in skill on end-user computing. They found that factors such as male gender, lower age, more experience with computers, less anxiety towards computers, higher levels of confidence in using computers, lower math anxiety, and creative cognitive style contribute towards higher computer skills. Ford et al. (2001) in a similar study investigated individual differences in Internet searching. They looked at cognitive styles, levels of prior experience, Internet perceptions, age and gender. They found that effectiveness in Web searching is related to male gender, low cognitive complexity, and cognitive style. Males having higher computer skills can also be due to the finding that males are more likely to have relevant computer literacy and programming courses than females (Foster et al., 2003). Although men are found to have a better educational background and computer skills, Spennemann and Atkinson (2003) found that female students tend to use e-mail more than males. Simon (2001) investigated satisfaction with Web sites on a global scale. This study focused on perception and satisfaction levels of subjects from different cultural backgrounds on four Web sites. The research found differences among different culture clusters, and between males and females in certain cultures.

So far we have identified research that has investigated the role of gender on user satisfaction. We now turn our attention to the type of portal. The type of portal is important since there is a difference between horizontal and vertical portals. While horizontal portals provide personalization and customization capabilities, these mega portals do not offer a sense of community or shared common interest that is the focus of vertical portals. Therefore, we expect user satisfaction to be affected by gender of the individual and the type of portal. Moreover, if we control for type of portal, we propose that there will be a difference in satisfaction between the genders. Similarly, we can expect difference in satisfaction based on the type of portal used if we control for gender. Therefore, we propose our hypotheses as follows:

Hypothesis 1: Satisfaction with a Web portal will be affected by gender and the type of portal used.

Hypothesis 1A: Controlling for type of portal used, there will be a difference in satisfaction of males and females with the portal.

Hypothesis 1B: Controlling for gender, there will be a difference in satisfaction based on the type of portal used.

Age of the user is also an important factor in computer interaction. Couper and Rowe (1996) examined respondent reaction to a self-administered component of a computer-assisted personal interview survey. They found that respondents’ age, education and computer experience influence their decision to self-complete the computer-assisted self-interview items. As mentioned earlier, Ford et al. (2001) also investigated the role of age in the retrieval effectiveness of information from a search engine. Therefore, we state our age-related hypotheses as follows:

Hypothesis 2: Satisfaction with a Web portal will be affected by age and the type of portal used.

Hypothesis 2A: Controlling for type of portal used, there will be a difference in satisfaction of various age groups with the portal.

Hypothesis 2B: Controlling for age, there will be a difference in satisfaction based on the type of portal used.

Experience is another factor that has been studied in the area of interaction with computer systems. In a study of the relationships between user satisfaction, job satisfaction changes, user’s behavior and demographics in an Internet-oriented workplace, Simmers and Anandarajan (2001) found that individuals with more training and experience have higher levels of user satisfaction. But, they did not find differences in the user satisfaction between men and women. We state our hypotheses as follows:

Hypothesis 3: Satisfaction with a Web portal will be affected by experience and the type of portal used.

Hypothesis 3A: Controlling for type of portal used, there will be a difference in satisfaction among different experience levels.

Hypothesis 3B: Controlling for experience, there will be a difference in satisfaction based on the type of portal used.

Next we propose that the extent of use of the Internet or Web and type of portal will impact satisfaction with portals. Our justification is as follows: as individuals spend more time using the Internet or the World Wide Web, they will have an increasing need to use a portal. A portal will provide a user with the ability to organize and personalize the information they have or need from the Web. For example, a user may use the Web to access credit information on a number of their credit cards. Rather than visiting different bank sites to access credit card balance and payment information, the user may provide all login information to a portal site, and this site may aggregate balance and payment information from multiple sites and present it to the user on one page. Therefore, we feel that as a user uses the Web more, the better informed or educated the person will be, and this will affect the individual’s satisfaction with a Web portal. Therefore, we state our hypotheses as follows:

Hypothesis 4: Satisfaction with a Web portal will be affected by the extent of Web use and the type of portal.

Hypothesis 4A: Controlling for type of portal used, there will be a difference in satisfaction based on the extent of Web use.

Hypothesis 4B: Controlling for Web use, there will be a difference in satisfaction based on the type of portal.

In summary, we have identified four factors that influence user satisfaction with Web portals. We have proposed research hypotheses for each of these factors. In the next section we report on the methodology used in this study.

[4]What is a portal anyway? http://www.acs.utah.edu/acs/news/portals.html

[5]Whatis.com http://whatis.com

Methodology

Sample

The research sample used in this study included 340 students at a large mid-Atlantic university in the USA. The sample size is comparable to similar studies on user satisfaction such as McHaney and Cronan (1998) and McHaney et al. (2002). Although the sample for this study was collected at a higher education institution, it represents the end user population well, because some students were full-time students while others were part-time from a variety of industries and management levels. Table 11-1 provides sample descriptive statistics. As the table indicates, the subjects are distributed evenly in male (52%) and female (48%). As for work experience, about 75% of the subjects have a few years of experience (35% + 41%), while 25% had more than three years of experience. Most of the subjects use computers at home (86%) and school (77%), and close to one half (41%) of the subjects use computers at work. Some subjects use computers at multiple places.

Data Collection

The data for this study were collected in 2002 as part of a larger research study on user satisfaction with Web-based information systems. A survey instrument was pilot tested with 30 graduate students and the instrument proved to be valid and reliable. The research questionnaire was then distributed in class to 340 graduate and undergraduate students. While collecting the data, students were informed that participation in this study was voluntary and anonymous. No personal identity information was collected in the survey. Hard copy questionnaires were distributed in class by the researcher. It took about 10 minutes to complete the survey.

Analysis

Analysis of variance (ANOVA) was performed to test the four hypotheses. A two-way ANOVA was used to test that the mean scores for user satisfaction (the dependent variable) varied by two factors (independent variables). For example, to test hypothesis H1, an ANOVA was run with user satisfaction with Web portals as the dependent variable, and gender and portal type as the independent variables. This was to test if the mean on user satisfaction with Web portals was equal for each category in the combinations of the two independent variables. Using the same test, we also tested the controlling effect of each independent variable in the pair of two (H1a and H1b). For example, in H1a, we tested that after controlling for the factor of portal type used, the means for user satisfaction varied by gender, or male and female users are satisfied to different extent. Table 11-2 shows the F values and significance level for each hypothesis.

We also used least squares means comparisons to identify which groups were significantly different from each other if a significant F value for the entire model was obtained. We used the T-test for significance of the tests, which allows for all possible combinations of group means to be tested. For example, Table 11-3a lists the least squares means of each category of work experience, which is after controlling for the factor of portal type used. And it lists the t-value and significance of each comparison of means between all different groups. The least squares means and comparison T-tests are shown in Table 11-3a, 11-3b, 11-4a, 11-4b, 11-5 and 11-6.

Results

As indicated in Table 11-2, all four hypotheses H1 to H4 are supported. For example, for H1, the data well supported that user satisfaction with a Web portal will be affected by gender and the type of portal used.

The two-way ANOVA also tested the controlling effect of either factor in each pair. For example, in H1, H1b suggests that after controlling for the factor of gender, users of horizontal and vertical Web portals have different satisfaction levels. H1a is not supported; after controlling for the factor of portal type used, we cannot tell that male and female users are satisfied to different extent. In the case of H2, portal type is still a significant factor after controlling for age (H2b), while age is not significant after controlling for portal type (H2a). In H3, both of the independent variables work experience and portal type are significant after controlling for the other factor (H3a and H3b). In H4, the two independent variables are also significant after controlling for the other factor (H4a and H4b).

Once it has been determined that differences of means exist in the groups, the least squares means are calculated and T-tests are run for comparisons of means between the groups. The comparison matrices of means in different groups are listed in Table 11-3a, 11-3b, 11-4a, 11-4b, 11-5 and 11-6.

Discussion and Conclusion

The results provide beneficial insights into differences in user satisfaction with Web portals based on behavioral and demographic groups.

From this study, we find out that different levels of work experience resulted in different levels of user satisfaction with Web portals. Based on the matrices, we find out that the users with seven to 10 years work experience are most satisfied with Web portals, while users with more than 10 years work experience report least satisfaction with Web portals. We also find that users that use Web portals the least (30 to 60 minutes per week) are least satisfied with them.

The Web portal type has been the one factor that always results in differences in satisfaction, and users of vertical (or customer community) portals are always more satisfied than users of horizontal (or mega) portals. One of the reasons this may have happened is that vertical portals are sites people go to because they share a common interest with other members or users of the portal. This may provide a kind of attraction to this portal that no other types of portal can provide. The sense of community and affinity towards a vertical portal may be a reason for their users’ high satisfaction scores. Our findings have serious consequences for horizontal or mega portals such as Yahoo! Horizontal portals should devise ways in which members of their portal get a sense of community — a number of portals such as Yahoo! provide chat facilities and games to its members. This can attract users who share a common interest.

There are no significant gender differences in user satisfaction with Web portals. And age doesn’t seem to be an important factor to influence user satisfaction here, either. This supports more recent research in information systems.

Any research has its limitations and this one is no different. The use of students can raise concerns about external validity and generalizability of our findings. Moreover, measures used in this study were perceptual in nature; actual use of portals was not determined. Although we evaluated use of portals, we did not differentiate between work-related and recreational use of these applications or Web sites.

In conclusion, the findings of the present study contribute to a better understanding of user satisfaction with Web portals. It examines behavioral and demographic factors and their impact on user satisfaction with Web portals, which is relevant for both academic research and IT practitioners.

References

Aladwani, A., & Palvia, P. (2002). Developing and validating an instrument for measuring user-perceived Web quality. Information & Management, 39, 467-476.

Aladwani, A. M. (2002). Organizational actions, computer attitudes, and end-user satisfaction in public organizations: An empirical study. Information Resources Management Journal, 14(1), 42-49.

Bailey, J. E., & Pearson, S. W. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29(5), 530-545.

Chen, L.D., Soliman, K. S., Mao, E., & Frolick, M. N. (2000). Measuring user satisfaction with data warehouses: An exploratory study. Information & Management, 37, 103-110.

Coopee, T. (2000). The Internet today. InfoWorld, 22(39), 52.

Couper, M. P., & Rowe, B. (1996). Evaluation of a Computer-assisted Self-interview Component in a Computer-assisted Personal Interview Survey. Public Opinion Quarterly, 60(1), 89.

Doll, W. J., & Torkzadeh, G. (1988). The measurement of end-user computing satisfaction. MIS Quarterly, 12(2), 259-274.

Doll, W. J., & Torkzadeh, G. (1991). The measurement of end-user computing satisfaction: Theoretical and methodological issues. MIS Quarterly, 15(1), 5-10.

Doll, W. J., Xia, W., & Torkzadeh, G. (1994). A confirmatory factor analysis of the enduser computing satisfaction instrument. MIS Quarterly, 18(4), 453-461.

Ford, N., Miller, D., & Moss, N. (2001). The role of individual differences in Internet searching: An empirical study. Journal of the American Society for Information Science and Technology, 52(12), 1049.

Foster, B. P., Karcher, J. N., & Levitan, A. S. (2003). Gender and accountants’ use of technology. The CPA Journal, 73(2), 68.

Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 19(2), 213-236.

Harrison, A. W., & Rainer, R. K. J. (1992). The influence of individual differences on skill in end-user computing. Journal of Management Information Systems, 9(1), 93, 19.

Isaacs, N. (1999). Types of portals. Retrieved from: http://websrv.cs.fsu.edu/~baldauf/projects/cgibin/scavenge/a2/portals.html

McHaney, R., & Cronan, T. P. (1998). Computer simulation success: On the use of the end-user computing satisfaction instrument. A comment. Decision Sciences, 29(2), 525-536.

McHaney, R., Hightower, R., & Pearson, J. (2002). A validation of the end-user computing satisfaction instrument in Taiwan. Information & Management, 39, 503-511.

McHaney, R., Hightower, R., & White, D. (1999). EUCS test-retest reliability in representational model decision support systems. Information & Management, 36, 109-119.

Simmers, C. A., & Anandarajan, M. (2001). User satisfaction in the Internet-anchored workplace: An exploratory study. JITTA: Journal of Information Technology Theory and Application, 3(5), 39 (23 pages).

Simon, S. J. (2001). The impact of culture and gender on Web sites: An empirical study. Database for Advances in Information Systems, 32(1), 18.

Spennemann, D. H. R., & Atkinson, J. S. (2003). A longitudinal study of the uptake of and confidence in using e-mail among parks management students. Campus-Wide Information Systems, 20(2), 55.

Endnotes

1 Who is Source http://www.whois.sc/internet-statistics/

2 What is a portal anyway? http://www.acs.utah.edu/acs/news/portals.html

3 Whatis.com http://whatis.com

4 What is a portal anyway? http://www.acs.utah.edu/acs/news/portals.html

5 Whatis.com http://whatis.com