Content-Type: text/html Credibility of Newspapers and Internet Credibility of Newspapers and Internet Newspapers & the Internet: A Comparative Assessment of News Credibility by Gregg A. Payne 51 Palm Beach Court Monarch Beach, California 92629 (949)496 2104; [log in to unmask] David M. Dozier Afsheen J. Nomai School of Communication, San Diego State University San Diego, CA 92182-4561 Paper submitted to the Mass Communication and Society Division of the Association for Education in Journalism and Mass Communication Annual Convention, Washington, D.C., August 2001. ABSTRACT An experiment examined differences in credibility assigned to news stories read in paper form and an identical story read on a web site. Randomly assigned control and test groups exposed to six identical news stories assessed the credibility of the articles they read, using an established, reliable credibility index. News appearing on a web site was evaluated as less credible for all three categories of news, when compared to paper distribution. However, only two were statistically significant. Credibility judgments differed as a function of news topics. NEWSPAPERS & THE INTERNET: A COMPARATIVE ASSESSMENT OF NEWS CREDIBILITY This study analyzes how and why audiences assign - or fail to assign - credibility to Internet news. Internet news is defined here as the content of web sites on which newspaper stories are posted. The study is motivated, in part, by a priori theorizing and anecdotal evidence suggesting that journalists and public relations practitioners, among others, believe news consumers apply different criteria when evaluating the credibility of Internet-disseminated news. This research addresses two central questions: 1. Do reader judgments of credibility vary across newspapers and the Internet as modes of news dissemination? 2. Does story content type (e.g., business, national, and international news) mediate credibility judgments across these two modes of news dissemination? Review of the Literature An interest in the attributes and dimensions of credibility in mediated communication has generated over the past several decades a substantial academic literature. The research literature has focused largely on the credibility of news disseminated by television and newspapers (Abel & Wirth, 1977; Berlo, Lemert, & Mertz, 1969-70; Burgoon, Burgoon, & Wilkinson, 1981; Carter, & Greenberg, 1965; Gantz, 1981; Gaziano & McGrath, 1985; Gaziano & McGrath, 1986; Hovland & Weiss, 1951; Jacobson, 1969; Johnson & Kaye, 1987; Lee, 1978; Meyer, P.,1988; Meyer, T, 1974; Mulder, 1980; Newhagen & Nass, 1989; Owen & Karrh, 1996); Regan & Zenaty, 1979; Rimmer & Weaver, 1987; Robinson & Kohut, 1988; Roper, 1985; Shaw, 1973). Only some research (e.g., , largely because the medium remains a relatively new conduit for news dissemination, is relative credibility of the Internet, especially when compared to more established media (e.g., Sundar, 1996; Sundar, 1999) Definitions Credibility has been defined variously as the perception of news messages as plausible reflections of the events they depict (Newhagen & Nass, 1989), as well as a global evaluation of the objectivity of a given story (Sundar, 1996). Dimensions of credibility include trustworthiness, expertise, fairness, lack of bias, accuracy, factual rendering, and completeness of a story (Gaziano & McGrath, 1986; Hovland & Weiss, 1951), as well as believability and community affiliation (Meyer, 1988). Conceptual and operational definitions of credibility in the research reported here were informed, in particular, by Gaziano and McGrath (1986) and Meyer (1988). Gaziano and McGrath used factor analysis to isolate 12 attributes of credibility for newspapers and television as media, without regard specific content. Respondents were asked to rate each of the 12 attributes on a 5-point scale, with 5 being most positive and 1 being least. Credibility scores for both newspapers and television were derived by summing responses. Attributes isolated by the research are listed below: 1. Fair or unfair 3. Biased or unbiased 5. Tells or doesn't tell the whole story 7. Is accurate or is not accurate 9. Invades or respects people's privacy 11. Does or doesn't watch after readers'/viewers' interests 13. Is or is not concerned about community well being 8. Does or does not separate opinions from facts 10. Can or cannot be trusted 12. Is concerned about the public interest or is concerned about making profits 14. Is factual or opinionated 16. Has well-trained or poorly trained reporters Subsequently, Meyer (1988), replicated the factor analysis, reducing to five the number attributes used to measure credibility, when credibility was defined as whether newspaper content is believed by readers. The five attributes are: 1. Fairness (Is the story fair?) 3. Lack of bias (Is the story unbiased?) 5. Tells the whole story (Is the entire story told?) 7. Accuracy (Is the story accurate?) 9. Trustworthy (Can the source be trusted?) In addition to such content variables, investigations of news credibility have taken into account modes of dissemination. Dissemination vehicles studied have included newspapers, magazines, radio, television, and multimedia presentations (Owen & Karrh, 1996). Credibility attributes of individuals as news sources (e.g., an on-air personality) and news organizations (e.g., the New York Times or CBS News) have also been assessed (Robinson & Kohut, 1988). News consumers seem influenced by the reputation of news organizations, such as the Wall Street Journal, or other sources routinely supplying news to the public. Such suppliers include major networks, local broadcasts, and local newspapers. Other studies have discovered relationships between news topics and assignment of credibility (Gaziano & McGrath, 1986). Topics evaluated included local news, election coverage, natural disasters, crime, business news, religious news, and news about events in the (then) USSR, the Middle East, and Latin America. The highest credibility score for newspapers was assigned to Federal government coverage. For television, greatest credibility was associated with crime and the arms race. Some researchers have concluded that criteria applied in judging credibility of various media and messages may change from one medium to another, and that weight assigned to various credibility criteria may vary across media (Gazianao and McGrath, 1986). Sundar (1999), however, found substantial similarities between factor structures underlying receivers' perceptions of print and online news, including those associated with credibility. This led Sundar to conclude that credibility measures are equally applicable across media. The same study, though, showed that news judged as disturbing negatively affected credibility assigned to online news stories, but had no similar effect on judgments of print news credibility. Relationships between credibility and demographic characteristics are mixed. Robinson and Kohut (1988) found that when such demographic characteristics as political affiliation were used to predict credibility, they contributed little to explained variance. On the other hand, Johnson and Kaye (1998) found gender, age, and education contributed significantly to assessments of credibility associated with online newspapers, news magazines, candidate literature, and issue-oriented sources. Generally females found the Internet more credible than males. Older respondents assigned less credibility to online newspapers, news magazines, and candidate literature than did younger ones. Similarly, better educated respondents found online newspapers less credible The conventional wisdom, popularized by Roper (1985), suggested that television news is more credible than print. However, this claim is not supported consistently by empirical studies. Credibility is governed, in part, by predispositions of news consumers. Those motivated to actively seek news regard newspapers as more credible than television (Mulder, 1980). Moreover, Carter and Greenberg (1985) found what they regarded as an anti-newspaper bias in Roper's instrumentation. The gist of the research is that reasons news consumers cite for assigning credibility to news are various and sometimes inconsistent. Credibility attributes across media may not be comparable, since criteria applied to both the medium and the message may differ. Moreover, where criteria are the same, they may be differentially weighted across media. Under some conditions, television news is found to be more credible than that disseminated by newspapers. In other cases, the converse is true. As noted, credibility judgments can be linked to age, gender, and other demographic characteristics, but the evidence is inconsistent. Credibility assigned to a medium varies, depending upon whether news is actively sought or passively received. Topical differences affect judgments of credibility, though the differences are not the same for all media examined. If credibility varies by news topic area, any comparative analysis of credibility for newspapers and the Internet must examine credibility under different conditions of news content. In the present study, three news topic areas were studied: business news, national news, and international news. Therefore, the first research question asks whether, in fact, credibility of the news varies across topic areas, without regard to the mode of distribution (paper or Internet). The t-test indicates credibility varies by topic (Table 1), but not significantly so. Business news was ranked most credible, with international news next. National news was ranked least credible. However, the differences are slight and are most likely the product of sampling error. Nevertheless, prior research indicates that differences in credibility across modes of news distribution (e.g., paper vs. Internet) may be affected by the particular news content of the articles being distributed, even if overall credibility does not vary by news topic areas. To provide for a more robust test of hypotheses, the design of the present study tested differences in credibility separately for three news topic area. Based on prior research (E.G., Carter & Greenberg, 1985; Mulder, 1980; Roper, 1985; Sundar, 1999) and logical deduction from theory, one could reasonably hypothesize that a newspaper story disseminated via paper would be perceived as more credible than the same story distributed over the Internet. Newspapers have a long history of exercising their gatekeeping responsibilities. When they exercise poor judgment regarding the truthfulness of what they publish, there may legal ramifications. In contrast, the Internet has few, if any, gatekeeping constraints. Even if a reputable news organization distributes news via the Internet, the "halo" effect of this mode of distribution arguable affects the reader's perception of credibility. This general argument provides the basis for the following four hypotheses. Hypotheses H1: Paper versions of newspaper stories will be scored on all credibility measures as more credible than Internet versions of the same story. H2: The paper version of business stories will be scored as more credible than the Internet version of the same story. H3: The paper version of international stories will be scored as more credible than the Internet version of the same story. H4: The paper version of domestic, national stories will be scored as more credible than the Internet version of the same story. The first hypothesis posits that the individual components of the credibility index will be significantly higher for newspaper versions of the stories, when compared to Internet versions of the same stories. For purposes of this test, differences in news topics were ignored. In hypotheses 2 through 4, differences between newspaper and Internet versions of the same story were tested for each of the news topic areas, using an index of credibility that combined the individual measures tested in the first hypothesis. Method Investigations of media credibility have traditionally relied on survey research. However, the present study first used focus group in an exploratory examination of the credibility of Internet news. Focus group findings were used to design an experiment to test relations between delivery medium and credibility. Focus Groups Two focus groups, each consisting of seven students, were conducted. Students were graduating seniors or graduate students at a West Coast university. Objectives of the focus groups included exploration of the relevance of five attributes of credibility (Meyers, 1988) to Internet-disseminated news. Additionally, insights from the focus groups were drawn upon to develop instrumentation for the experiment. Consistent with Sundar (1999), credibility attributes were judged by participants to be equally relevant to news material, whether distributed by newspapers or the Internet. However, introspection in a qualitative setting is not the best way to determine if the mode of news delivery is unrelated to credibility. Although participants in the focus groups reported that mode of news delivery makes no difference, prior research (Carter & Greenberg, 1985; Mulder, 1980; Roper, 1985; Sundar, 1999) indicated that the delivery medium does matter. In particular, Mulder's (1980) research provided a theoretical basis for positing the hypotheses above for rigorous test, using an experimental design. Experiment A posttest-only control group design was used, with the print version of the newspaper article treated as the placebo. The questionnaire allowed subjects to assess credibility of six news articles. Consistent with focus group results and Sundar (1999), the credibility index for both media and each of the six articles included the following six questions: 1) In your view, how trustworthy is the source of this article? 2) In your view, how accurate is the content of the article? 3) In your view, is the content of the article fair? 4) In your view, how well does the article tell the whole story? 5) In your view, is the article source unbiased? 6) Overall, how credible is the story? Ninety-two university students volunteered to evaluate the credibility of six news articles. Subjects were assigned randomly to either the control or test group. The control group (N=47) read the six stories on paper. The test group (N=45) read the same six stories on computer monitors using the university communication department's web site. Two stories were selected from the Wall Street Journal, two from the Los Angeles Times, and two from the New York Times. Stories 1 and 2, taken from the Wall Street Journal, were concerned with business-related matters (litigation involving the U.S. tobacco industry and the Microsoft monopoly litigation). Stories 3 and 4, taken from the New York Times, were concerned with international events (Colombian drug wars and the bombing by Breton separatists of a Paris McDonald's fast-food restaurant). Items taken from the Los Angeles Times, stories 5 and 6, were concerned with domestic matters (an Al Gore campaign address and a Washington, D.C. gay-rights rally) Stories were clipped from each of the newspapers. Subjects who read the placebo stories (in paper form) had a copy of each story attached to the questionnaire. For subjects exposed to the treatment, the same six stories were uploaded to a web site. Each story was identified by a source and date for the experimental and control conditions. Participants were asked to read the stories, then rate the credibility of each. Findings Data were coded and entered into an SPSS data file for analysis. Data from control and test groups were aggregated and a credibility index computed for all 92 participants for all six variables. Reliability coefficient (Cronbach's alpha) was .89. Analysis of variance was used to test hypotheses. Because H1-H4 specify greater credibility for print over Internet news distribution, a one-tailed test of significance was used. Because the total sample size is less than 100 subjects, tests are reported at the 95% decision rule (alpha=.05) and 90% decision rule (alpha=.10). The latter decision rule is sometimes appropriate with sample sizes smaller than 100 to help assess the probability of Type 2 error (Broom & Dozier, 1990; Williams, 1986). Stories read on paper were more credible on all credibility measures than the same material appearing on the web (Table 2). On four of the six measures, the difference was statistically significant, according to analysis of variance (ANOVA), the 95% decision rule (alpha = .05), and a one-tailed test specified in the hypotheses. For "fairness" (as defined above) the relationship was statistically significant at a 90% level of confidence (alpha = .10). To test H2, H3, and H4, credibility indexes were computed for business, international, and national stories. Analysis of variance supported the three hypotheses (Table 3) with regard to greater credibility of articles distributed in paper form, though only two of the three were statistically significant. For business and international news, articles distributed in paper form were rated as significantly more credible than Internet versions of the same articles, according to the F-test, the 95% decision rule, and the one-tailed test implied in the hypotheses. However, the relationship was not significant for national news, according to the same statistical test and decision rule. News articles distributed in paper form are generally regarded as more credible than the same articles distributed via the Internet, but this effect is specified to certain categories of news. Discussion The present study used an experimental design to isolate the causal impact of news distribution medium on perceived credibility of identical news stories. The research design permitted separate tests of the relationship between medium and credibility for three distinct news topic areas. In general, subjects exposed to paper versions of the same news stories (aggregated across all six stories) assigned greater credibility to those articles than did subjects exposed to the same six stories on a simulation of the Internet. However, differences in credibility are not consistent across all three topic areas. Photocopies of actual newspaper articles were deemed more credible than their Internet equivalents for business and international news. The difference in credibility was insignificant for national news, however, though subjects in the paper condition tended to give higher credibility scores than subjects in the Internet condition. When data were aggregated for both the paper and Internet groups, credibility differed significantly by news topic area. This squares with some prior research (e.g., Gaziano & McGrath, 1986) that credibility differs by news topic. The present study underscores the importance of treating both the medium of news distribution and news topic areas as important factors in perceptions of news credibility. The heuristic implications are plentiful. The objective should be to tease out, where they exist, differences that make a difference. Doing that will require issues addressed here be revisited with larger, more diverse samples. In addition to further inquiry regarding differences in news credibility assessments that may be a function of Internet dissemination, there should be additional investigation of the link between news type, i.e., national, international, etc., credibility, and medium Central to future research should be a closer look at the relationship between credibility and perceived functional differences across media, with particular emphasis on the Internet. BIBLIOGRAPHY Abel, J., & Wirth, M. (1977). Newspaper vs. TV credibility for local news. Journalism Quarterly, 54, 371-375. Berlo, D., Lemert, J., & Mertz, R. (1969-70). Dimensions for evaluating the acceptability of message sources. Public Opinion Quarterly, 33, 563-576. Broom, G., & Dozier, D. (1990). Using research in public relations: Applications to program management. Englewood Cliffs, NJ; Prentice-Hall. Burgoon, M., Burgoon, J., & Wilkinson, M. (1981). Newspaper image and evaluation. Journalism Quarterly, 58, 419-433. Carter, R., & Greenberg, B. (1965). Newspapers or television: Which do you believe? Journalism Quarterly, 42, 29-34. Gantz, W. (1981). The influence of researcher methods on television and newspaper news credibility evaluations. Journal of Broadcasting, 25, 155-169. Gaziano, C., & McGrath, K. (1985). The media credibility problem: Putting the research into perspective. Paper presented at the 40th annual conference of the American Association for Public Opinion Research, McAfee, NJ. Gaziano, C., & McGrath, K. (1986). Measuring the concept of credibility. Journalism Quarterly, 65, 451-462. Hovland, C., & Weiss, W. (1951). The influence of source credibility on communication effectiveness. Public Opinion Quarterly, 15, 635-650. Jacobson, H. (1969). Mass media believability: A study of receiver judgments. Journalism Quarterly, 46, 20-28. Johnson, T., & Kaye, B. (1988). Cruising is believing?: Comparing Internet and traditional sources on media credibility measures. Journalism & Mass Communication Quarterly, (75),2, 325-340. Lee, R. (1978). Credibility of newspaper and TV news. Journalism Quarterly, 55, 282- 287. Media: High ratings are tempered. (1985, August 12). Los Angeles Times, p. 4. Media in the dock. (1984, October 22). Newsweek, p. 66-72. Meyer, P. (1988). Defining and measuring credibility of newspapers: Developing an index. Journalism Quarterly, 65, 567-574. Meyer, T. (1974). Media credibility: The state of the research. Public Telecommunications Review, 2(4), 48-52. Mulder, R. (1980). Media credibility: A uses-and-gratifications approach. Journalism Quarterly, 57, 474-476. Newhagen, J., & Nass, C. (1989). Differential criteria for evaluating credibility of newspapers and TV news. Journalism Quarterly, 66, 277-284. Owen, A., and Karrh, J. (1996). Video news releases: Effects on viewer recall and attitudes. Public Relations Review, 22(4), 369-378. Regan, J., & Zenaty, J. (1979). Local news credibility: Newspapers vs. TV revisited. Journalism Quarterly, 56, 168-172. Rimmer, T., & Weaver, D. (1987). Different questions, different answers? Media use and media credibility. Journalism Quarterly, 64, 28-44. Robinson, M., & Kohut, A. (1988). Believability and the press. Public Opinion Quarterly, 52, 174-189. Roper, B. (1985). Public attitudes toward television and other media in a time of change. New York: The Television Information Office. Shaw, E. (1973). Media credibility: Taking the measure of a measure. Journalism Quarterly, 50, 306-311. Sundar, S. (1996). Effect of source attribution on perception of online news stories. Journalism and Mass Communication Quarterly, (75)1, 55-68. Sundar, S. (1999). Exploring receivers' criteria for perception of print and online news. Journalism Quarterly, (76),2, 373-386. Williams, F. (1986). Reasoning with statistics (3rd ed.), New York: Holt, Rinehart & Winston. Credibility of Newspapers and Internet Credibility of Newspapers and Internet Table 1 Impact of Topic on Perceived Credibility Of Business, International, and Domestic News Articles Mean d.f. t-value Sig. Pair Business/ 2.9 91 .45 >.05 1 International 2.8 Pair International/ 2.8 91 .40 >.05 2 National 2.7 Pair Business/ 2.9 91 .94 >.05 3 National 2.7 Credibility of Newspapers and Internet Credibility of Newspapers and Internet Table 2 Impact of Paper and Internet Distribution of News Articles on Perceived Credibility Mean Mean Credibility Internet Paper Measure Score Score F-ratio d.f. Sig. Trustworthiness 2.4 3.0 7.88 1/90 <.05 Accuracy 2.6 3.2 6.86 1/90 <.05 Fairness 2.4 2.6 1.85 1/89 <.10 Tell complete story 2.1 2.3 1.06 1/90 >.10 Lack of bias 2.5 3.1 7.77 1/90 <.05 Overall Credibility 2.5 3.1 7.72 1/90 <.05 Credibility of Newspapers and Internet Credibility of Newspapers and Internet Table 3 Impact of Paper and Internet Distribution on Perceived Credibility of Business, International, and Domestic News Articles Story Type Mean Mean Total F-ratio d.f. Sig Hypothesis (source) Internet Score Paper Score Both Media 2 Business News (Wall St. Journal) 2.6 3.2 2.9 3.72 1/90 <.05 3 International News (New York Times) 2.2 3.4 2.8 9.55 1/90 <.05 4 National News (Los Angeles Times) 2.5 2.9 2.7 1.5 1/90 >.10