DO QUOTES AFFECT PERCEPTION OF ONLINE NEWS STORIES?
By
S. Shyam Sundar
Assistant Professor
College of Communications
Pennsylvania State University
219, Carnegie Building
University Park, PA 16802
voice: (814) 865-2173
fax: (814) 863-8044
e-mail: [log in to unmask]
Paper submitted to the Communication Technology and Policy Division to be
considered for presentation at the annual AEJMC convention, Aug. 10-13, 1996,
Anaheim, CA
Running Head: QUOTES IN ONLINE NEWS
DO QUOTES AFFECT PERCEPTION OF ONLINE NEWS STORIES?
A B S T R A C T
Are quoted sources in online news as psychologically meaningful as those in
printed news and broadcast news? A within-subjects experiment was designed
answer this question. On a web site, subjects (N=48) read three online news
stories with quotes and three without. They rated stories with quotes
significantly higher in credibility and quality than identical stories without
quotes. However, quotes did not seem to affect their ratings of liking for -
and representativeness (newsworthiness) of - online news.
DO QUOTES AFFECT PERCEPTION OF ONLINE NEWS STORIES?
Most online users receive "news" from various sources - electronic mail from
cyber-acquaintances, mailings from distribution lists, newsgroup postings, web
sites and a growing number of online news services. Given such a multiplicity
of sources, it is likely that online users may not be psychologically capable of
remembering which source supplied which piece of information or news. That is,
they may not store information content along with source attribution in their
long-term memory. This phenomenon is similar to Hovland's source-containment
sleeper effect whereby the source is said to be disassociated from the message
over time such that the effectiveness of the positive source decreases while
that of the negative source increases (Hovland & Weiss, 1951; Kelman & Hovland,
1953).
Applied to the context of online news delivery, the sleeper effect may result
in the "I-read-somewhere-that" phenomenon wherein users may use information
obtained by online means in their non-online lives without adequately adjusting
for the validity or credibility of the information based on who or what the
source is. An important implication of this phenomenon is the gradual decline
in the psychological importance of sources in online news stories. While
sources represent the bread and butter of a news story in traditional media like
print and broadcast, they may not be as important to users' judgments of the
verdidicality of online news stories.
An experiment was designed to test this possibility. "Source" was
operationalized as a person or institution quoted within a news story. The
presence or absence of a quoted source constituted the independent variable.
Users' evaluations of news stories constituted the dependent variable. The
purpose of this experiment was to determine if the presence of quoted sources in
online news stories made any difference to users' psychological conceptions of
news content.
This paper will first explicate the concept of source and review past research
on the psychological effects of quotes. Next, it will describe the development
of psychological criteria for determining perceptions about news stories. It
will then present the methods and results of an experiment designed to test the
effect of quotes on the perception of online news stories.
"Source" explicated
The concept of "source" is so integral to communication research that it is
surprising that a literature search of the entire field failed to yield a single
thorough explication of the concept. This is perhaps because the common-sense
understanding of the term "source" has sufficed for most researchers. The
Oxford American Dictionary defines "source" as "the point of origin" or "the
place from which something comes or is obtained." It also offers a more
media-centric meaning of "source": "a person or book, etc., supplying
information." As these definitions imply, "source" need not necessarily refer
to the sender in the SMCR models; it could even refer to the message or the
channel, depending upon who or what is perceived by the receiver to be the
source of the communication. As Chaffee (1982) points out, receivers do not
differentiate clearly between a person who generates a message ("source") and
one who relays a message that was created elsewhere ("channel").
This ambiguity in the psychological definition of source has led to a confusing
multiplicity of meanings associated with the word "source." As Newhagen and
Nass (1989) point out, a critical omission in the studies of source effects is
that frequently no distinction is made between a person as a source as opposed
to an organization as a source. This problem is compounded when some
researchers (e.g., Carter & Greenberg, 1965; Abel & Wirth, 1977) treat the media
channels (newspapers and television) as competing sources of information and
influence.
In the media effects paradigm (e.g., Bryant & Zillmann, 1994), for example, the
"source" of the communication is either a mass medium like television (e.g.,
Gerbner, Gross, Morgan & Signorielli, 1994), a media channel such as the
Washington Post or NBC (e.g., McCombs, 1994), message content such as violent
acts (e.g., Jo & Berkowitz, 1994), a human being modeling a certain behavior
(e.g., Bandura, 1994), or a program genre such as comedy (Zillmann & Bryant,
1991).
The source credibility literature is equally broad in its interpretation of the
term "source." A message source may be a person (e.g., Walter Cronkite), a
group (e.g., a random sample of the US population), an institution (e.g., the
Supreme Court), an organization (e.g., American Medical Association) or even a
label (e.g., "conservative") that has a favorable or unfavorable connotation for
the message recipient (Hass, 1988). In fact, the first study in this tradition,
by Hovland and Weiss (1951), confounded source with media channel by comparing
well-known publications with well-known individuals on a credibility dimension.
This is because the experimenters did not distinguish along the humanness
dimension of "source." Rather, they conceptualized "communicator" as being
either high or low in credibility. Credibility was the independent variable of
interest while persuasion was their dependent variable. In general, the source
effects literature operationalizes source characteristics in three ways:
credible versus not-credible; physically attractive versus unattractive;
ideologically similar versus dissimilar (Wilson & Sherrell, 1993). This is done
regardless of the fact that one of the values in a given dichotomy is a human
being while the other is a mass media channel.
Other studies have conceptualized source as the image of the communicator
(Sargent, 1965), as encoder and decoder (Papa & Tracy, 1988), as vocal
attributes of the speaker (Addington, 1971), as social representations
(Moscovici, 1984) and as anything other than self (e.g., Ackerman, 1992).
With the arrival of new communication technologies, there is yet another
contender for the title of "source" - namely the physical manifestation of the
technology itself. As Nass, Steuer & Tauber (1994) have summarized, receivers
sometimes treat the medium itself (i.e., computer box or television set) as an
autonomous source worthy of human social attributions.
Even communication receivers can be thought of as sources. The growing
acceptance of the idea of an active audience has spawned a great deal of
research that looks at communication as a dependent variable as opposed to an
independent variable. Communication is sought to be studied as a function of
audience activity. Audience activity includes everything from traits,
dispositions and subjective states of audience members to the purposive
selection of communication. The message learning approach to persuasive
communication (Petty & Cacioppo, 1981, p. 60) sometimes attributes attitude
change to such receiver characteristics as involvement, gender, and
intelligence. Thus, the receiver (or his/her characteristics) constitutes the
source of communication in this paradigm.
In selective exposure research (Zillmann & Bryant, 1985), the source of
communication is the receiver. That is, the receiver initiates the
communication by selecting the particular content to be consumed based on his or
her affect, mood and other dispositions. Thus, it could be argued that
regardless of who manufactured the content, the source of content remains the
receiver. It follows that given the receiver's sense of perceived choice, the
receiver's perception of content would be different from his or her perception
of the same content selected by someone else (e.g., Eagly & Whitehead, 1972;
Himmelfarb & Arazi, 1974; Jones & Brehm, 1967).
In summary, there are three dominant conceptions of source in the communication
literature. Source credibility researchers consider the gatekeeper as the
source while technology researchers consider the medium or channel as the
source. Selective exposure researchers consider the receivers as sources.
According to Sundar (1995), none of these are truly original sources of
communications. They are merely "selecting sources" that decide which news
stories are consumed and which are not. In his typology, Sundar (1995)
identifies only one "original source," and that is information provider within a
news story, the person or entity that is quoted in the story. This definition
is in line with what journalists refer to as "sources" in their parlance.
This paper will hencetoforth adopt the journalistic conception of "source."
According to this conception, a source in the online news scenario is not
America Online, CNN Interactive, or the computer terminal used to access the
news. Instead, it is the entity quoted within the digital narration of a piece
of news rendered by online means.
Quotes as Sources
For journalists, "source" represents the bread and butter of a news story.
Journalistic sources are those people or organizations that are quoted in news
stories. Editors constantly demand that reporters get "quotes" for their
stories. "Quotes" are quotations from the right sources, from sources that
would not only be relevant but also credible given the context. Getting quotes
is a golden rule in all of journalism, regardless of the medium of news delivery
(e.g., Jones, 1976; Newson & Wollert, 1985). While the print media publish the
quotes, as far as possible within direct quotation marks and sometimes with
photographs of the quoted "sources," the electronic media make elaborate
arrangements to record sources for broadcasting. Sometimes, television crews
travel hundreds of miles just to get a one-line quote from a source on camera.
Even radio news stories often air quotes - if not directly from the source's
mouth then by having somebody else other than the main newsreader say it - to
lend veridicality to the stories. This phenomenon of "sourcing" is so integral
to good journalism that a truly newsworthy statement of fact is rarely if ever
published if it is not properly attributed to a legitimate source (e.g., Izard,
Culbertson & Lambert, 1971; Fox, 1977). Ontologically then, the original source
of a news story is the primary information provider in the form of quotations
from the source. Psychologically, the effect of quotations is unclear.
Much of the journalistic preoccupation with source is premised on the belief
that receivers actively monitor the source while processing and evaluating the
piece of news presented to them. To that extent, journalists present themselves
as mere conduits passing along information from sources to receivers. They
treat themselves a part of the medium of transmission rather than as originators
or sources of information.
Whether this assumption is psychologically valid is a question up for debate,
especially given the research findings in source credibility and gatekeeping
effects that emphasize the psychological effects of the information presenter
over those of the content itself. That is, do receivers process sources
embedded within news stories or do they evaluate journalistic information based
on who delivers it to them? While there is substantial psychological evidence
for receivers' bias in favor of human sources (as opposed to statistical
sources) in the information processing and perceptions of story content (e.g.,
Hamill, Wilson & Nisbett, 1980; Zillmann, Perkins, & Sundar, 1992), the evidence
on the effects of within-story sources on readers' evaluations of
newsworthiness, story credibility, and other aspects of the story is
inconclusive.
An experiment comparing direct quotes with paraphrases, for example, failed to
yield differences on readers' ratings not only of story attributes such as
accuracy, objectivity, believability, informativeness, and readability, but also
of source attributes such as dramatic, informed, effective, colorful, precise,
and emotional (Weaver, Hopkins, Billings & Cole, 1974). Another study found no
differences in believability between the following three versions of the same
news stories: with specific attribution to a named source, with general
attribution to source consisting of a title but no name, and no attribution at
all. The perceived accuracy of one of the two news stories in the experiment
did vary with source attribution: The version with specific attribution was
rated slightly but significantly more accurate than the other two versions
(Hale, 1984). But, an earlier study had the reverse result: Stories without
sources or with unidentified sources were rated more accurate and more
significant, and subjects agreed with them more, than stories with a named
source (Fedler & Counts, 1981).
Culbertson and Somerick (1976) found no differences in perceived accuracy or
truthfulness between news stories with and without named sources. This is not
to say that readers do not notice sources within news stories. In a later
analysis, the researchers found that people tend to maintain a symbiotic
relationship with news sources; regular readers tend more than others to depend
on source attribution. Specifically, they found that readers'
"print-orientation" (i.e., greater reliance on - and preference for - print
compared to other media) corresponded positively with attentiveness to sourcing
in news stories (Culbertson and Somerick, 1977).
Gibson and Zillmann (1993) also found a medium difference when they
investigated the effects of quotations upon impression formation. They found a
big difference between quotations and paraphrases on issue perception, but this
effect was observed for print and not radio. Subjects presented with quotations
from sources questioning the safety of amusement parks perceived the overall
safety of such parks to be less adequate than did subjects given the same
information in paraphrased form, or without any sources.
The psychological effects of quoted sources within news stories can be
summarized as follows: Readers, especially those with a print orientation, seem
to notice sources of quotes as well as to form impressions based upon the
sources' quoted remarks. However, their reliance upon quoted sources for
evaluating the quality, credibility, and newsworthiness of news stories is
unclear given the ambiguous findings in past research.
Search for Psychological Criteria
Since this study attempts to find psychological effects, if any, as a function
of within-story sources in online news, there is need to develop psychological
criteria for determining the dimensions along which human receivers of
communication differ in their consideration of content, specifically news
content. In other words, what are the psychological determinants of news that
distinguish between different news sources? When readers make processing
decisions and evaluation judgments of news stories, what psychological variables
are implicated? How do we know what to measure in a reader if we wanted to find
out the psychological difference in the reader as a function of, for example,
the credibility of the news source? Along which dimension(s) will a news user
differ in his or her evaluation of two news stories that are identical in
content but are delivered by two different sources?
Evaluation criteria for news stories are plentiful. While journalism textbooks
prescribe a set of normative criteria for determining whether something is news
or not, communication research has used a wide variety of attributes or
characteristics to describe news.
In his textbook on news reporting, Mencher (1994) defines news in terms of news
values such as timeliness, potential impact, prominence of people involved,
proximity to audience, and novelty of the event. Almost all journalism
textbooks consider accuracy (whereby all published information is verified),
attribution (proper identification of information source), balance, fairness,
objectivity, brevity, and clarity as essential components of a good news story
(e.g., Brooks, Kennedy, Moen & Ranly, 1992). These criteria emerged as a
consequence of the social responsibility era of the press, which began with the
Hutchins Commission on Freedom of the Press stating in 1947 that what a free
society needs from journalists is "a truthful, comprehensive and intelligent
account of the day's events in a context which gives them meaning."
Academic research has also focused on the social responsibility role of news.
Self (1988), for example, classified news in terms of its task: to give facts
objectively, to explain the facts, or to report all sides of an issue fairly.
Others have looked into particular aspects of news like accuracy, objectivity,
believability, sincerity, bias, informativeness, readability, fairness, and
truthfulness (e.g., Weaver et al., 1974; Burgoon, Burgoon & Wilkinson, 1983;
Carter & Greenberg, 1965; Sargent, 1965; Culbertson & Somerick, 1976; Fedler &
Counts, 1981; Hale, 1984; Austin & Dong, 1994). These qualities of news are
typically used (in their adjectival form) as dependent variables administered to
subjects or respondents on quantitative scales ranging from three to ten points.
In addition to variables eliciting evaluations of stories, some studies have
used variables that describe the self-reported effect of news stories upon
subjects - variables such as happy, sad (e.g., LeBouef and Matre, 1977),
pleasing, and disturbing (e.g., Leshner, 1994).
Taken together, the attributes used to describe a news story fall into four
categories: Credibility, Liking, Quality, and Representativeness. The concept
of credibility, as applied to a news story, may be defined as a global
evaluation of the believability of the story (e.g., Wilson & Sherrel, 1993;
Hovland & Weiss, 1951; McGuire, 1985; Gunther, 1987). Liking is overall
affective reaction (e.g., Zajonc, 1980, 1984). Applied to a news story, liking
is an indicator of a news receiver's feelings toward - or evoked by - the
overall content of the news story (e.g., Nass, Reeves & Leshner, in press).
Quality means the degree or level of overall excellence of a news story. It
signifies an evaluation of the goodness of a communication message (e.g., Gibson
& Zillmann, 1993, Leshner, 1994). Representativeness of a news story is a
summary judgment of the extent to which the story is representative of the
category of news. In other words, it is the answer to the following question:
What is the probability that the story, taken as a whole, belongs to the class
of entities that we call "news"? This definition of representativeness is
borrowed from the work of Tversky and Kahneman (1974) on the representativeness
heuristic, which is basically a relevancy judgment that, under uncertainty,
produces a short-cut probability estimate for the question of the form, How
probable is it that A belongs to category B? This heuristic neglects key
relevant factors and instead relies on the degree of resemblance between the
object A and the stereotype associated with category B. To the extent a story
manifests features that are considered integral to the broad psychological
notion of news, it would be considered a news story (e.g., Rosch, 1975, 1978;
Tversky, 1977). That is, it would be judged as representative of news.
Method
This study investigates whether psychological reactions of communication
receivers to online news stories will be different if the stories had sources
quoted in them or not. Specifically, the present investigation looks for
differences in receivers' ratings of credibility, liking, quality, and
representativeness of the content as a function of the presence or absence of
quoted sources through a within-subjects experiment. This is done by keeping
content constant and controlling for selecting sources. The context for this
research is online news, the independent variable is Quotes (with two values -
presence and absence), and the dependent variables are users' ratings of
Credibility, Liking, Quality, and Representativeness of online news stories. In
order to control for selecting sources, a fourth of the subjects were told that
the news stories were selected by gatekeepers, another one-fourth were told that
they were selected by the computer terminal, yet another one-fourth were told
that they were chosen by other members (or users) of the online news service,
and the final one-fourth were given a pseudo-selection task leading them to
believe that the stories were chosen by themselves. Since the selecting source
variable did not interact significantly with the independent variable on any of
the dependent variables, it will not be discussed hencetoforth in this paper.
Subjects
Sixty-four undergraduate and graduate students enrolled in communication
classes participated in the experiment. However, usable responses were obtained
from only forty-eight subjects. Each subject was paid $10 for participation.
The experiment was administered to subjects in groups ranging in size from
three to twelve persons.
All subjects were asked to sign the informed consent form before commencing the
experiment. They were then instructed, as a group, to read six news stories
through an online service and answer a paper-and-pencil questionnaire after
every story. As promised in the consent form, all sessions of the experiment
lasted a little under 45 minutes.
Design Overview
In an attribution style within-subjects experiment, all subjects were exposed
to identical content, but half the content had one value of the independent
variable (presence of quotes) while the other half had another value (absence of
quotes).
Operationally, the design may be summarized as follows: All subjects read six
news stories each on an online news service. Three of these six stories had
quotes in them while the other three did not have any quotes. After reading
each story, subjects filled out a paper-and-pencil questionnaire evaluating
their liking for - and the credibility, quality, and representativeness of - the
news story they had just read.
Experimental Treatment Conditions
As mentioned earlier, the independent variable had two within-subjects values -
presence and absence of quotations in the news story. These two values will
henceforth be referred to as Quote and No-Quote.
All subjects in the experiment read six news stories - one each in the
following common categories of news: National, International, Local, Business,
Sports, and Entertainment. Six news stories were created especially for this
study by rewriting articles that had already appeared in mainstream newspapers.
These stories were chosen because they were routine and would not evoke
particularly strong negative or positive reactions. The national story was
about a Supreme Court ruling in a child custody case, the international story
concerned a family planning program in Iran, the local story related to funding
for a highway link in San Jose, the business story centered around statistical
data about American business productivity in the last five years, the sports
story was about the Buffalo Bills' cheerleaders winning a ruling from the
National Labor Relations Board to form the first employees union, and the
entertainment story dealt with a television personality moving from one network
to another.
All subjects in the experiment read the same six stories, with minor variations
in attribution of quoted sources as described below:
Two versions of each of the six stories were created such that one had
quotations and one did not. The stories were equated for content in an effort
to make sure that the only difference between the Quote and No-Quote versions
was attribution to a quoted source. For example, the Quote version of the
business news story read as follows:
WASHINGTON--American business productivity improved in 1994 for the fifth
straight year, boosted by the largest output growth since 1984. At the same
time, businesses held labor costs to the smallest gain in 30 years. Some
observers say that this is "an indication of further growth" but other observers
feel that it could be "a sign of peaking before decline in growth."
Productivity--defined as output per hours worked--jumped 2.2 percent last year,
faster than the 1.5 percent advance in 1993. It was the fifth consecutive gain
since productivity fell 0.9 percent in 1989.
Over time, productivity determines the nation's living standards and the
competitiveness of its products overseas. Strong productivity is likely to hold
off inflationary pressures.
Some analysts maintain that growth in productivity will slow and then decline
as companies continue to add workers and the current business cycle continues.
But many disagree.
"The opposing view is that employers will continue to get more out of their
workers by re-engineering their manufacturing and service processes, and by
outsourcing," said Stephen Roach, an economist with Morgan Stanley & Co.
Growth in productivity slowed to a seasonally adjusted annual rate of 1.8
percent in the final three months of 1994, from a strong 3.2 percent in the
previous quarter.
This story had actually appeared in a major metropolitan newspaper. It had
attribution to two sources, which were retained for the Quote version. However,
for the No-Quote version of the same story, the quotation marks and the source
attributions were removed, but the content of the quoted remarks was retained.
(One can imagine a third condition wherein the quotation marks are removed but
the source attribution is retained. Since this study attempted to study the
effects of quoted sources and not quotation marks per se, it was decided not to
include this possible third condition in this study). The No-Quote version of
the above story read as follows:
WASHINGTON--American business productivity improved in 1994 for the fifth
straight year, boosted by the largest output growth since 1984. At the same
time, businesses held labor costs to the smallest gain in 30 years. This is an
indication of further growth but it could be a sign of peaking before decline in
growth.
Productivity--defined as output per hours worked--jumped 2.2 percent last year,
faster than the 1.5 percent advance in 1993. It was the fifth consecutive gain
since productivity fell 0.9 percent in 1989.
Over time, productivity determines the nation's living standards and the
competitiveness of its products overseas. Strong productivity is likely to hold
off inflationary pressures.
Growth in productivity will slow and then decline as companies continue to add
workers and the current business cycle continues.
The opposing view is that employers will continue to get more out of their
workers by re-engineering their manufacturing and service processes, and by
outsourcing.
Growth in productivity slowed to a seasonally adjusted annual rate of 1.8
percent in the final three months of 1994, from a strong 3.2 percent in the
previous quarter.
Similarly, two versions (Quote version and No-Quote version) of the same story
were created for each of the other five types of news stories used in this
experiment.
Although the independent variable in this study was varied within subjects,
each subject read only one of the two versions of each story. Of the six
stories read by each subject, three belonged to the Quote value and three
belonged to the No-Quote value. For example, if the national, international,
and local stories read by a subject all had quotes in them, the sports,
business, and entertainment stories would not have any quotes. However, for
every subject who got this combination of story type and manipulation, there was
another subject who read the No-Quote versions of the national, international,
and local stories and the Quote versions of the sports, business, and
entertainment stories. Six different combinations of story type and
manipulation were created such that equal numbers of subjects read the Quote and
the No-Quote versions of each of the six news stories.
Dependent Measures
Content perception was the main dependent variable in this study. This was
operationalized in terms of subjects' ratings of credibility, liking, quality
and representativeness of the news story.
Specific measures aimed at capturing the concepts of credibility, liking,
quality, and representativeness were obtained from two sources: (1) past
research involving these concepts (see the section on Psychological Criteria),
and (2) a pretest eliciting receivers' thoughts and feelings, in their own
words, toward news stories.
All measures were in the form of adjectives so that subjects could easily rate
characteristics of news stories on a Likert-type scale.
The following six measures comprised the credibility variable in this study:
Accurate, Believable, Biased, Fair, Objective, and Sensationalistic.
The following five adjectival measures were used to measure subjects' liking
for a news story: Boring, Lively, Enjoyable, Interesting, and Pleasing.
The following battery of five measures was used for assessing news story
quality: Clear, Coherent, Comprehensive, Concise, and Well-written.
The following five measures comprised the representativeness variable in this
study: Disturbing, Important, Informative, Relevant, and Timely.
In all, 21 measures were used to capture the four dependent variables in the
study. These measures were in the form of adjectives placed at the left-hand
side of a 10-point scale anchored between "Describes Very Poorly" and "Describes
Very Well"
Procedure
An online site was established on the World Wide Web especially for this
experiment .
The experiment was conducted in a journalism laboratory which had 18 Macintosh
computers with access to the Internet. The Web site used for this experiment was
accessed through these computers. The experimenter cleaned the desktops of the
computers and issued the necessary network commands for accessing the online
news site before the arrival of subjects.
When subjects arrived at the lab, all monitors displayed the first screen of
"Online News." They were first seated around a center table away from the
computers. They were welcomed and told briefly about the study. Specifically,
they were told they would be "reading news stories online and responding to
questions in the booklet" given to them.
Since the experiment was administered in groups, the experimenter directed
subjects through the procedure, one segment at a time. That is, subjects were
instructed to work at their own pace, but were asked to read one story only or
fill out one questionnaire only, then turn to the separation page and wait for
further instructions to continue. After all the subjects had completed a
particular assignment, they were together guided to either the online screen or
to obtrusively numbered pages in the booklet for the next task.
After subjects finished answering questions about the last (sixth) news story,
they were asked to fill out a "Final Questionnaire" that had questions about the
entire interaction, not individual news stories. Embedded in these questions
was the manipulation check for the controlling variable, which read as follows:
"The news stories you read today were selected by: ______________" After they
completed this questionnaire, subjects were asked to furnish some personal
information for the purpose of making payments.
Questionnaire booklets were then collected from all subjects. They were
debriefed, thanked for their participation, and dismissed.
Data Analysis
A confirmatory factor analysis was conducted on the 21 dependent measures in
order to confirm that the measures grouped into the four theoretically relevant
factors of credibility, liking, quality, and representativeness.
The factors obtained from this analysis were labeled and checked for internal
consistency. Indices were created by summing the measures that grouped together
as a factor. Indices were computed for each of the six stories separately. In
order to control for differences between the six story types, the mean score of
all subjects for each of the six story types on a given index was subtracted
from every subject's score for the corresponding story types on that index.
That is, the mean score of all subjects for each of the six story types on each
of the indices was first computed. Then, this mean was subtracted from each
subject's score on a given index for a particular story type. For example, if
the overall mean score (all subjects combined) on the credibility index was x
for the national story, y for the international story, z for the local story, p
for the sports story, q for the business story, and r for the entertainment
story, and a subject's score on the credibility of national story was n, the
credibility of international story was i, of the local story l, of the sports
story s, of the business story b and of the entertainment story e, then that
subject's rating of the national story on the credibility index was given by
(n-x). Similarly, the subject's credibility rating for the international,
local, sports, business, and entertainment stories were given by (i-y), (l-z),
(s-p), (b-q), and (e-r) respectively.
The indices obtained by this method were then used as dependent variables, one
at a time, in a series of 2x4 mixed analyses of variance, with the quote
manipulation as the within-subjects factor and the selecting sources
manipulation (controlling variable) as the between-subjects factor. Results of
these analyses were examined for significant differences as a function of (1)
the presence of quotations in news stories, and (2) interaction between the
presence/absence of quotations and the type of source that selected the news
stories. Since there were no hypothesized relationships between the four
dependent variables, it would be difficult to interpret source effects upon
combinations of these four variables. Therefore, it was decided not to run
multivariate analyses of variance.
Results
Questionnaires filled out by 48 subjects were deemed usable for analysis. There
were equal number of subjects (12) in each of the four selecting source
conditions of the experiment.
Confirmatory Factor Analysis
A principal components analysis with varimax rotation confirmed that the 21
dependent measures grouped into the four factors of credibility, liking,
quality, and representativeness, as expected. These factors together accounted
for 68.7 percent of the variance.
Analyses of Variance
Differences in the dependent variables (i.e., ratings of Credibility, Liking,
Quality, Representativeness, and Insightful) as a function of the six different
story types were controlled for by subtracting the overall mean of a given
dependent variable for a particular story type from every subject's rating on
the variable for that story type. Furthermore, two means were computed for each
subject: one was the average of the subject's ratings on the three news stories
with quotations, and another was the average of the ratings on the three stories
without quotations. Thus, each subject had two lines of data instead of six.
The two lines referred to the two levels of the within-subjects factor of
Original Source in this experiment. This reduced the 288-row dataset to 96
rows. The data were then cleaned for multivariate outliers before further
analysis.
Credibility: When the mixed factorial 2x4 analysis of variance was run with the
Credibility index as the dependent variable, a main effect for the quote
manipulation was observed such that news stories with quotes were perceived by
subjects as being significantly more credible than the same news stories without
quotes, F (1,44) = 37.52, p < .001 (See Figure 1 for means on a ten-point
scale).
Liking: When the Liking index was used as the dependent variable in the mixed
factorial 2x4 analysis of variance, the main effect for quotes was
non-significant, F (1,44) = 0.00, p = .99. The interaction term was also
non-significant. However, a significant main effect for the controlling
variable was observed. This result is of no consequence to the current
investigation.
Quality: With the Quality index as the dependent variable in the analysis of
variance, the interaction between the two types of source manipulations was not
significant. However, a significant main effect was observed for the quote
manipulation, F (1,44) = 20.94, p < .001. Subjects rated news stories with
quotes significantly higher in quality than the same stories without quotes (See
Figure 1).
Representativeness: The interaction term was non-significant when the
Representativeness index was subjected to the analysis of variance. Nor was
there a difference in the Representativeness rating as a function of the
independent variable. However, the main effect for the controlling variable was
significant - a result similar to the one obtained with the Liking index, and,
again, of no consequence to this study.
In summary, results from data analyses suggest that the presence of quotes in
online news stories enhances the perceived credibility and the perceived quality
of the stories. However, it does not seem to influence the perceived
representativeness (or newsworthiness) of the stories. Nor does it seem to
affect readers' liking for the online stories.
Discussion
The experiment yielded significant differences in the psychological perception
and evaluation of online news content as a function of the presence/absence of
quotations in online news stories. Moreover, the findings are clear and
unambiguous, with differences showing up as either extremely statistically
significant or extremely insignificant.
As mentioned in the literature review, journalism research abounds with
conflicting findings about the effects of quotations. This is perhaps due to
the absence of clearly explicated dependent variables. Most studies seem to use
single questionnaire items instead of psychologically relevant composites as
dependent measures. In contrast, the present investigation employed four highly
reliable indices as dependent variables.
Controlling for the effects of selecting sources, the experiment reported in
this paper found that quoted sources clearly mattered on two out of the four
psychological criteria used by receivers to evaluate online news stories.
Highly significant main effects for the quote manipulation were obtained on
Credibility and Quality, but not on Liking and Representativeness (See Figure
1). Receivers estimated the credibility and quality of stories with quotations
to be significantly higher than identical stories without quotations. However,
they did not seem to think that quotations made a difference to either the
representativeness of - or their liking for - news stories.
First and foremost, these results confirm that journalists' preoccupation with
getting quotes for news stories is a psychologically valid concern. More
importantly, they dissect the psychological effects of quotes and pinpoint the
areas in which quoted sources have an effect and areas in which they do not seem
to have any effect. For example, receivers' perceptions of newsworthiness of a
news story are not significantly affected by quotations. Neither is their
liking for stories affected by the presence or absence of original sources.
Quotes play a significant role only in receivers' perceptions of credibility and
quality of news stories.
This suggests that the "print-orientation" noticed by Culbertson and Somerick
(1977) is applicable not only to traditional print media but also to online
media. It appears that online news users are just as likely as newspaper
readers to use the presence or absence of quoted sources to make judgments about
the credibility and quality of online news stories. However, quoted sources do
not seem to enhance the perceived newsworthiness of an online news story.
By demonstrating the psychological importance of quotes in online news, this
study implies that the current proliferation of online news sources is unlikely
to lead to the "I-read-somewhere-that" phenomenon. Tidbits of news consumed via
the computer screen are not all processed in a cyber-haze but attended to as
deliberately as news stories printed in a newspaper. Online users do notice
quotes in news stories transmitted digitally. While they may not factor them
into their decisions about the importance or liking of a piece of news, they do
consider quotes while evaluating the credibility and quality of online news.
These results should be interpreted by keeping in mind an important
methodological limitation of the experimental stimuli. In an effort to keep the
information content constant between the two conditions, the quotation marks and
the accompanying attributions to original sources were simply removed from the
Quote version of every news story to create the corresponding No-Quote version.
That is, the language of the quoted remarks was not changed for the No-Quote
version save for minor transitionary phrases. This meant that some sentences
which would be appropriate in a news story only when enclosed in quotation marks
appeared in the No-Quote version without any attribution. This might have made
the news story read like an opinion article because the sentences with quotation
marks removed would presumably be attributed to licentious writing on the part
of the journalist. Subjects in the experiment might have been reacting to this
perceived editorializing on the part of the writer when they made their
evaluations, and not really responding to the presence or absence of quotation
marks in the stories.
Future research should attempt to create news stories, whose Quote and No-Quote
versions do not differ in any way other than the presence and absence of
quotation marks. This may be difficult because news stories without strongly
opinionated quotations are not interesting enough to read. Another line of
study could investigate if the presence of quotations in non-news stories, like
editorials and opinion columns, influences receivers' ratings of credibility and
quality. A third solution would be to employ stimulus sampling by comparing an
exhaustive sample of stories with quotes with an equally large but different
sample of stories without quotes.
Since most of the quoted remarks in the present study were attributed to
people, future studies should use a diversity of original sources that include
institutions and organizations, to determine if the ontological differences
between people as sources and institutions as sources are psychologically
meaningful.
Yet another line of investigation can concentrate on the characteristics of
"online-orientation" in order to discover if preference for - or dependence upon
- online media (as opposed to traditional media) dictate aspects of
psychological processing of news.
Figure 1
News Story Ratings as a Function of Quotes
(NQ = No-Quote Condition; Q = Quote Condition. Higher scores indicate higher
ratings. Comparisons between the two conditions are indicated by lowercase
superscripts placed on top of the bars in the graphs. Conditions with different
superscripts differ at p < .05 by multiple t or F-test).
References
Abel, J. D., & Wirth, M. O. (1977). Newspaper vs. TV credibility for local
news. Journalism Quarterly, 54, 371-375.
Ackerman, B. P. (1992). The sources of children's source errors in judging
causal inferences. Journal of Experimental Child Psychology, 54, 90-119.
Addington, D. W. (1971). The effect of vocal variations on ratings of source
credibility. Speech Monographs, 38, 242-247.
Austin, E. W., & Dong, Q. (1994). Source v. content effects on judgments of
news believability. Journalism Quarterly, 71, 973-983.
Bandura, A. (1994). Social cognitive theory of mass communication. In J.
Bryant & D. Zillmann (Eds.), Media effects: Advances in theory and research
(pp. 61-90). Hillsdale, NJ: Lawrence Erlbaum Associates.
Brooks, B. S., Kennedy, G., Moen, D. P., & Ranly, D. (1992). News reporting
and writing (4th ed.). New York: St. Martin's Press.
Bryant, J., & Zillmann, D. (Eds.). (1994). Media effects: Advances in theory
and research. Hillsdale, NJ: Lawrence Erlbaum Associates.
Burgoon, M., Burgoon, J. K., & Wilkinson, M. (1983). Newspaper image and
evaluation. Journalism Quarterly, 58, 411-419,433.
Carter, R., & Greenberg, B. (1965). Newspaper or television: Which do you
believe? Journalism Quarterly, 42, 29-34.
Chaffee, S. H. (1982). Mass media and interpersonal channels: Competitive,
convergent, or complementary? In G. Gumpert & R. Cathcart (Eds.), Inter/Media:
Interpersonal communication in a media world (2nd ed., pp. 57-77). New York:
Oxford University Press.
Culbertson, H. M., & Somerick, N. (1976). Cloaked attribution - What does it
mean to news readers? American Newspaper Publishers Association News Research
Bulletin, No.1.
Culbertson, H. M., & Somerick, N. (1977). Variables affect how persons view
unnamed news sources. Journalism Quarterly, 54, 58-69.
Eagly, A. H., & Whitehead, G. I. (1972). Effect of choice on receptivity to
favorable and unfavorable evaluations of oneself. Journal of Personality and
Social Psychology, 22 (2), 223-230.
Fedler, F., & Counts, T. (1981). Variations in attribution affect readers'
evaluations of stories. Newspaper Research Journal, 2, 25-34.
Fox, W. (1977). Writing the news: Print journalism in the electronic age.
New York: Hastings House.
Gerbner, G., Gross, L., Morgan, M., & Signorielli, N. (1994). Growing up with
television: The cultivation perspective. In J. Bryant & D. Zillmann (Eds.),
Media effects: Advances in theory and research (pp. 17-41). Hillsdale, NJ:
Lawrence Erlbaum Associates.
Gibson, R., & Zillmann, D. (1993). The impact of quotation in news reports on
issue perception. Journalism Quarterly, 70, 793-800.
Gunther, A. C. (1987). Extremity of attitude and trust in media news coverage
of issues. Unpublished doctoral dissertation, Stanford University, Stanford,
CA.
Hale, F. D. (1984). Unnamed news sources: Their impact on the perceptions of
stories. Newspaper Research Journal, 5 (2), 49-56.
Hamill, R., Wilson, T. D., & Nisbett, R. E. (1980). Insensitivity to sample
bias: Generalizing from atypical cases. Journal of Personality and Social
Psychology, 39, 578-589.
Hass, R. G. (1988). Effects of source characteristics on cognitive responses
and persuasion. In R. E. Petty, T. M. Ostrom, & T. C. Brock (Eds.), Cognitive
responses in persuasion (pp. 141-175). Hillsdale, NJ: Lawrence Erlbaum
Associates.
Himmelfarb, S., & Arazi, D. (1974). Choice and source attractiveness in
exposure to discrepant messages. Journal of Experimental Social Psychology, 10,
516-527.
Homer, P. M., & Kahle, L. R. (1990). Source expertise, time of source
identification, and involvement in persuasion: An elaborative processing
perspective. Journal of Advertising, 19 (1), 30-39.
Hovland, C. I., & Weiss, W. (1951), The influence of source credibility on
communication effectiveness. Public Opinion Quarterly, 15, 635-650.
Izard, R. S., Culbertson, H. M., & Lambert, D. A. (1971). Fundamentals of news
reporting. Dubuque, IA: Kendall/Hunt Publishing Company.
Jo, E., & Berkowitz, L. (1994). A priming effect analysis of media influences:
An update. In J. Bryant & D. Zillmann (Eds.), Media effects: Advances in
theory and research (pp. 43-60). Hillsdale, NJ: Lawrence Erlbaum Associates.
Jones, J. P. (1976). Gathering and writing the news. Chicago: Nelson-Hall.
Jones, R. A., & Brehm, J. W. (1967). Attitudinal effects of communicator
attractiveness when one chooses to listen. Journal of Personality and Social
Psychology, 6 (1), 64-70.
Kelman, H. C., & Hovland, C. I. (1953). "Reinstatement" of the communicator in
delayed measurement of opinion change. Journal of Abnormal and Social
Psychology, 48, 327-335.
LeBouef, R. A., & Matre, M. (1977). How different readers perceive magazine
stories and characters. Journalism Quarterly, 54, 50-57.
Leshner, G. (1994). Switching channels: The effects of channel specialization
and differentiation on judgment, evaluation, and memory of television
information. Unpublished doctoral dissertation, Stanford University, Stanford,
CA.
McCombs, M. (1994). News influence on our pictures of the world. In J. Bryant
& D. Zillmann (Eds.), Media effects: Advances in theory and research (pp.
1-16). Hillsdale, NJ: Lawrence Erlbaum Associates.
McGuire, W. J. (1985). Attitudes and attitude change. In G. Lindzey & E.
Aronson (Eds.), The Handbook of Social Psychology (3rd ed.). Reading, MA:
Addison-Wesley.
Mencher, M. (1994). News reporting and writing (6th ed.). Madison, WI: Brown
& Benchmark Publishers.
Moscovici, S. (1984). The phenomenon of social representations. In R. M. Farr
& S. Moscovici (Eds.), Social representations (pp. 3-69). New York: Cambridge
University Press.
Nass, C., Reeves, B., & Leshner, G. (in press). Technology and roles: A tale
of two TVs. Journal of Communication.
Nass, C., Steuer, J., & Tauber, E. (1994, April). Computers are social actors.
Paper presented to CHI'94 conference of the ACM/SIGCHI, Boston, MA.
Newhagen, J., & Nass, C. (1989). Differential criteria for evaluating
credibility of newspapers and TV news. Journalism Quarterly, 66, 277-284.
Newson, D., & Wollert, J. A. (1985). Media writing: News for the mass media.
Belmont, CA: Wadsworth Publishing Company.
Papa, M. J., & Tracy, K. (1988). Communicative indices of employee performance
with new technology. Communication Research, 15, 524-544.
Petty, R. E., & Cacioppo, J. T. (1981). Attitudes and persuasion: Classic and
contemporary approaches. Dubuque, IA: Wm. C. Brown Company Publishers.
Rosch, E. (1975). Cognitive reference points. Cognitive Psychology, 7,
532-547.
Rosch, E. (1978). Principles of categorization. In E. Rosch & B. B. Floyd
(Eds.), Cognition and Categorization. Hillsdale, NJ: Erlbaum.
Sargent, L. W. (1965). Communicator image and news reception. Journalism
Quarterly, 42, 35-42.
Schramm, W. (1949). The gatekeeper: A memorandum. In W. Schramm (Ed.), Mass
communications (pp. 175-177). Urbana, IL: University of Illinois Press.
Self, C. (1988). Perceived task of news report as a predictor of media choice.
Journalism Quarterly, 65, 119-125.
Sundar, S. S. (1995). Effects of source on the perception and evaluation of
online news. Unpublished doctoral dissertation, Stanford University, Stanford,
CA.
Tversky, A. (1977). Features of similarity. Psychological Review, 84,
327-352.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics
and biases. Science, 185, 1124-1131.
Weaver, D. H., Hopkins, W. W., Billings, W. H., & Cole, R. R. (1974). Quotes
vs. paraphrases in writing: Does it make a difference to readers? Journalism
Quarterly, 51, 400-404.
Wilson, E. J., & Sherrell, D. L. (1993). Source effects in communication and
persuasion research: A meta-analysis of effect size. Journal of the Academy of
Marketing Science, 21 (2), 101-112.
Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences.
American Psychologist, 35 (2), 151-172.
Zajonc, R. B. (1984). On the primacy of affect. American Psychologist, 39
(2), 117-123.
Zillmann, D., & Bryant, J. (1991). Responding to comedy: The sense and
nonsense of humor. In J. Bryant & D. Zillmann (Eds.), Responding to the screen:
Reception and reaction processes (pp. 261-279). Hillsdale, NJ: Lawrence
Erlbaum Associates.
Zillmann, D., & Bryant, J. (Eds.). (1985). Selective exposure to
communication. Hillsdale, NJ: Lawrence Erlbaum Associates.
Zillmann, D., Perkins, Jr., J. W. & Sundar, S. S. (1992). Impression-formation
effects of printed news varying in descriptive precision and exemplifications.
Medien Psychologie, 4, 168-185.
Quotes in Online News
7
|
