Measuring recall
Page
Toward a systematic method of measuring free recall
from printed news stories
Gigi Durham, Ph.D.
Assistant Professor
Department of Journalism
University of Texas at Austin
Austin, TX 78712
(512) 471-1980 (work)/(512) 443-5121 (home)
Paper submitted to the Theory and Methodology Division of the
Association for Education in Journalism and Mass Communication, April
1994
Running head: Measuring recall Toward a systematic method of
measuring free recall
from printed news stories
Gigi Durham
This paper proposes a systematic method of measuring subjects' free
recall from printed hard news stories, based on schema theories of
cognition. Citing literature that demonstrates the role of text
structures and text schemas in the recall of written text, the author
suggests incorporating these processes into the assessment of recall
of
news. In this paper, the first steps toward developing such a
measure
are taken; the procedure hinges on parsing stimulus and response
passages according to a schematic news structure. A pilot study of the
instrument's reliability and validity is included, although further
work
needs to be done to refine the instrument.
Measuring recall
Page
Toward a systematic method of measuring free recall from printed news
stories
In the past quarter-century of mass communication scholarship, a
substantial body of research has focused on audience recall of the
content of news stories. The bulk of these studies deal primarily with
the quantification of news recall (e.g. Booth, 1970; Neuman, 1976;
Gunter, 1980, 1981; Findahl & Hoijer, 1975, 1981, 1985; Edwardson,
Grooms & Pringle, 1976; Edwardson, Grooms & Proudlove, 1981; Edwardson,
Kent & McConnell, 1985; Furnham & Gunter, 1989; DeFleur & Cronin,
1991;
Wicks, 1992; DeFleur, Davenport, Cronin & DeFleur, 1992; Beentjes,
Vooijs & Van der Voot, 1993; Feccorro & DeFleur, 1993). Remarkably, in
each of these and other studies, different measurement instruments
have
been developed to assess subjects' recall of the stimulus news
stories.
Many of these measures are loosely based on standard tests of aided
and
unaided recall, also referred to as recognition and recall. While
these
measures have each been shown to be reliable within the discrete
contexts of the studies, no attempt has been made, thus far, to develop
a theoretically derived, reliable, valid and systematic method of
accurately measuring memory for news that could be used with consistent
results across a variety of experimental methods.
The goal of this paper is to present the first steps in developing such
a measure .
The method presented in this paper is based on the
information-processing model of schema theory.
Schema Theory and the Recall of Printed Text
Research indicates that recall is an interactive operation in which a
reader brings his/her prior knowledge and beliefs into play while
encoding new information and later activates that knowledge for
retrieval of the information (e.g. Bobrow & Norman, 1975; Schank &
Abelson, 1977; Pearson, Hansen & Gordon, 1979; Graesser & Nakamura,
1982; Freebody & Anderson, 1983; Stahl & Jacobson, 1986). The
theoretical position on which these findings are predicated is known as
schema theory.[1]
Schema-based models of information processing have led to increased
understanding of the recall of written text. Central to the
development
of such models is the evolution of the construct of the schema as a
paradigm for cognition.
The psychological concept of the schema emerged initially as a reaction
to the traditional associationist models of memory and learning
(e.g.
Ebbinghaus, 1964), in which recall occurred simply as a response to a
stimulus. The associationist model gradually gave way to the trace
theory of mental representation, which evolved into schema theory as we
know it today. (For a more complete account of the history of
schema
theory, see Hastie, 1981). The notion of a schema was first used in
studies of memory and remembering and later applied in the study of
reasoning, learning, language processing, problem solving, reading, and
countless other cognitive and psychosocial processes.
In general terms, a schema may be defined as a dynamic, generic mental
framework for the hierarchical representation of knowledge.
Anderson
(1977) asserts, "A schema represents generic knowledge; that is, it
represents what is believed to be generally true of a class of things,
events, or situations" (p. 2). Graesser and Nakamura (1982), in an
extensive exposition on the role of schemas in comprehension and
memory,
define schemas as "generic knowledge structures that guide the
comprehender's interpretations, inferences, expectations, and attention.
A schema is generic in that it is a summary of the components,
attributes and relationships that typically occur in specific exemplars"
(pp. 60-61).
In the schematic view of cognition, incoming information is encoded and
stored via an appropriate schema or pre-existing mental knowledge
structure.
Thus, in the context of recall of written text, schema theory is
predicated upon the notion that knowledge is organized into dynamic
knowledge structures in the brain that are activated during the reading
process, as well as during other types of cognitive processing. As
Adams and Collins (1977) have observed:
The goal of schema theory is to specify the interface between the
reader and the textDto specify how the reader's knowledge interacts
with and shapes the information on the page and to specify how
that
knowledge must be organized to support the interaction. (p. 5)
In relation to the processing of mass media news messages, several
studies have shown that prior knowledge is vital to comprehension and
recall of news (cf. Findahl & Hoijer, 1981; Findahl & Hoijer, 1985).
In
other words, the existence of a schema for a news topic or for some
other aspect of a news story will improve recall of the story. As
Findhal and Hoijer point out:
Schema theory stresses the organization of earlier knowledge in memory
in general or prototypical schemata, representing standard
situations, events, or structures. Two different kinds of schemata
have been proposed: one deals with knowledge about recurrent
events
and situations . . . ; the other deals with knowledge about the
typical structure of stories . . . . In news comprehension and
recall,
both kinds of schemata (about recurrent events and about the
structure of news items) are probably activated. (1985, p. 390).
In a departure from the strictly numbers-oriented tradition of this line
of research in mass communication, Woodall, Davis, and Sahin
(1983)
proposed a theoretical framework for memory and understanding of news
based on principles of episodic memory and on the trace theory of
memory
and understanding. Another pioneer in this domain, Doris Graber
(1988),
conducted in-depth interviews with 21 subjects to study their
schema-based strategies for processing information gained mainly from
the news media. These theoretical perspectives are strongly tied to
the
schematic model of information processing.
Text Structures and the Reading Process
In his groundbreaking book Remembering (1932), Frederic Bartlett of
Cambridge University, England, proposed that recall of written
material
depends on a reader's schema for the structure of a written passage.
Such structures, also known as text grammars, have been theorized to
be
fundamental in the organization of all written text (Rumelhart,
1975;
Rumelhart, 1977; Mandler & Johnson, 1977; Meyer, 1977a, 1977b; Stein
&
Glenn, 1979; Marshall & Glock, 1979; Taylor, 1980; Taylor & Samuels,
1983; Page & Stewart, 1985; Waters & Hou, 1987; Griffith & Ripich,
1988;
Cook & Mayer, 1988; Troyer, 1992).
News stories possess a structure or text grammar uniquely their own. As
van Dijk (1988a) observed, "[I]n English we may use the term
'news
story,' and this suggests that news might be a special kind of
narrative. Yet, we also know that it differs from the kind of stories
we tell in everyday conversations or in children's books or in
novels.
Hence, we must specify why and how news stories are different." (p.
1).
That news stories follow a set structural pattern is a notion that has
been intuitively acknowledged for many years. In the textbook
Writing
for Mass Communication, Hutchison (1986) points out that all hard
news
stories[2] should have a formal structure, beginning with a lead:
In a good lead, the important things come first. They provide the
umbrella under which all details of the story will fit
comfortably. .
. . The details usually flow from the lead in order of descending
importance into the succeeding paragraphs. A simple news story
about
a minor traffic accident or a minor house fire will look like an
inverted pyramid. (p. 125).
In another popular journalism textbook, Fedler (1989) describes the
structure of a hard news story thus:
The lead in an inverted pyramid story summarizes the topic, and each
of the following paragraphs presents some additional information
about
it: names, descriptions, quotations, conflicting viewpoints,
explanations, background data and so forth. Most paragraphs are
self-contained units that require no further explanation, and only
the
summary of the entire story appears in the lead. News stories end
with their least important details. (pp. 135-136).
The structuring of hard news stories as inverted pyramids is so
entrenched in the newswriting process that Tuchman (1978) claims that
most news stories consist of prestructured patterns of words into
which
reporters insert "factoids."
The traditional inverted pyramid structure of news stories corresponds
to the concept of a schematic structure or text grammar. Van Dijk
(1983,
1988a, 1988b) observed that a news story can be viewed in terms of
schematically structured discourse. Using cognitive models, van Dijk
examined media discourse and its representation in memory, and his
analysis of hundreds of international newspaper stories led him to
postulate an underlying structure for printed news:
The overall organization of news discourse reflects the importance of
macrostructures. These will typically be expressed by titles or
headlines, by initial or final summaries, or by leads . . . The
lead,
often printed in bold type . . . will express, in a first few
sentences (which are, by definition, "thematic sentences"), the full
macrostructure of the news discourse. Following sentences will
then
progressively specify further details of the events, with the
less
important ones at the end (with the practical consequences that
these
can, if necessary, be cut by the editor). Unlike argumentatively
structured discourse, such as the scholarly paper, where the
important
conclusion comes at the end, news in the daily press is organized by
the principle of relevance or importance, along a dimension of
decreasing prominence with respect to the macrostructure. (van Dijk,
1983,
p. 35).
Newsom and Wollert (1988, p. 120) assert that most hard news stories
have the following elements:
1. The lead (the main point)
2. Secondary points in a tie-in transition
3. Elaboration on the main point
4. Support for the lead
5. Background
6. Development of the main idea
7. Details
They offer two diagrammatic representations of the inverted pyramid: the
"traditional" inverted pyramid (Figure 1-1) and the "modified"
inverted
pyramid (Figure 1-2), represented on the following pages.
LEAD: Who, what, when, where, why & how (16 to 25 words)
TIE-IN: One sentence connecting one element of the lead to the body
BODY: Development of the most important WWWWWH elements of lead
Second most important element of WWWWWH
Further development of most important element
Other elements
The least important facts in the storyDnothing new introduced
[--- Pict Graphic Goes Here ---]
Figure 1-1. Newsom and Wollert's modified inverted pyramid
LEAD: Major theme, could be significance of event, rather than fact
May be two sentences
May not include 5W & H
TIE-IN: The leftovers of the 5W & H not mentioned in the lead
1ST GRAPH: Explication of the lead incident, quote, meaning or
background of eventDhow something came to be
2ND GRAPH: Additional information about most important fact of lead.
Something to give credibility or significance to lead information
3RD GRAPH: Secondary theme or supporting documentation for the lead
4TH GRAPH: Any other details, in order of significance to lead
[--- Pict Graphic Goes Here ---]
Figure 1-2. Newsom and Wollert's modified inverted pyramid
Each element in these structures roughly corresponds to the concept of a
"basic node" in the Mandler and Johnson story grammar (Mandler &
Johnson, 1977)Dthe elements comprising the surface structure of a story,
comprised of a sentence or several sentences. The basic nodes
govern
the way a story is written or rewritten after having been recalled.
For the purposes of this study, Newsom and Wollert's modified inverted
pyramid structure will be adopted as the typical structure for a
breaking hard news story in an American newspaper. The modified
inverted pyramid includes quotes and the possibility of a secondary
theme in the story, which more complex news stories often contain; the
traditional pyramid does not accommodate these elements. In this
respect, the Newsom and Wollert pyramid is a more useful descriptor than
the Fedler pyramid, which makes a provision for quotes but not for a
second theme within a story.
Defining Recall
Recall is the dependent variable of interest in this study. Van Dijk
(1987) observes that "one result of understanding a text is a
representation of the meaning of the text in (episodic) memory" (p.
165), the direct implication being that text comprehension always
results in the storage of information in long-term memory for later
retrieval, i.e. text comprehension always precedes long-term recall of
text information. Voss (1984) corroborates this notion. He writes
While reading, the individual is assumed to interpret the text
contents in terms of his or her own knowledge, interests, and
attitude. During the interpretive process the individual develops a
representation of the contents of the text. Learning is thus
presumed
to involve the storage of information via the development of the
representation (p. 197) (emphasis added)
As Belli (1986) observes, rival psychological theories have resulted in
very different interpretations of the memory process. The
mechanistic
model of memory, for example, views it as a passive process, whereas
schema-based models regard memory as an active, adaptive operation.
The
latter position will be adopted for the purposes of this study.
The term "recall" possesses different meanings in different contexts.
In the mass communication literature, recall is further
classified into
aided and unaided recall. Facorro and DeFleur (1993) note, "In
unaided
recall, the subject reports to the experimenter all the facts of the
stimulus material that he or she can remember. In aided recall, cues
are provided" (p. 592). British psychologist Martin Le Voi refers to
unaided recall as "free recall" (1986, p. 105) and describes the
process
as happening in a situation where "the subject is free to recall any
items . . . and create and use helpful cues in any way he or she
wishes"
(p. 105). Generally, unaided or free recall means the unprompted
remembrance of information; aided recall, on the contrary, refers to the
process of remembering information in a situation where prompts or
cues
are provided.
In most studies of recall from printed news sources, research questions
are generally framed in terms of long-term benefits to readers
from the
assimilation of information in news stories. Thus, free recall is
of
greater interest than aided recall. The measure proposed in this
paper
is geared to the evaluation of free recall of news story content.
Measuring Free Recall
Free recall is often measured very informally, usually by means of a
request to "write down brief descriptions" of what is recalled
(Gunter,
1980) or requests for lists of facts remembered from stimulus
stories
(Wicks, 1993).
This paper proposes a more formal method of measuring recall from a
stimulus news passage, based on a procedure developed by Meyer
(1975),
adapted by Taylor (1980) and Taylor and Samuels (1985) and later
used by
McGee (1982) for scoring recall of expository (nonfictional) textDa
method similar to the scoring procedure followed by Mandler and Johnson
(1977) for measuring recall of narrative (fictional) text. The
procedure is based on schema theory and the adherence of text passages
to schematic text grammars.
The measurement of recall inevitably involves a comparisonDthe recalled
text is compared to the original stimulus passage, and a recall
score is
awarded based on the similarity of the two.
In a schema-based method, the proportion of elements from the initial
passage recalled per structural node would be calculated. The
recall
score would be based on the overall proportion of the stimulus
passage
recalled. The use of a text grammarDin this case the inverted
pyramid
structureDrenders such a measure more rigorous in its capacity to
account for each significant element of the story, making use of the
reader's schematic processing of the text. In addition, the
sequencing
of elements is built into the structure of the stimulus passage as
well
as that of the recalled passage so that the measure can, if desired,
a
lso provide an assessment of subjects' sensitivity to news
structures.
Since van Dijk (1983, 1988a, 1988b) has established that most printed
hard news stories conform to a specific text grammar (or
"superstructure" in his terms), the first step in developing a
rigorous
measurement instrument is the parsing of a given stimulus passage
according to that grammar.
The stimulus passages is then administered to subjects in the context of
an experiment; afterward, subjects are asked to write down an
account of
the stimulus passage, keeping as close to the original version as
possible.
In scoring, the recalled passage is also parsed according to the news
text grammar, and the number of propositions remembered from each
terminal node in the original text is noted. If less than half of an
original sentence is remembered, the sentence is not counted as
having
been recalled. If about half the sentence is remembered, it is
counted
as 0.5. If more than half of the sentence is remembered, it is
counted
as a full sentence. Depending upon the intent of the study,
propositions recalled outside of their nodes could be scored as though
they had been recalled in the proper sequence, given an alternate
scoring scale, or discounted.
An Example
To test a recall measure based on this system, a sample news story was
administered to 69 undergraduates. Subjects were given
sufficient time
to read the passages; the stimuli were then removed.
Subjects were next asked to perform a series of distractor tasks,
including filling out a multiple-choice questionnaire requesting
demographic information and the West Informal Reading Inventory (West,
1978) to assess their reading level.
The stimulus passage, taken from an Associated Press wire story, was as
follows:
PLAN GIVES BIG CATS BOOST
Breeding program for panthers OK'd
MIAMI (AP)DTen Florida panthers roaming the wilds of South Florida are
about to be chosen for a new life in captivity that may have
important
consequences for the survival of the endangered species.
A captive-breeding program has been approved by federal and state
officials to boost the shrinking panther population from an estimated 30
to 50 in the wild.
"There are certain purists who say, 'Let them die a natural death out in
the wild'," said John Christian of the U.S. Fish and Wildlife
Service.
"We need to consider their views, but on the other hand we are
charged
with halting the extinction of the species and moving toward its
recovery."
"It's no question the population is stressed. When you get down t o a
total population of 30 to 50 animals, you get to the point of facing
the
brink of extinction," said Dennis Jordan, Fish and Wildlife's
Florida
panther coordinator. "We consider we have one viable sustaining
population now in South Florida and none anywhere else."
But some wildlife managers say the Florida panther, a type of cougar, is
in nowhere near the danger of the California condor or the
black-footed
ferret when their entire population was rounded up for captive
breeding.
The goal of the new program is 500 breeding adults in captivity and
three wild colonies in 20 years using high-tech methods such as
radio-telemetry collars and possibly in vitro techniques.
The nocturnal cats, with adults weighing 60 to 120 pounds, are smaller
and darker than most cougars and have a unique tail crook and a
cowlick
in the middle of their backs.
The panther, which hunts deer and smaller game, is a solitary hunter
that needs lots of roomDat least 40 square miles for a female and more
than 200 square miles for each male, with little overlap.
The panther once ranged from Louisiana to South Carolina, but widespread
hunting and urban sprawl have pushed it into the Everglades and the
undeveloped center of South Florida.
Without help, experts estimate, the panther will vanish in 25 to 40
years.
After the distractor tasks, subjects were asked to recall the stimulus
passages and write them down using language as close to the
originals as
possible. Finally, a measure of reading comprehension of the
passage
was administered. This instrument (see Appendix A) was based on
Pearson
and Johnson's description of textually explicit and scriptally
implicit
questions (Pearson & Johnson, 1978) as well as on the comprehension
questions used in Johns' (1988) Basic Reading Inventory, a standardized
informal reading evaluation instrument.
Recalls were scored by parsing the subject-generated passages and
recording the number of text elements recalled from the original from
each node in the appropriate text grammar. An example of one such
recalled passage and its scoring is provided below. PLAN GIVES BIG CATS
BOOST,
Panther breeding OK'd
10 Florida panthers will be taken into captivity in an effort to
increase their population from 30 to 50. By the year 2000 they want 500
in captivity and 3 wild colonies. The cats require 200 square miles
for
males and 40 square miles for females for the nocturnal hunters,
with
little overlap.
This isn't so bad as the California Condor or blackfooted ferret whose
entire population was rounded up for captive breeding.
So-and-so, from the Fish and Wildlife Service, said, "Some people say we
should just leave them alone to die in the wild but we're also
charged
with bringing them back from extinction."
The panther, which once roamed in an area from Louisiana to South
Carolina has declined in population due to hunting and urban sprawl
(and
is now confined to the Everglades and the unpopulated area of
central
South Florida).
The original stimulus passage was parsed as follows:
Node I. (Lead) Ten Florida panthers roaming the wilds of South Florida
are about to be chosen for a new life in captivity that may have
important consequences for the survival of the endangered species. (1
sentence)
Node 2. (Tie-in) A captive-breeding program has been approved by federal
and state officials to boost the shrinking panther population from
an
estimated 30 to 50 in the wild. (1 sentence)
Node 3. (Elaboration of lead) "There are certain purists who say, 'Let
them die a natural death out in the wild'," said John Christian of
the
U.S. Fish and Wildlife Service. "We need to consider their views,
but on
the other hand we are charged with halting the extinction of the
species
and moving toward its recovery." (2 sentences)
Node 4. (Support for the lead) "It's no question the population is
stressed. When you get down to a total population of 30 to 50 animals,
you get to the point of facing the brink of extinction," said Dennis
Jordan, Fish and Wildlife's Florida panther coordinator. "We consider
we
have one viable sustaining population now in South Florida and none
anywhere else." (2 sentences)
Node 5. (Background) But some wildlife managers say the Florida panther,
a type of cougar, is in nowhere near the danger of the California
condor
or the black-footed ferret when their entire population was rounded
up
for captive breeding. (1 sentence)
Node 6. (Development of the main idea) The goal of the new program is
500 breeding adults in captivity and three wild colonies in 20 years
using high-tech methods such as radio-telemetry collars and possibly
in
vitro techniques. (1 sentence)
Node 7. (Details) The nocturnal cats, with adults weighing 60 to 120
pounds, are smaller and darker than most cougars and have a unique
tail
crook and a cowlick in the middle of their backs.
The panther, which hunts deer and smaller game, is a solitary hunter
that needs lots of roomDat least 40 square miles for a female and more
than 200 square miles for each male, with little overlap.
The panther once ranged from Louisiana to South Carolina, but widespread
hunting and urban sprawl have pushed it into the Everglades and the
undeveloped center of South Florida.
Without help, experts estimate, the panther will vanish in 25 to 40
years. (4 sentences)
In the recalled story, the number of sentences remembered from each
terminal node in the stimulus text structure is noted, per the
scoring
system described on page 11. Thus, in the recalled passage provided
above, the first sentence the subject wrote was: "10 Florida panthers
will be taken into captivity in an effort to increase their
population
from 30 to 50." The subject has recalled the first part of the
sentence
but not the second, and therefore is given 0.5 for partial recall of
the
sentence comprising the first node.
In the first sentence, the subject recalled part of the tie inDthe goal
of the program to boost the panther population from 30 to 50 in
the
wild. Again, the subject is given partial credit for recalling about
half of the sentence in the node. (0.5)
The subject's second sentence is "By the year 2000, they want 500 in
captivity and 3 wild colonies." This sentence is not very similar
to
anything in the second node, but it is similar to the sentence in the
sixth node, the development of the main idea. The subject is given
0.5
for partial recall of the sixth node, because in this example
sequencing
is not being considered as part of the recall evaluation.
The subject then wrote, "The cats require 200 square miles for males and
40 square miles for females for the nocturnal hunters, with
little
overlap." This is similar to the second sentence in the seventh node.
Again, the subject is given 0.5 for that sentence.
"This isn't so bad as the California Condor or blackfooted ferret whose
entire population was rounded up for captive breeding." The
subject has
almost exactly recalled Node 5, and is given a full point (1.0) for
this
node.
So-and-so, from the Fish and Wildlife Service, said, "Some people say we
should just leave them alone to die in the wild but we're also
charged
with bringing them back from extinction." This captures part of
Node 3.
The first sentence in this node was almost completely remembered, with
the exception of the name of the quoted speaker, so tha subject
receives
1.0 point. The subject remembered about half of the second
sentence, so
0.5 was awarded. The total score on this node is 1.5.
The subject's final sentence was, "The panther, which once roamed in an
area from Louisiana to South Carolina has declined in population
due to
hunting and urban sprawl (and is now confined to the Everglades and
the
unpopulated area of central South Florida)." This is a complete
sentence from Node 7, so the subject receives a score of 1.0 for this
sentence. The subject's recall score is computed as follows, based on
the
proportion of each node recalled:
Node 1. 0.5/1 = 0.5
Node 2. 0.5/1 = 0.5
Node 3. 1.5/2=0.75
Node 4. 0/2 = 0.0
Node 5. 1/1 = 1.0
Node 6. 0.5/1 = 0.5
Node 7. 2/4 = 0.5
Total score = 0.5 + 0.5 + 0.75 + 0 + 1 + 0.5 + 0.5 = 3.75/7 = 54%
Reliability and Validity of the Measurement Instrument
Validity
"The validity of a measuring instrument may be defined as the extent to
which differences in scores on it reflect true differences among
individuals on the characteristics that we seek to measure" (Selltiz,
Wrightsman & Cook, 1976, p. 169). In other words, the validity of a
measure refers to the extent to which it is a true gauge of the
construct it is supposed to measure. Here, the crucial question would
be whether the recall instrument described above was an accurate
measures of the subjects' memory for the stimulus passage.
Content Validity
Content validity is an estimate of the extent to which the measurement
instrument is an adequate sample of the domain or process being
measured. Content validity is often assessed by submitting the
measurement instrument to the scrutiny of experts, who verify that all
facets of the construct or domain under investigation are
represented in
the instrument. The instruments used in this experiment possessed
considerable content validity because they were derived from the
stimulus passage itself, were constructed following the methods used by
earlier investigators of similar phenomena, and measured recall of
each
structural proposition within each passage. The measure thus
represented an adequate sample of the processes under investigation.
Construct Validity
In this study, the dependent variable "recall" is a construct or
abstraction that describes a trait possessed by the subjectDi.e., the
ability to remember and to understand text. Construct validation
refers
to the process of estimating to what extent the measurement
instruments
measure these latent traits.
Construct validation may be accomplished by means of examining patterns
of correlation of a measure with other validated measures of the
same
trait (convergent validity) and by showing that the trait as measured
by
the instrument in question can be differentiated from other traits
or
constructs (discriminant validity).
Curtis and Jackson (1962) have suggested that high correlations between
measures intended to measure different but theoretically related
constructs provide evidence of convergent validity. In this case,
comprehension and recall of the stimulus passage could be expected to
vary together: they are theoretically related but conceptually
distinct
constructs. The construct validity of the recall measure was thus
estimated by measuring the degree of correlation between subjects'
scores on a measure of comprehension and the measure of recall based on
the same stimulus passage. The Pearson correlation coefficient was
found to be 0.33 (N=69); this correlation was statistically significant
(p=.003).
Reliability
Reliability of a measurement instrument refers to the steadiness of
scores on the instrument. Reliability may be measured in terms of
stability or consistency of scores over time; internal consistency,
sometimes called homogeneityDthe similarity of items within a test or
other instrument; and equivalence, or consistency across different
forms
of the same instrument.
Reliability of the instrument was measured using the alternate-forms
method. Twenty-four undergraduate students were asked to
participate in
the reliability study. The students responded to the various
measures
in the experimental sequence described above. Two days later, the
experiment was repeated with the same class; however, students were
given different stimulus passages on the second day. Thus, they were
effectively given alternative forms of a single test.
A coefficient of stability and equivalence was computed according to the
formula:
rAB = S(A-MA) (B-MB)
SDA SDB
where A represents a subject's score on the first test (Test A) and B
represents the subject's score on the second test (Test B), MA
represents
the mean score on Test A, MB represents the mean score on Test B, SDA
represents the standard deviation of scores on Test A, and SDB
represents
the standard deviation of scores on Test B. (For a more detailed
explanation of the alternative forms method of reliability assessment,
see Walsh & Betz, 1985, pp. 50-51, and Horvath, 1985, pp. 71-85).
The reliability coefficient computed for the recall measure was 0.64.
This rather low coefficient could be attributed to the small size
of the
sample used in the reliability test. Selltiz, et al. (1976) point
out
that low reliability coefficients are not necessarily indicators of
low
validity of measurement instruments (pp. 194-197). They argue that
in
some cases, fluctuations in scores on measurement instruments from
one
test administration to another or even within a test are desirable
in
that tests that produce extremely homogenous results are not as
useful
for making fine discriminations among responses and may in fact
reflect
a high degree of content error. They note that the assessment of
reliability and validity occurs along a continuum from convergence of
scores to divergence, depending on the correlations being computed,
and
that "if a measure can be shown to be reasonably valid . . . it must
ipso facto be reasonably reliable, since a measure with a large error
component could not show such consistent relationships" (p. 197).
Discussion
As noted earlier in this paper, research on recall of printed text has
been greatly influenced by schema theories. In the case of news
stories, there is empirical evidence to show that hard news stories
usually follow a prescribed text structure and can easily be parsed
into
fairly standard sequential elements. It would appear that the
schematic
processing of text is a primary influence in the recall of a printed
news story's content, as it has been shown to be in the case of other
types of printed text. This study represents an attempt to take
advantage of the schematic structuring of hard news stories to devise a
systematic method of assessing memory for news.
This research contributes to the development of a theoretically rigorous
model for understanding the process of recalling news and using
it in
the evaluation of memory for news content. However, this inquiry is
still in its preliminary stages, and a more rigorous investigation of
news text structures and the role of schemas in the processing of
news
messages needs to be undertaken before a standard measurement tool
can
be created. The measurement method described herein needs to be
retested and refined, particularly in terms of its reliability, so that
it can be applied to all hard news stories across a variety of
experimental methods.
Appendix A
The Comprehension Measure
1) How many Florida panthers are left in the wild?
2) What is the goal of the new captive breeding program?
3) Briefly describe how the range of the Florida panther has changed
over time.
4) Two men were quoted in the story you just read. With what federal
agency were they affiliated?
5) To what other North American big cats might the Florida panther be
related?
6) Why is the Florida panther facing extinction?
7) What is meant by the term "wildlife conservation"?
8) Why are conservation efforts critically important in today's world?
9) What was they key point of this story?
10) How well do you feel you understood this story?
References
Adams, M. J., & Collins, A. (1977). A schema-theoretic view of reading.
(Report No. BBN-3548). Urbana, IL: University of Illinois, Center
for
the Study of Reading. (ERIC Document Reproduction Service No. ED
142
971).
Anderson, R. C. (1977). Schema-directed Processes in lanquaae
comprehension. (Technical Report No. 50). Urbana, IL: University of
Illinois, Center for the Study of Reading. (ERIC Document
Reproduction
Service No. ED 142 977).
Bartlett, F. C. (1932). Remembering: A study in experimental and social
psychology (5th ed.). Cambridge, England: Cambridge University
Press.
Beentjes, J.W.J., Voojis, M.W., & Van Der Voot, T.H.A. 1(993).
Children's recall of televised and printed news as a function of test
expectation. Journal of Educational Television, 19, 3-13.
Belli, R.F. (1986). Mechanist and organicist parallels between theories
of memory and science. Journal of Mind and Behavior, 7, 63-86.
Bobrow, G. H., & Norman, D. A. (1975). Some principles of memory
schemata. In D. G. Bobrow & A. Collins (Eds.), Representation and
understanding: Studies in cognitive science, (pp. 131-149). New York:
Academic Press.
Booth, A. (1970). The recall of news items. Public Opinion Ouarterly,
34, 604-610.
Cook, L.K. & Mayer, R.F. (1988). Teaching readers about the structure of
scientific text. Journal of Educational Psychology, 80, 448-56.
Curtis, L. F., & Jackson, E. F. (1962). Multiple indicators in survey
research. American Journal of Sociology, 68, 195-204.
DeFleur, M.L. & Cronin, M.M. (1991). Completeness and accuracy of recall
in the diffusion of the news from a newspaper vs. a television
source.
Sociological Inquiry, 61, 148-166.
Defleur, M.L., Davenport, L., Cronin, M., & DeFleur, M. (1992). Audience
recall of news stories presented by newspaper, computer, television
and
radio. Journalism Quarterly, 69, 1010-1022.
Ebbinghaus, H. (1964) Memory: A contribution to experimental psychology.
New York: Dover.
Edwardson, M., Grooms, D., & Pringle, P. (1976). Visualization and TV
news information gain. Journal of Broadcasting, 20, 373-380.
Edwardson, M., Grooms, D., & Proudlove, S. (1981). Television news
information gain from interesting video vs. talking heads. Journal of
Broadcasting, 25, 15-24.
Edwardson, M., Kent, K., & McConnell, M. (1985). Television news
information gain: Videotex vs. a talking head. Journal of Broadcasting &
Electronic Media, 29, 367-378.
Faccorro, L.B., & DeFleur, M.L. (1993). A cross-cultural experiment on
how well audiences remember new stories from newspaper, computer,
television and radio sources. Journalism Quarterly, 70, 585-601.
Fedler, F. (1989). Reporting for the print media (4th ed.). San Diego:
Harcourt Brace Jovanovich.
Findahl, O., & Hoijer, B. (1975). Effect of additional verbal
information on retention of a radio news program. Journalism Ouarterly,
52, 493-498.
Findahl, O., & Hoijer, B. (1981). Studies of news from the perspective
of human comprehension, Mass Communication Review Yearbook, 6,
379-396..
Findahl, O., & Hoijer, B. (1985). Some characteristics of news memory
and comprehension. Journal of Broadcasting & Electronic Media, 29,
379-396.
Freebody, P., & Anderson, R. C. (1983). Effects of vocabulary
difficulty, text cohesion, and schema availability on reading
comprehension. Reading Research Ouarterly, 18, 277-294.
Furnham A., & Gunter, B. (1989). The primacy of print: Immediate cued
recall of news as a function of the channel of communication. Journal
of
General Psychology, 116, 305-310.
Graber, D. A. (1988). Processing the news: How people tame the
information tide (2nd ed.). New York: Longman.
Graesser, A. C., & Nakamura, G. V. (1982). The impact of a schema on
comprehension and memory. In H. Bower (Ed.), The psychology of
learning
and motivation, 16, 59-109.
Griffith, P.L., & Ripich, D.N. (1988). Story structure in
hearing-impaired learning-disabled and nondisabled children.
American
Annals of the Deaf, 133, 43-50.
Gunter, B. (1980). Remembering television news: Effects of picture
content. Journal of General Psychology, 102, 127-133.
Gunter, B. (1981). Forgetting the news. Intermedia, 9, 4143.
Hastie, R. (1981). Schematic principles in human memory. In E. T.
Higgins, C. P. Herman, & M. P. Zanna (Eds.) Social cognition: The
Ontario symposium: Vol. I Hillsdale, NJ: L. Erlbaum.
Horvath, M. J. (1985). Statistics for educators. Seattle, WA: Special
Child Publications.
Hutchison, E. R. (1986). Writinq for mass communication. New York:
Longman.
Le Voi, Martin E. (1986). Encoding and retrieval in recognition and
recall. In G. Cohen, M. W. Eysenck, & M. E. Le Voi (Eds.), Memory: A
Cognitive Approach, (pp. 104-161). Milton Keynes, England: Open
University Press.
Mandler, J. M., & Johnson, N. S. (1977). Remembrances of things parsed:
Story structures and recall. Cognitive Psychology, 9, 111-151.
Mandler, J.M. (1984). Stories, scripts, and scenes: Aspects of schema
theory. Hillsdale, NJ: Lawrence Erlbaum.
Marshall, N., & Glock, M. (1979). Comprehension of connected discourse.
Reading Research Ouarterly, 16, 10-56.
McGee, L. (1982). Awareness of text structure: Effects of children's
recall of expository text. Reading Research Quarterly, 17, 581-590.
Meyer, B. J. F. (1975). The organization of prose and its effects on
memory. Amsterdam: North Holland Publishing.
Meyer, B. J. F. (1977a). The structure of prose: Effects on learning and
memory and implications for educational practice. In R. C. Anderson,
R.
Spiro, & W. E. Montague (Eds.), Schooling and the acquisition of
knowledge (pp. 179-200). Hillsdale, NJ: Lawrence Erlbaum.
Meyer, B. J. F. (1977b). What is remembered from prose: A function of
passage structure. In R. O. Freedle (Ed.), Discourse production and
comprehension: Vol. 1 (pp. 307-336). Norwood, NJ: Ablex.
Neuman, W. R. (1976). Patterns of recall among television news viewers.
Public Opinion Ouarterly, 40, 115-123.
Newsom, D., & Wollert, J. A. (1988). Media writing: Preparing
information for the mass media (2nd ed). Belmont, CA: Wadsworth
Publishing Co.
Page, J.L., & Stewart, S.R. (1985). Story grammar skills in school-age
children. Topics in Language Disorders, 5, 16-30.
Pearson, P. D., Hansen, J., & Gordon, C. (1979). The effects of
background knowledge on young children's comprehension of explicit and
implicit information. Journal of Readinq Behavior, 11, 201-209.
Rumelhart, D. E. (1975). Notes on a schema for stories. In D. G. Bobrow
& A. M. Collins (Eds.), Representation and understanding: Studies in
cognitive science (pp. 211236). New York: Academic Press.
Rumelhart, D. E. (1977). Understanding and summarizing brief stories. In
D. LaBerge & S. J. Samuels (Eds.), Basic processes in reading:
Perception and comprehension, (pp. 265-303). Hillsdale, NJ: Lawrence
Erlbaum.
Schank, R. C., & Abelson, R. (1977). Scripts plans. aoals and
understanding. Hillsdale, NJ: Lawrence Erlbaum.
Stahl, S. R., & Jacobson, M. G. (1986). Vocabulary difficulty, prior
knowledge, and text comprehension. Journal of Readinq Behavior, 18,
309-323.
Wicks, R. H. (1986, August). Methodological approaches to schema
measurement: Applying 'script' measures to mass media information
processing. Paper presented at the meeting of the Association for
Education in Journalism and Mass Communication, Norman, OK.
Wicks, R.H. (1992). Improvement over time in recall of media
information: An exploratory study. Journal of Broadcasting and
Electronic Media, 36, 287-302.
Woodall, W. G., Davis, D. K., & Sahin, H. (1983). From the boob tube to
the black box: TV news comprehension from an information processing
perspective. In D. C. Whitney & E. Wartella (Eds.), Mass
communication
review yearbook: Vol. 4, (pp. 173-194). Beverly Hills, CA: Sage.
Selltiz, C., Wrightsman, L. S., & Cook, S. W. (1976). Research methods
in social relations (3rd ed.). New York: Holt, Rinehart, & Winston.
Stein, N. L., & Glenn, C. G. (1979). An analysis of story comprehension
in elementary school children. In R. O. Freedle (Ed.), Advances in
discourse processes: New directions in discourse processing: Vol II
(pp.
53120). Norwood, NJ: Ablex.
Taylor, B. M. (1980). Children's memory for expository text after
reading. Reading Research Ouarterly, 15, 399411.
Taylor, B. M., &-Samuels, S. J. (1983). Children's use of text structure
in the recall of expository material. American Educational Research
Journal, 20, 517-528.
Tuchman, G. (1978). Making news. New York: Free Press.
Waters, H.S., & Hou, F.-T. (1987). Children's production and recall of
narrative passages. Journal of Experimental Child Psychology, 44,
348-63.
Troyer, S.J. (1992, December). The effects of text structure on fifth
graders' comprehension. Paper presented at the annual meeting of the
National Reading Conference, San Antonio, Texas.
van Dijk, T. A. (1983). Discourse analysis: Its development and
application to the structure of news. Journal of Communication, 33,
20-43.
van Dijk, T. A. (1987). Episodic models in discourse processing. In R.
Horowitz and S. J. Samuels (Eds.), Comprehending oral and written
language, (pp. 161-196). San Diego, CA: Academic Press.
van Dijk, T. A. (1988a). News as discourse. Hillsdale, NJ: Lawrence
Erlbaum.
van Dijk, T. A. (1988b). News analysis: Case studies of international
and national news in the press. Hillsdale, NJ: Lawrence Erlbaum.
Voss, J. F. (1984). On learning and learning from text. In H. Mandl, N.
L. Stein, & T. Trabasso (Eds.), Learning and comprehension of text,
(pp.
193-212). Hillsdale, NJ: Lawrence Erlbaum.
West, E. (1978). The use of an unobtrusive screening device to
approximate reading levels of adults. The 27th Yearbook of the National
Reading Conference.
Walsh, W. B., & Betz, N. E. (1985). Tests and assessment. Englewood
Cliffs, NJ: Prentice-Hall.
[1] In this paper, the word "schema" will be
pluralized as "schemas," per the style
used by Mandler
(1984, p. 2. note 2), rather than the traditional "schemata."
[2] The i
nverted pyramid story structure is characteristic of hard news stories, i
.e.
stories that are factual accounts of events, usual
ly with a time element. Soft news
stories (news stori
es with a human interest focus, written in a lighter vein) or
feature stories often do not follow the inverted pyramid structur
e. The present
research is therefor confined to hard
news stories.
|
