Modeling micro and macro: A multilevel model to predict memory for
television content
Brian G. Southwell*
Paper submitted for the 2003 AEJMC convention
Communication Theory & Methodology Division
*Southwell is an Assistant Professor at the University of Minnesota.
Author contact information
Mailing address: Dr. Brian Southwell
School of Journalism and Mass Communication
111 Murphy Hall
206 Church Street SE
Minneapolis, MN 55455
E-mail: [log in to unmask]
Phone: 612-624-2491
AUTHOR'S NOTE: Data reported in this paper result partially from work
funded by the National Institute on Drug Abuse (contract number
N01DA-8-5063) through a primary contract with Westat, Inc., of Rockville,
Maryland, and a subcontract with the University of Pennsylvania's Annenberg
School for Communication. I also am grateful to Ogilvy & Mather of New
York for providing the media time purchase data used in the
analyses. Robert Hornik of the University of Pennsylvania and David Maklan
of Westat are Co-Principal Investigators for the NIDA project.
A/V NEEDS: If this paper is accepted, I will present these results using a
Microsoft PowerPoint presentation, which will require a computer projector.
MODELING MICRO AND MACRO
MASS MEDIA AND MEMORY TRACES
Modeling micro and macro: A multilevel model to predict memory for
television content
Abstract
Whenever a study engages an array of variables that should involve
different units of analysis, the risk of misleading results
lurks. Questions about memory for media content, for example, invite
investigation of not only variables describing individuals, but also
(relatively speaking) macro-level constructs concerning content. This
paper uses multilevel modeling techniques to avoid basic pitfalls and
predict memory for electronic media content using data from U.S.
adolescents and data regarding nationally available health campaign
advertisements.
MODELING MICRO AND MACRO
MODELING MICRO AND MACRO 4
Modeling micro and macro: A multilevel model to predict memory for
television content
In a crucial issue of Communication Research on units of analysis more than
10 years ago, Price, Ritchie, and Eulau argued that much communication
research lies at an intersection of macro-level theorizing and available
micro-level measurement and could be informed by cross-level or multi-level
approaches. That same issue also included Pan and McLeod's (1991)
recommendation to let the theoretical locus of variance of the dependent
variable in question and the mechanisms hypothesized to account for that
variance determine the appropriate level of analysis. Nevertheless, truly
multilevel approaches are not yet widespread or common in communication
research in general. We return to that call here with an effort to use
multilevel modeling techniques to predict memory for content from a recent
national strategic communication campaign.
Given the role of exposure as an explanation for the presence or lack of
campaign effects (Hornik, 1997), locating and understanding the immediate
imprint of exposure in individuals is a worthwhile endeavor. If one asks
what predicts memory for exposure to media-based campaign efforts, in turn,
there are a variety of candidate explanations that arise. As we might
expect in light of the discussion above, those explanations do not all
reside on the same plane: individual-level explanations, for example,
contrast with explanations that concern some aspect of the media content in
question.
On one level, Cappella has argued that investigation about possible media
effects should begin with consideration of the individual mental processes
and structures that constrain audience member responses. Studying media
exposure among humans, after all, means that biological and cognitive
constraints bound what is possible. While such individual-level
consideration is undoubtedly relevant and useful, nonetheless, all
individual engagement with electronic media also occurs in a social,
cultural, institutional, and organizational context . Certainly, for
example, the simple environmental prevalence of particular media content
should affect individual memory for it in some fashion. What is most
appropriate, then, is to understand memory for media content as the likely
product of a multilevel model of predictors.
Whenever a study, such as the present one, engages a series of variables
that by definition should be located on different planes of measurement,
the risks of misdirected assignment of units of analysis and misleading
results lurk . Bryk and Raudenbush (1988), for example, point out that
education data are routinely analyzed solely at the student level. Such a
move assumes that educational interventions or organizational contexts,
i.e., school-level variables, are constant across all students. Insofar as
effects vary both among students and among contexts, conventional
approaches may be misleading. Similarly, all media campaign content is not
equal, either in terms of general environmental prevalence or in terms of
various content features. In light of those ideas, we will outline and
explicitly assess here a multilevel model of individual memory for
exposure, or what we can call encoded exposure, to a recent national
campaign effort.
Encoded exposure as a dependent variable
Theoretically, an individual can be said to have encoded exposure for any
particular unit of media content when he or she holds a retrievable memory
trace (available upon prompting) that corresponds to that content and
offers some sense of the frequency of past engagement with that
content. Undoubtedly, researchers have identified a plethora of existing
memory systems and types that differ considerably in complexity and nature
in comparison to this basic construct (see Bower, 2000, for a
discussion). Nonetheless, while other aspects of memory also are
noteworthy, this basic concept should be central for the purposes of many
campaign evaluations and offers a reasonable focal point for the present
study.
Given the notion of a minimal memory trace, then, at least two individual
memory performance task options are relevant as potential measures: a
recognition task or a recall task. The two types of memory measures are
related; measures of each often covary . Nevertheless, recognition can be
differentiated from unaided recall of information. We can think about
unaided recall as the ability to offer detail about particular content when
asked an open-ended question at some point after initial opportunity to
engage the content. Recognition, in contrast, is a more basic ability to
respond to a closed-ended question about past engagement with specific
content when presented that content once again. Whereas recall suggests a
relatively high degree of current information salience and accessibility,
recognition involves a somewhat lower standard of past cognitive engagement .
In light of this distinction, recognition-based tasks theoretically should
offer appropriate indicators of encoded exposure. As Lang has argued,
recognition measures likely indicate if the information in question ever
has been encoded, suggesting that such encoding resides at a different
conceptual level than the retrieval ability likely tapped by recall
tasks. While unaided questions may provide a keener sense of what is most
salient to a respondent at the time of interview, measuring recognition
should more precisely and efficiently tap basic encoded exposure . This
paper describes such a recognition-based measure, validates that measure
through a demonstrated relationship between it and the simple environmental
prevalence of media content, and then explores additional predictors of the
variable through multilevel modeling efforts.
Hypothesized individual-level and content-level predictors
There are several individual-level variables that should matter in
predicting encoded exposure for health advertisements that appear on
television. On a simple level, these variables must include individual
television use and other indicators of opportunities for exposure. More
extensive television watching should lead to higher levels of encoded
exposure.
The relevance of an advertisement's topic also should matter. The more
cause to believe that a particular type of media content should be relevant
for an individual, the more likely that individual should be to report
encoded exposure, all else being equal. Two prominent and complementary
social psychological models of persuasion, namely the Elaboration
Likelihood Model (Petty & Cacioppo, 1986a; and the Heuristic-Systematic
Model , offer some relevant insight in this regard. Though minor
differences1 can be enumerated, the two models converge to suggest that
perceived personal relevance motivates effortful processing and should lead
to encoded exposure (by virtue of affecting depth of processing and
facilitating storage in memory).
Increased perception of the personal relevance of a message is associated
with increased thinking about that message . Increased elaboration, in
turn, should be predictive of more enduring possibility for later retrieval
or recognition of the various instances in which a message was encountered
in one's media environment. Variables indicating ostensible personal
relevance of particular media content, then, should positively affect an
individual's encoded exposure to that content. (With regards to anti-drug
advertisements, the focus of the present study, a key indicator of
perceived relevance will be the extent of one's past drug use.)
In addition, conversation with others about the general topic of the
advertisement also should bear a relationship (at least one of association
if not causation) to later reports of encoded exposure. Engagement with
mass media does not occur in a vacuum. Social networks play a role in
shaping a person's initial engagement with such content, their retention of
such engagement, and their action as a result of such engagement
. Accordingly, the degree to which someone has conversations with others
about the general topic of the content in question also should predict
reported exposure in a positive manner.
There are two ways in which conversations that do not necessarily
explicitly refer to particular television content could nonetheless impact
encoded exposure reporting about that content. First, in the present case
of anti-drug advertisements, a person who has engaged such an advertisement
and who then discusses the general topic of drugs with another person might
reinforce their cognitive imprint of the content in question through
activation of related nodes during the course of conversation. Theoretical
backing for this idea lies in Anderson's network model of memory, which
both posits the possibility that repeated activation of certain memory
nodes can reinforce the accessibility of adjacent nodes.
Insofar as information units related to "marijuana" are stored in connected
memory nodes that are activated every time a person encounters the word,
for example, conversation about drugs should arouse or activate not only
nodes directly involved in that conversation, but also nodes where images
of anti-drug advertisements are stored. In this manner, conversation about
the topic should make any stored image of anti-drug advertising more
salient and should increase the likelihood of that person recognizing the
advertisement when it is presented in a survey.
A second possibility is that conversation about drugs provides cognitive
fodder for later processing and recognition of related media content. A
person who has a conversation with another person about drugs in general
might bolster or enrich their schemata with reference to drugs such that
they later engage a particular presentation of drug-related media content
more efficiently than they would have otherwise. In turn, they should be
more likely to report encoded exposure for unit of media content when
presented with it in the future.
At the advertisement content level, both the sheer prevalence of an
advertisement and the formal features of that advertisement should predict
(average) encoded exposure. The justification for including a prevalence
variable in our model is straightforward. Certainly, the simple
environmental prevalence of particular media content should affect
individual exposure to it in some fashion; without such prevalence, we
could not hope for widespread memory of past engagement. Many commercial
entities underscore this point, depicting exposure, for example, as a
function of simple correspondence between the prevalence of content within
an information environment and aggregate availability of individuals to
engage that content .
Beyond the simple environmental prevalence variable noted above, at least
one formal feature of advertisements should affect the degree to which
respondents report encoded exposure. Specifically, what we can call the
context instability of a unit of media content should bear a generally
negative relationship to reported encoded exposure for that media
content. Context instability refers to the degree to which a unit of media
content transitions between distinct depictions of time or space,
transitions that in combination should present significant processing
hurdles for individuals.
Evidence and arguments from a variety of sources highlight the relevance of
depicted transitions between different points in either time or space that
transcend normal human expectations for movement through either of those
dimensions. For example, the limited-capacity approach to understanding
human engagement with mass media, which builds upon earlier work by
Broadbent (1958) and has been posited cogently by Lang , suggests that
individuals are limited in their ability to process media content by
cognitive capacity constraints. The approach suggests that content
sometimes can overload one's processing system, resulting in presented
information not being processed and stored.
At the center of this potential for overload is the frequency of new
information appearance and the processing it demands. While
information-rich presentations can arouse attention under some
circumstances, Lang and others have suggested that formal features of a
message that introduce substantial amounts of new information also can
inhibit processing and later recognition ability. Visual context
instability, then, should affect the memory encoding potential for media
content insofar as it tends to overtax individual processing systems. The
greater the context instability presented, the less encoded exposure we
should expect, all else being equal.
Justification of a formal multilevel approach
By separating analyses into individual-level and advertisement-level
approaches, we could present initial evidence that encoded exposure is
rightly understood as a product of multiple levels of predictors. At the
same time, research on multilevel modeling, e.g., , suggests that
simultaneous estimation of all predictor levels is more appropriate. Also,
there are additional worthwhile analyses to explore. Formal fitting of a
multilevel model will highlight answers to three important questions about
memory for campaign advertisements among adolescents: one regarding the
(hypothesized) multilevel distribution of encoded exposure variance, one
regarding the plausibility of a multilevel model, and one regarding
possible cross-level interactions.
Any theory positing that encoded exposure to media content warrants a
multilevel understanding assumes that a content-level grouping of data
generated to study the phenomenon will account for a significant amount of
the overall variance in the dependent variable. To test that assumption
here, initial assessment of the intraclass correlation2 as it relates to a
specific advertisement grouping will offer a sense of the specific
proportion of total variance in encoded exposure that lies between
advertisements.
Beyond data structure questions, do the various predictors hypothesized
above demonstrate significance when included in a single multilevel
model? To address this question, we need an approach that affords explicit
modeling at two levels of analysis so that the estimated effects of
independent variables at one level of analysis can be adjusted
simultaneously for effects at the other level of analysis. Such an
analysis is presented here.
Lastly, in addition to main effects, advertisement-level predictors may
curtail or attenuate the effects of independent-level variables on encoded
exposure. Fitting a multilevel model will shed light on whether that is
the case. We can assess whether a significant amount of random variation
exists in any estimated individual-level predictor coefficient associated
with initial model estimation. Such random variation in a coefficient is
predictable (potentially) as a function of advertisement-level
variables. A multilevel approach not only estimates individual-level
effects within each macro-level group but also assumes that such
individual-level effects might vary between groups as a function of
macro-level variables. For any such compelling possibilities, we can model
the individual-level coefficient in question as a function of content-level
predictors, i.e., environmental prevalence and context instability.
Methods
Procedure
Beginning in 1999, the National Survey of Parents and Youth (NSPY) has been
funded by the National Institute on Drug Abuse to evaluate federal
government efforts to discourage drug use through a national media
campaign. As a part of those media-based efforts, campaign organizations
placed anti-drug advertisements in national network, cable, and in-school
television programming, as well as in local television programming in over
100 U.S. metropolitan areas. One of the main objectives of NSPY is to
track memory for, and assess the impact of, those advertisements among U.S.
adolescents.
From November 1999 through December 2000, a multistage cluster sample3
representing all U.S. youth ages 9- to 18-years-old and their parents or
caregivers participated in two waves of NSPY. In a first wave, from
November 1999 through May 2000, interviewers administered surveys with
3,312 youth aged 9 to 18 in 2,373 households. From July 2000 through
December 2000, interviews also were conducted with 2,362 youth aged 9 to 18
in 1,726 households. Respondents used touch-screen laptop computers and
headphones brought into their homes by an interviewer to view each question
(or listen to a prerecorded reading of the question) and to respond. For a
complete discussion of the first two waves of the NSPY study, see Hornik et
al. and Hornik et al. .
The first challenge to be met in fitting a multilevel model to NPSY data
was organizational in nature. More than 5,000 adolescents contributed
responses for the two waves of NSPY analyzed. Each respondent contributed
data in response to a series of interview presentations involving up to
four advertisements from the 23 general market advertisements from the
campaign (as discussed in detail below in the measures section). This
situation resulted in a stacked dataset, whereby each respondent
contributed more than one case of advertisement-specific measures.
In order to organize that data into usable form for a multilevel modeling
endeavor, several steps proved useful. First, all cases corresponding to
either non-eligible or non-general-market advertisements were removed from
the dataset. For example, cases involving bogus advertisements that were
shown to NSPY respondents but that did not actually air were
removed. Second, one case was selected randomly from each respondent.4
This move resulted in an initial set of 5,521 cases. After sorting this
data by the name of the advertisement for which a respondent contributed
data, advertisement-level variables for the 23 advertisements then were
merged and linked to the 5,521 cases.
From this original set of 5,521, 9- to 11-year-old respondents and others
with missing values on the main independent variables (reiterated below)
were dropped via listwise deletion from analyses for this study. The
default dataset of 12- to 18-year-old respondents for all analyses in this
paper has an n of 2,623. The resulting data set allowed analysis of both
individuals and of 23 groups of individuals (grouped by advertisement).5
Measures
Dependent variable measurement warrants special attention, given the
multilevel nature of the present challenge. Fortunately, NSPY included a
variety of questions that afford appropriate measurement of encoded
exposure. During each NSPY interview, campaign television advertisements
that had aired in the two months prior to a particular interview were shown
to respondents on the laptop computer used for the interview. Generally,
the interview program played up to four advertisements for respondents,
depending on the number of eligible advertisements. After seeing each
advertisement, each respondent was asked, "Have you ever seen or heard this
ad?" If they responded in the affirmative, they then were asked, "In
recent months, how many times have you seen or heard this ad?" Response
categories were "not at all," "once," "2 to 4 times," "5 to 10 times," and
"more than 10 times." In order to produce a reasonable interval measure,
these categories were recoded into scores of 0, 1, 3, 7.5, and 12.5 for
analysis. "Don't know" responses to the initial question were recoded as
0.5. Summed across the general market advertisements eligible for a
respondent, this recoded question offered an indicator of individual
exposure.
At the individual level, we could assess encoded exposure across the series
of advertisements shown to a person during an interview. Given the
reorganized, multilevel dataset discussed above in the procedure section,
however, it was more useful to look at the number of times a respondent
reported being exposed to one randomly selected advertisement. This
encoded exposure measure offers both individual-level variation, i.e.,
person-to-person variance, and aggregate-level variation, i.e., differences
in mean levels of the measure between different advertisements. Such
variation affords the basis for the multilevel analysis presented here: A
single encoded exposure measure (EXPOSEAD) stands to be analyzed at two
different levels simultaneously in the same multilevel model.
Independent variable measurement also warrants explanation. Four
television use measures (TVUSE, TVPROGS, CABLE, and ONE), a past drug use
indicator (LNUSEDEP), a measure of recent school attendance (MISSCHL), and
at least one conversation variable (DRUGCONV) served as available
indicators for the individual-level variables noted earlier in our
discussion. A brief discussion of each follows below.
Several different NSPY questions in combination offered independent
measures of various dimensions of television use. For example, all youths
were asked, "How much TV do you estimate watching on an average weekday?"
and were offered response categories including "none," "half-hour or less,"
six separate options for one through six hours, and "7 or more
hours." Following that question, youths also were asked for an estimate of
their TV watching during an "average weekend" and were offered categories
including "none," "less than one hour," options for "1 to 2 hours" through
"9 to 10 hours" and "11 or more hours." I combined responses from these
two questions into a weekly estimate of television watching (TVUSE) by
assigning interval-level numbers to each of the categories6, multiplying
the weekday measure by five, and adding the weekday total to the weekend
measure.
In addition, for 12- to 18-year-olds, NSPY also included up to 15 questions
regarding whether the respondent had ever seen particular television
shows. Shows included in each wave of surveys were selected from the list
of primetime and daytime shows (including both general market and highly
watched African-American shows) in which national anti-drug campaign staff
intended to purchase airtime, such as "ER," "Dawson's Creek," and "The
Steve Harvey Show". Respondents who read (or listened to) and answered the
survey exclusively in Spanish were presented with a list of
Spanish-language shows targeted by the campaign. As a result, this measure
also offered an indicator of a respondent's opportunity for engagement with
campaign advertisements by virtue of their engagement with relevant
television content. For analysis purposes, all of the items were
dichotomized into two categories: having "never" seen a show or reporting
at least some past watching. The items then were combined into an additive
index (TVPROGS) that ranged from zero to 15.
Because the ONDCP campaign focused not only on network television, which is
largely available to most American youths, but also on venues such as cable
television and in-school programs such as Channel One, two additional
measures of television use also are useful. In reference to cable
programming, 12- to 18-year-old respondents were asked how often in the
past 30 days had they watched different types of channels: channels focused
on music television, all-sports programming channels, channels with
programming intended primarily for African Americans, or Spanish-language
channels (for those interviewed in Spanish). After converting original
response categories into interval levels7, these measures were added
together to construct an index of relevant cable programming use
(CABLE). In regards to in-school programming, a NSPY question asked of 12-
to 18-year-olds regarding drug-related information available via Channel
One includes the option to report that one's school does not have the
channel. This measure afforded a dichotomous indicator of Channel One use
(ONE). Tendency to miss class (MISSCHL) was measured with a question
asking how many days in the past 30 days one had skipped school. (Because
it serves as a simple indicator of opportunity for in-school exposure,
MISSCHL should bear a negative relationship to EXPOSURE.)
USEDEPTH indicates the depth of an adolescent's past marijuana use,
depending on whether a respondent reported no past marijuana use whatsoever
(USEDEPTH = "1"), previous trial but no regular use (USEDEPTH = "2"), or
any previous instance of regular use (USEDEPTH = "3"). (Because of
skewness in the USEDEPTH distribution, analysis presented below employs the
natural log of the measure, which we can call LNUSEDEP.8)
With regard to conversation about drugs, all youth NSPY respondents were
asked, "In the last 6 months, how often have you and either of your
{parents/caregivers} talked about drugs?" Available response categories
included "Never," "Once," "2 to 3 times," "4 to 5 times," "6 to 10 times,"
and "More than 10 times". Similarly, youth respondents were asked, "In the
last 6 months, how often have you and your friends talked about
drugs?" Similar response categories were offered. For both questions, a
recoded9 measure offered an interval-level indicator of recent drug
conversation frequency. The analyses presented here employs a single
summary measure of drug conversations (DRUGCONV) that is a simple additive
index combining frequency of recent drug conversations with parents or
caregivers and frequency of such conversations with friends.
In addition, because the advertisements in question vary with regard to the
age and race or ethnicity of people depicted, dummy indicators of race and
ethnic groups (AFAM, HISP, and OTHER, in comparison to WHITE as a reference
group) and age (D14to18 and D16to18, in comparison to 12- to 13-year-olds
as a reference group) also were included in the model presented and
relevant interactions were explored.
At the content level, sources beyond NSPY provided measures of
environmental prevalence (GRPS) and context instability (LNCUTS). For
example, a Gross Rating Points (GRPs) estimate for each advertisement, as
reported by campaign contractors based on estimates of the reach and
frequency obtained for each advertisement, served as a reasonable proxy for
the environmental prevalence of a particular advertisement. A GRP is a
conventional unit used by advertisers to measure a population's simple
physical opportunities for exposure to media content and is the product of
underlying estimates of reach and frequency .
Measuring context instability, or the degree to which a unit of media
content depicts different locations in time or space in sequence, offers an
additional challenge. The present study uses as a measure of context
instability the number of cuts per second in a campaign
advertisement.10 Insofar as a cut here is essentially a transition from
one depicted location in time or space to another, that operational
definition should offer a useful measure of the construct described earlier.
Analysis
The family of multilevel models known as hierarchical linear models offers
a reasonable set of tools for the present challenge. Estimation of a
hierarchical linear model (HLM) often is more appropriate than ordinary
least squares regression (OLS) methods because HLM acknowledges a unique
error structure at each level, whereas OLS approaches do not automatically
do so . Such models have been applied to a variety of research problems,
including modeling academic achievement as a function of student and school
variables, e.g., , and understanding individual and neighborhood crime
variables, e.g., . We also should be able to apply them
here. Accordingly, version 5.03 of the HLM program (Raudenbush, Bryk, &
Congdon, 2001), which offers maximum likelihood estimation of hierarchical
linear models, was useful for this study.
The HLM framework directly accommodates the three major issues posed
earlier. The question of whether a multilevel model is more appropriate
than a single-level model, for example, can be addressed by looking at two
types of statistics: intraclass correlation and reliability estimate of
group means. Careful explication of the basic equations underlying these
statistics will facilitate all later discussion and so is quite worthwhile.
HLM 5 allows assessment of the degree to which dependent variable variance
can be decomposed into significant within-group, e.g., individual-level,
and between-group, e.g., advertisement-level, components. Two equations,
adapted from , illustrate this decomposition.
1) Within-advertisement-group model
Yij = b0j + rij
Yij is the encoded exposure score for respondent i in advertisement group
j, b0j is the mean score for the advertisement group, and rij is a random
error for individual i in group j that is normally distributed with mean 0
and variance s2. The within-group variance (s2) will prove useful below.
2) Between-advertisement-group model
b0j = n0 + U0j
In this equation, n0 is the grand mean of encoded exposure and U0j is a
random error term that is normally distributed with mean 0 and variance t.
These two equations parallel a standard one-way random effects ANOVA model
for this situation, in which advertisement group would be considered to be
a random factor with varying numbers of respondents in each
group. Following from these two equations, we can use the within- and
between-group variance components to compute an intraclass correlation with
the following equation, also adapted from .
3) Intraclass correlation
r = t /( s2 + t)
In this instance, the r parameter essentially is an estimate of the
proportion of total variance in encoded exposure that lies between
advertisement groups. A relatively high r value would suggest that a
relatively large amount of the total variance in encoded exposure lies
between advertisements. If a sizable amount of variance can be classified
as lying between advertisements, then we will have further evidence of the
necessity of approaching encoded exposure as a function of multilevel
influences.
Based on these components and the sample size of each group, HLM also
offers easy calculation of a measure of the reliability of an estimated
group mean. For each group, HLM computes a reliability estimate, aj, with
the equation, aj = t /(t + s2/nj), where nj is the sample size for group
j. We then can assess the average reliability of the advertisement group
mean by looking at the value of aj / k, where k is the number of
advertisement groups (23, in the present analysis). If the average
reliability for all groups is relatively high, then we also can have
further confidence that between-group analyses of encoded exposure can be
presented with relatively less concern about potential dependent measure
error (.
Answers to both the second and third research problems posed above also can
draw upon HLM results as useful evidence. Before addressing complex
questions of cross-level interactions, for example, it is crucial to know
first whether a simultaneously estimated two-level model of encoded
exposure composed of hypothesized predictors lends any support to our
speculation about main effects. For this purpose, the HLM 5 program allows
simultaneous estimation of the following two equations (using restricted
maximum likelihood methods to generate parameter estimates and robust
standard errors for those estimates).11
4) Level one model
EXPOSEAD = b0 + b1 (TVUSE) + b2 (TVPROGS) + b3 (CABLE) + b4 (ONE) + b5
(AFAM) + b6 (HISP) + b7 (OTHER) + b8 (LNUSEDEP) + b9 (DRUGCONV) + b10
(D14to15) + b11 (D16to18) + b12 (MISSCHL) + r
5) Level two model
b0 = n00 + n01 (GRPS) + n02 (LNCUTS) + u0
Also, each predictor coefficient is considered to be a function of an
intercept and error term. For example, b1 = n10 + u1.
Beyond these parameter estimations, we also will want to talk about the
degree to which any estimated overall model explains variance in encoded
exposure. A useful and computable statistic for this purpose is the
proportion reduction arising from the introduction of an explanatory model
(relative to the simple two-level model without predictor variables
outlined in equations 1 and 2). This proportion reduction can be
interpreted as an indicator of the strength of the explanatory model and
can be calculated separately for each level of a proposed two-level model
(Bryk and Raudenbush, 1988). Individual-level and advertisement-level
explanatory power, in this framework, can be assessed with the following
equations.
6) Proportion variance reduction for level one
(s2 of model 1) – (s2 of model 2)
(s2 of model 1)
7) Proportion variance reduction for level two
(t of model 1) – (t of model 2)
(t of model 1)
In addition to producing fixed effects estimates to support or overturn
hypothesized relationships, the HLM program also estimates residual
variance components for all of the individual-level predictor slopes
estimated. This information will shed light on the third issue raised
earlier, namely the possibility of cross-level interactions. Indications
of a significant amount of residual variance remaining in the estimated
slope for a first-level predictor will suggest the potential usefulness of
a more extensive model that includes slopes as outcomes.
In such a more elaborate model, second-level predictors would not only
account for differences in group means but also can account for differences
in first-level predictor slopes. Not only b0 but also b1, for example,
might be a function of content prevalence or content features. In that
instance, HLM can produce estimates for the following model: b1 = n10 + n11
(GRPS) + n12 (LNCUTS) + u1. When appropriate, we also can test such
additional models below.
Results
Within-advertisement-group versus between-advertisement-group variance
Decomposition of the variance in EXPOSEAD suggests that a significant and
sizable proportion of the variance lies between advertisements, t = 5.14,
df = 22, p < .01. Drawing upon equation 3 from above and the estimated
values of s2 = 11.07 and t = 1.75, we can see that r = .14. This
intraclass correlation suggests that approximately 14 percent of the total
variance in encoded exposure lies between advertisement groups. In
addition, the average reliability estimate for advertisement-group exposure
means was 0.91, which justifies dependent variable measurement at the group
level. Both findings suggest macro-level influence on memory.
Multilevel model of encoded exposure: Main effects
Table 1.1 summarizes the results of an estimated multilevel model. Both
individual- and advertisement-level explanatory variables were successful
in explaining variance in this context. The extent to which an adolescent
had seen television programming targeted by the campaign, attendance at a
Channel One school, and reported conversations about drugs all bear
positive relationships to encoded exposure, p < .01 for each. In addition,
past drug use bears a negative relationship to encoded exposure, p <
.01. In comparison to 12- to 13-year-olds, 16-to-18-year-old respondents
report less encoded exposure and white respondents report more encoded
exposure than respondents who are not African-American, Hispanic, or
white. Moreover, GRPs predict encoded exposure in a positive fashion and
context instability holds a negative relationship to the dependent
variable, p < .01 for each, as predicted.
Relatively speaking, this model appears to account for a greater percentage
of the explainable between-group variance in encoded exposure than of the
within-group variance (though it is worthwhile to recall that the majority
of overall exposure variance lies at the individual level in this
sample). At the individual level, s2 initially was 11.07 and is 9.61 after
estimation of this explanatory model, resulting in a 13 percent reduction
of variance. At the advertisement level, t initially was 1.75 and is 1.29
after estimation of this explanatory model, resulting in a 26 percent
reduction of variance.
Table 1.1
Multilevel model of encoded exposure (equations 4 and 5)
Variable
B
(predicting group mean)
B
(mean fixed effect)
SE B
df
Level one (n = 2,623)
TVUSE
.01
0.01
22
TVPROGS
.10**
0.02
22
CABLE
.01
0.003
22
ONE
.31**
0.11
22
Race/ethnicity
African-American
.30
0.30
22
Hispanic
-.11
0.20
22
Other
-.77*
0.28
22
LNUSEDEP
-.04**
0.01
22
DRUGCONV
.07**
0.02
22
Age comparisons
14- to 15-years-old
-.15
0.21
22
16- to 18-years-old
-.49**
0.16
22
MISSCHL
-.11
0.08
22
Level two (23 groups)
GRPS
.04**
0.003
20
LNCUTS
-.35**
0.07
20
Constant
-1.35**
0.35
20
Note. Via level two, this model accounts for 26 percent of encoded
exposure variance between groups and, via level one, 13 percent of variance
within groups. The reference groups for racial and ethnic and age
comparisons are whites and 12- to 13-year-old respondents, respectively.
* p < .05. ** p < .01. Robust standard errors are reported, as
recommended by Raudenbush, Bryk, and Congdon (2001), though estimation of
fixed effects without robust standard errors told a similar story.
Beyond such results, however, the non-significant coefficient for TVUSE
warrants attention. Could it be that the relationship of TVUSE to EXPOSEAD
is a function of content-level influences? For some advertisements, the
relationship between TVUSE and EXPOSEAD might be weak enough to dilute the
average reported relationship. For example, a cross-level interaction
between GRPs and TVUSE could have produced the above pattern; without any
prevalence, no amount of TVUSE will produce exposure. We turn to that
possibility next.
Multilevel model of encoded exposure: Cross-level interactions
We can assess the aforementioned cross-level influence possibility by
looking at whether there is significant random variation in the TVUSE slope
that is potentially attributable to an advertisement-level variable. For
example, if we assume that the TVUSE slope itself is a function of n10 +
u1, then we can assess whether u1 significantly differs from zero. Table
1.2 highlights the final estimation of such error terms associated with the
results in table 1.1.
Table 1.2
Variable
Random effect variance component
c2
df12
TVUSE
.0006*
31.44
18
TVPROGS
.007
18.79
18
CABLE
.00006
13.06
18
ONE
.10
14.32
18
Race/ethnicity
African-American
1.01**
45.24
18
Hispanic
.26
17.26
18
Other
.42
11.29
18
LNUSEDEP
.002
17.78
18
DRUGCONV
.004*
32.94
18
Age comparisons
14- to 15-years-old
.49*
34.89
18
16- to 18-years-old
.20
16.45
18
MISSCHL
.03
17.83
18
Constant
1.29
23.30
16
Random effects for individual-level predictors from table 1.1
Note. * p < .05. ** p < .01.
Among other results13, analysis of variance components does point to the
existence of a significant random effect for the TVUSE slope, c2 = 31.44,
df = 18, p < .05. This suggests that there remains between-group variance
in the relationship of TVUSE and EXPOSEAD that we can attempt to model as a
function of level-two predictors. Additionally, table 1.2 also suggests
that significant (and potentially explainable) between-group variance
exists in the relationship of DRUGCONV to EXPOSEAD.
The possibility that both of these individual-level patterns are a function
of macro-level influences is theoretically interesting. Such evidence
could highlight the primacy of campaign information prevalence in
determining the relationship of individual-level variables to reported
campaign exposure. Such evidence also could demonstrate the amplification
or dampening effect of individual variables for content-level influences.
We can test these possibilities by estimating a model that is identical to
the model outlined above except that it also assumes coefficients for
TVNEWS and DRUGCONV to not only be a function of a constant and an error,
but also a function of GRPs and LNCUTS. In other words, we can assess the
usefulness of including b1 = n10 + n11 (GRPS) + n12 (LNCUTS) + u1 and b9 =
n90 + n91 (GRPS) + n92 (LNCUTS) + u9 among the elements to be estimated,
where b1 is associated with the main effect of TVUSE and b9 is associated
with the main effect of DRUGCONV.
If either content-level variable, i.e., GRPs or LNCUTS, is useful in
accounting for variance in the TVUSE slope, for example, then we would
expect the successful level-two predictor to garner a significant
coefficient, e.g., n11 or n12 from the equation above. We would expect a
similar pattern if either GRPs or LNCUTS can account for variance in the
DRUGCONV slope. In addition, the new model including these new terms
should account for even more advertisement-level variance than the model
outlined in table 1.1.
Table 1.3 outlines the results from estimation of this alternative
explanatory model. Results again highlight the predictive power of
TVPROGS, ONE, LNUSEDEP, and age and racial and ethnic comparisons, p < .01
for each. Cross-level dynamics are also now apparent.
Table 1.3
Multilevel model of encoded exposure (with cross-level interactions)
Variable
B
B
(mean fixed effect)
SE B
df
Level one (n = 2,623)
TVUSE
-.02
0.01
20
TVPROGS
.10**
0.02
22
CABLE
.005
0.003
22
ONE
.33**
0.11
22
Race/ethnicity
African-American
.32
0.30
22
Hispanic
-.10
0.20
22
Other
-.86**
0.27
22
LNUSEDEP
-.04**
0.01
22
DRUGCONV
-.01
0.02
20
Age comparisons
14- to 15-years-old
-.13
0.22
22
16- to 18-years-old
-.50**
0.16
22
MISSCHL
-.12
0.08
22
Level two (23 groups)
Prediction of level-one intercept
GRPS
.02**
0.005
20
LNCUTS
-.24**
0.07
20
Constant
-.56
.34
20
Prediction of TVUSE B
GRPS
.001**
0.0001
20
LNCUTS
-.002
0.002
20
Constant
-.02
0.01
20
Prediction of DRUGCONV B
GRPS
.002**
0.0002
20
LNCUTS
-.02
0.01
20
Constant
-.01
0.02
20
Note. Via level two, this model accounts for 49 percent of the encoded
exposure variance between groups and, via level one, 13 percent of the
variance within groups. The reference groups for racial and ethnic and age
comparisons are whites and 12- to 13-year-old respondents, respectively.
* p < .05. ** p < .01. Robust standard errors are reported, as
recommended by Raudenbush, Bryk, and Congdon (2001), though estimation of
fixed effects without robust standard errors told a similar story. (No
probability of p < .01 reported above exceeded .05 in the non-robust
analysis.)
The relationship between TVUSE and EXPOSEAD and the relationship between
DRUGCONV and EXPOSEAD are associated with the environmental prevalence
(GRPS) achieved by a particular advertisement. (LNCUTS is not a
significant predictor in this capacity by conventional standards, though
was marginally significant with regards to the DRUGCONV slope, p =
.05.) In other words, the environmental prevalence of advertisements
either moderates the relationship of particular individual-level variables
or itself is moderated by such individual-level variables in its influence
on encoded exposure. Television use, for example, appears to have a
markedly different relationship with exposure depending on the degree to
which the advertisement in question was prevalent on U.S. airwaves. Figure
1.1 illustrates this relationship.
Figure 1.1
Cross-level interaction (GRPs and TVUSE) to predict exposure
Hours of weekly television
Estimated recent encoded exposure
For campaign television advertisements that received prominent airplay,
individual television use plays a significant role in explaining encoded
exposure. For advertisements receiving little such airplay, however,
individual television use is not an important predictor. We see an upward
slope between TVUSE and EXPOSEAD at high levels of GRPs, whereas the
relationship between TVUSE and EXPOSEAD is essentially flat at the lowest
levels of GRPs.
A similar pattern exists with regard to the predictive ability of past
conversation about drugs. As table 1.3 suggests, the positive relationship
between DRUGCONV and EXPOSEAD is strongest for those advertisements for
which campaign staff purchased or obtained a relatively high degree of
environmental prevalence.
Importantly, inclusion of GRPs as a predictor of the relationship of TVUSE
and DRUGCONV appears to have eliminated any significant random effects
remaining for the coefficients of those two individual-level
variables. While table 1.2 indicated significant variance in the
coefficients initially estimated for each individual-level variable, the
model fit and outlined in table 1.3 resulted in insignificant residual
variance component estimates for TVUSE and DRUGCONV, p > .10 for
each. This evidence again highlights the importance of paying attention to
content-level prevalence differences.
Beyond these findings, however, we also can begin to parse out the
directional nature of the conversation-exposure relationship. At least two
possibilities are plausible. First, it might be the case that encoded
exposure to anti-drug campaign advertisements (which itself is a function
of environmental prevalence) simply tends to generate discussion, which
explains the positive association between the two measures. As noted
earlier, however, there are theoretical reasons to suspect a second
possibility, as conversation about drugs might either sensitize a person's
drug-related media content encoding tendencies or might arouse memory of
past anti-drug advertisements and facilitate later recognition ability
whenever drugs are discussed.
Results presented up to this point essentially go no further than
demonstrating an association between conversation and encoded exposure and
allowing for the reciprocal relationship possibilities. Because of the
simultaneous estimation of both individual- and content-level effects
presented, however, we also should be able to generate an additional piece
of evidence regarding the nature of that conversation-exposure relationship
by looking at the role of environmental prevalence. Specifically, we can
ask whether widespread availability of media content leads to increased
discussion or whether there is no relationship between macro-level
anti-drug advertisement availability and micro-level discussion. In the
first instance, we could view the individual-level conversation-exposure
relationship as essentially a symptom of (or mechanism for) a general
prevalence-conversation relationship. If there is no relationship between
advertisement GRPs and the amount of drug conversation reported by
respondents associated with that advertisement, however, then it will be
reasonable to understand table 1.3 as suggesting that drug conversation
moderates the impact of advertisement GRPs on encoded exposure. We might
think of this phenomenon as a memory trace amplification effect.
Using DRUGCONV as a dependent variable, we can predict the mean level of
drug conversation in advertisement respondent group simply as a function of
GRPS and an error term. (This HLM analysis directly parallels the main
analysis above in which GRPS predicted EXPOSEAD group mean). Results of
this analysis undermine the possibility that reported general drug
conversation is a function of the environmental prevalence of recent
anti-drug advertisements. First, a decomposition of variance suggests that
almost all of the variance in DRUGCONV lies within advertisement groups,
not between them. Only roughly 1 percent (0.48 / 34.98) of the variance in
DRUGCONV lies between advertisement groups. Second, GRPs do not bear a
significant predictive relationship to the intercept of DRUGCONV, B = .007,
SE B = 0.008, df = 21, p > .10. These results suggest that conversations
about drugs between adolescents and their friends and parents do not appear
to be a function of the prevalence of specific campaign advertisements
available during recent months.
In light of this pattern, general drug-related conversation in an
adolescent's immediate social network (at least that network comprised of
friends and parents or caregivers) appears to moderate the degree to which
an anti-drug advertisement's prevalence translates into later memory trace
retrieval. From this perspective, figure 1.2 depicts the cross-level
interaction between GRPs and DRUGCONV in an appropriate manner, not only
reiterating the general positive relationship between GRPs and encoded
exposure but also suggesting that the relationship increases in strength
when the number of drug conversation increases.
Figure 1.2
Cross-level interaction (GRPs and DRUGCONV) to predict exposure
Estimated encoded exposure for ad
Gross ratings points for advertisement
Discussion
On an individual-level, then, encoded campaign exposure among 12- to
18-year-olds in the U.S. appears largely to be a function of their media
habits, general conversation about drugs with friends and parents, and the
extent of their own past drug use (albeit in a different manner than
hypothesized with regard to the last predictor). Age and race differences
also exist, which in part can be explained by targeting efforts on the part
of ONDCP campaign staff.
The present results also offer some important contextual constraints for
our discussion, however. For example, environmental prevalence and content
features strongly predict encoded exposure levels; level two of the final
model presented here accounts for about half of the group-level variance in
encoded exposure. Nonetheless, it is also worth noting that total
between-group variance represents a minority (about 14 percent) of the
overall variance in encoded exposure among 12- to 18-year-old adolescents
in the U.S., albeit a sizable minority. In other words, while we would be
remiss to overlook macro-level effects when discussing encoded exposure
(and in fact have avoided such an oversight here by documenting some
striking macro-level effects), there is a considerable amount of
individual-level variance that remains both outside the domain of
macro-level main effects and unaccounted for by the individual variables
highlighted here.
At the same time, the HLM efforts of the present study also offer more
than simple confirmation or context. Allowing content-level variables not
only to predict mean level of encoded exposure but also either to attenuate
the relationships between individual-level variables and encoded exposure
or to have their own relationships with exposure moderated by
individual-level variables markedly improved the predictive power of the
multilevel model in question. At the advertisement level, initial efforts
accounted for approximately 26 percent of between-group variance in
exposure, whereas an alternative model in which GRPs were allowed to
predict the slopes of TVUSE and DRUGCONV in their relationships with
exposure accounted for approximately one-half of all between-group variance
in exposure. In other words, heeding the possibility for cross-level
interaction resulted in a doubling of second-level predictive power.
If such an approach had not been taken, the cross-sectional nature of the
individual-level measures employed in this study would limit discussion
about the relationship between conversation and encoded exposure. In
contrast, allowing a macro-level measure that theoretically precedes
exposure encoding, i.e., environmental prevalence, to operate in a
multilevel analysis afforded some clarification of the likely nature of the
relationship between individual-level conversation and encoded
exposure. Given the lack of a group-level relationship between GRPs and
general drug conversation reported, past increases or decreases in
advertisement prevalence do not appear to have preceded or (linearly)
motivated recent general drug conversation involving 12- to
18-year-olds. Instead of solely being a product of encoded exposure, then,
conversation, however it arises, appears to enhance memory retrieval
ability for advertisements and also likely facilitates or moderates the
tendency of an advertisement's environmental prevalence to result in
encoded exposure reports. Without multilevel modeling results such as
those highlighted here, such speculation would enjoy less empirical
evidence.
Conclusions
By employing multilevel modeling techniques, this study produced three
types of useful evidence regarding memory for television content among U.S.
adolescents. First, basic variance decomposition confirmed that the
distribution of encoded exposure itself invites a multilevel
understanding. A significant and sizable proportion of exposure variance
can be attributed to between-group differences when respondents are grouped
according to the advertisement about which they were queried in the
selected dataset. Second, an overall predictive model involving a variety
of individual-level and content-level predictors confirms in most instances
both the significance and the nature of the predictive power of each
included variable. Beyond such results, the multilevel models fit
presently also support the hypothesis that advertisement-level variables
can interact with individual-level variables in having a joint effect on
encoded exposure.
In general, then, the results highlighted here confirm that encoded
exposure is rightly understood as a multilevel phenomenon. Importantly,
however, this study also highlights ways in which multilevel modeling
techniques, such as maximum likelihood estimation of hierarchical linear
models, can be useful for approaching communication research questions
involving both individual variables and variables that describe mass media
content. Not only do various successful predictors of encoded exposure
theoretically reside at different levels of measurement, but it also
appears that some of these variables moderate the influence of variables
located at a different level. Individual adolescents in the U.S. exert
some limited influence over their own exposure to mass media campaigns, but
they also appear to be living in a web of influences, ranging from
conversations with others to particular features of media content, that
affect their memory for campaign material in a variety of ways.
Notes
1 Eagly and Chaiken (1993) note, for example, that the HSM permits
heuristic and systematic processing to occur simultaneously and that
heuristic processing, and heuristic cues, can affect systematic
processing. Moreover, the HSM holds that motivational variables can not
only invite systematic processing but also can affect heuristic processing
as well.
2 See for a useful and thorough overview of intraclass correlation and its
relevance to multilevel modeling.
3 The youth and their parents were found by door-to-door screening of a
scientifically selected sample of about 34,700 dwelling units for Wave 1
and a sample of 23,000 dwelling units for Wave 2. These dwelling units
were spread across about 1,300 neighborhoods in Wave 1 and 800
neighborhoods in Wave 2 in 90 primary sampling units. The sample provided
an efficient and nearly unbiased cross-section of America's youth and their
parents. Youth living in institutions, group homes, and dormitories were
excluded. Parents were defined to include natural parents, adoptive
parents, and foster parents who lived in the same household as the sample
youth. Stepparents were also usually treated the same as parents unless
they had lived with the child for less than 6 months. When there were no
parents present, an adult caregiver was usually identified and interviewed
in the same manner as actual parents. Among selected youth, the response
rate was approximately 91 percent in Wave 1 and 92 percent in Wave 2,
meaning that 91 or 92 percent of the youth received parental consent,
signed to their own assent, and completed an extended interview. Among
sample parents, 88 percent completed the extended interview in Waves 1 and 2.
4 This random selection was accomplished by first using both dwelling unit
identification number and roster identification number as grouping
variables (making each respondent into a single group, in other words) and
then randomly selecting one case from each created group. SPSS syntax for
this operation was adapted from the follow SPSS advice web site on February
4, 2002:
"http://pages.infinit.net/rlevesqu/Syntax/RandomSampling/Select2CasesFromEachGroup.txt".
5 The present analyses employ NSPY weights that reflect sample selection
probabilities and compensate for non-response (Hornik et al, 2001). As
present analyses utilize HLM 5, however, replicate weights adjustment
available through other programs, such as WesVarPC, could not be
employed. Accordingly, I emphasize those results with p < .01, as opposed
to results at the conventional .05 level.
6 For both weekday and weekend watching, the "none" category was assigned
"0". For weekday watching, "half-hour or less" was assigned ".5" and, for
weekend watching, "less than one hour" also was assigned ".5". For weekday
watching, the "about 1 hour" through "about 6 hours" categories were
assigned "1" through "6", respectively. The "7 hours or more" category was
assigned "8" for weekday watching. For weekend watching, the "1 to 2
hours" through " 9 to 10 hours" categories were assigned "1.5", "3.5",
"5.5", "7.5", and "9.5", respectively, and the "11 hours or more" category
was assigned "12".
7 The original NSPY questions asked how often the respondent had watched
each of the following in the past 30 days: "a music television station,
such as MTV, VH1, or TNN (The Nashville Network)", "an all-sports channel,
such as ESPN", or "a channel focused on African Americans or Blacks such as
BET." Spanish-language interviews also asked how often one had watched "a
channel especially for Latinos or Hispanics such as Telemundo, Univision,
or Galavision" in the past 30 days. Original response categories included
"never", "1 to 4 days", "5 to 14 days" and "15 to 30 days" and were
assigned the interval levels of "0", "2.5", "9.5", and "22.5", respectively.
8 The mean of USEDEPTH was .31, SD = .63, skewness = 1.84. This reflects
the fact that most youth report no past marijuana use, though a small
number report past regular use. The positive skew suggested the usefulness
of a variable transformation. The natural log of USEDEPTH (which we can
label LNUSEDEP) demonstrated much less skew than the original variable and
was useful for analysis. The skewness of the LNUSEDEP distribution was
1.37 (mean = -8.94, SD = 4.88).
9 "Never" was recoded into 0, "Once" was recoded into 1, "2 to 3 times" was
recoded into 2.5, "4 to 5 times" was recoded into 4.5 times, "6 to 10
times" was recoded into 8, and "More than 10 times" was recoded into 12.
10 A cut is a transition to a different camera perspective that results in
the depiction of a new visual environment or entirely new visual
information. The following rules further clarify that notion. Any
transition to a new physical environment (one that is not visible in, or
contiguous with, the previous shot) counts as a cut. A transition to a
close up of a face (at least 1/5th of the screen) also counts as a cut,
even if the face was partially visible in the establishing shot and the
same environment is depicted. This idea is based on Lang's (personal
communication, 2001) recommendation. If transition depicts the exact same
room but results in the depiction of an entirely new face in the same room,
it will count as a cut the first time that the person (or people) in
question appears. Each subsequent repetition of the person will not count
as a cut (unless, of course, the environment has changed between shots of
the face and the shot with a face now represents a cut from a different
visual environment). If the same people are depicted in the exact same
room in a sequence of shots that could not physically have occurred without
editing, e.g., alternative versions of the same scenario, the first
transition to a repeated scenario will count as a cut. Each subsequent
repetition in the uninterrupted sequence will not count as a cut. Any
transition from whole screen to split screen with different environments
depicted is cut. Each new introduction of new scene in each separate
screen (in case of split screen) is a cut. Transition to whole screen from
split in which one of the scenes is enlarged to become the whole screen is
an edit and not a cut. Transition to black (or other color) screen with
text is a cut. Transition from one line of text to another, however, is an
edit and not a cut. Special effects allow for some transitions in which
only a part of a screen display changes, e.g., an abstract image changing
one-fourth at a time. In these cases, at least half of the total screen
area needs to change to a new image in order to constitute a cut.
11 Robust standard errors are consistent even ordinary least squares
assumptions about constant variance of outcomes across groups are incorrect.
12 The degrees of freedom are equal to 18 in this instance because only 19
of the original 23 groups had sufficient data for HLM computation of c2 to
test random effects. Reported fixed effects and variance components,
nonetheless, are based on all data.
13 One age and one race comparison also suggests significant random effects
in table 1.2. None of the content-level variables used for this study,
however, produced an alternative model that reduced this additional random
coefficient variance for age or race significantly. Future investigation
of different content-level variables might account for this coefficient
variation.
References
|
