Tracing the Effects of Public Journalism on Civil Society: 1994 -2002
Ph.D. Candidate
Lewis A. Friedland
Professor
Jaeho Cho
Doctoral Student
Hernando Rojas
Doctoral Student
Dhavan V. Shah
Associate Professor
School of Journalism And Mass Communication
University of Wisconsin Madison
821 University Avenue
Madison, WI, 53706
Phone: 608/263-4527
Fax: 608/262 1361
Email: [log in to unmask]
Paper submitted to the Civic Journalism Interest Group of the
Association for Education in Journalism and Mass Communication Annual
Conference
Kansas City, July 30 – August 2, 2003.
Tracing the Effects of Public Journalism on Civil Society: 1994 -2002
Abstract
This study examines 561 cases of public journalism, published between
1994-2002, to address previously identified methodological shortcomings in
the existing public journalism research literature. Using hierarchical
multiple regression analyses, the study traces the effects of
organizational features, particular projects, story frames and roles played
by citizens on improvements in citizenship, political processes and
volunteerism. Specific effects on civil society are discussed, study
limitations are addressed, and insights for future research and practice
are offered.
Tracing the Effects of Public Journalism on Civil Society: 1994 -2002
Public journalism began as a series of experiments in local newspapers in
the late 1980s and early 1990s, and soon developed into what Michael
Schudson has called "the most impressive critique of journalistic practice
inside journalism in a generation" and "the best organized social movement
inside journalism in the history of the American press" (Schudson,
1999).[1] Also known as civic journalism, the movement arose in response to
a perceived crisis in the role of the press in constituting a public sphere
in which citizens could understand and engage productively with the issues
of the day. As it began to grow, it became increasingly more aware of the
range of innovative civic efforts already underway in communities across
the country. The movement has since generated an impressive array of
innovative practices in newsrooms and communities, as well as an extensive
network of practitioners, educators and organizations committed to
reshaping professional and institutional norms.
The philosophy of public journalism, as manifest in the writings of its
leading theorists and practitioners, can be summarized in the following
terms (Merritt, 1998; Rosen, 1999; Sirianni & Friedland, 2001):
1. Journalists must assume responsibility for helping to constitute vital
"publics" with the usable knowledge that enables them to deliberate about
complex issues and to engage in collective problem-solving.
2. Since journalists invariably frame and narrate the story of our common
life in their reporting of "the facts," they should do so with an eye to
how their stories permit people to build knowledge and see themselves as
citizens, rather than as mere spectators, victims, or consumers of
information.
3. While they should not compromise their objectivity through advocacy
journalism, or by taking the lead in developing solutions to problems, they
can play convening and catalytic roles that bring citizens together to
deliberate among themselves, and with those who hold positions of power, so
that citizens may help fashion strategies and responses.
4. They can shine a comparative spotlight on "solutions" that seem to have
worked reasonably well in other communities, in order to expand citizens'
knowledge of potentially useful models and to generate a sense of efficacy.
5. They should not advocate for specific models or succumb to feel-good
news and superficial optimism. Indeed, civic journalists can be tough on
those in power by challenging them to respond to citizens' own agendas and
real life concerns and by holding them accountable to their commitments.
6. Civic journalists must hold citizens themselves accountable for
grappling with the full complexity of issues and acting responsibly to
solve common problems. The emphasis of public journalism is on the
democratic work of citizens in a self-governing republic.
Still, after more than a decade of practice of public journalism, our
empirical knowledge of whether and how public journalism has met these
goals remains largely based on in-depth case studies. Early literature
tended to focus on the cases of the Wichita Eagle or the Charlotte
Observer, the two Knight-Ridder papers that were the generally acknowledged
seed-beds of the public journalism movement . Subsequent comparative
research expanded upon these initial efforts to other examine other best
cases, especially Madison, WI; Norfolk, VA; San Francisco, CA . These
cases examined changes in newsroom reporting and editing practices,
community recognition of public journalism efforts, and shifts in community
problem-solving and public deliberation. Researchers found positive
evidence in each of these areas, but these elements were not disaggregated
and the case studies were often idiosyncratic, making it difficult to
measure change or establish clear relationships among these elements in
more general terms.
In a synoptic review of forty-seven evaluative studies of public
journalism, Massey and Haas found that public journalism practices have
had limited effects on the attitudes, beliefs and behaviors of news
audiences. They criticize existing research for focusing on "a handful of
showcase public-journalism news organizations and projects," and find
methodological shortcomings in much of the existing research (p. 576). In
response, they recommend that future research on public journalism should
capture a wider array of news organization experiments and should attempt
to trace the audience effects of these efforts
Our research, which began in 2000, was designed to address similar
shortcomings, although as our results show, we differ from the Massey and
Haas study in our evaluation of public journalism's success and effects.
Even though one of us had conducted substantial case research on public
journalism , we also felt that broad, quantitative evidence on public
journalism news room organization, framing and reporting practices, and
community level efforts to build on civic problem-solving and public
deliberation were lacking. Indeed, experienced researchers could not even
say with any level of precision how many public journalism projects had
been undertaken. To remedy this gap, the Center for Communication and
Democracy (CCD) of the University of Wisconsin-Madison gained access to the
archives of the Pew Center for Civic Journalism (the Pew Center). We
inventoried the entire archive, and found 651 civic journalism projects
conducted between 1994 and 2002. The initial account of our descriptive
findings was presented as a report to the Pew Center .
Next, collaborating with members of the Mass Communication Research Center
(MCRC) at the UW-Madison, CCD researchers engaged in a more precise
analysis of various features of news organizations, public journalism
projects, story frames, and efforts to involve the public on three civic
society goals: (1) to improve civic skills among citizens, (2) to influence
the policy making process, and (3) to increase levels of civic
volunteerism. Using hierarchical regression analysis, we then trace the
effects of the 651 civic journalism projects conducted between 1994 and 2002.
Literature Review
The literature on public journalism is extensive, but extracting clear
empirical propositions is problematic. First, much of the best literature
on public journalism is normative, advocating that the role of the press is
to improve public life and civic problem-solving. This literature tends to
draw case-based observations about the changes that public journalism
creates in news organizations. Second, while there is a large body of case
literature on public journalism, it is very uneven, ranging from anecdote
and polemic to systematic qualitative and comparative case observation.
Given these inconsistencies, it is difficult to untangle the effects of
public journalism on civil society. Still, there are important insights to
be gained from the extant research on the practice of public journalism,
which we review below.
Research has concentrated on flagship public journalism newsrooms and
projects especially the Wichita Eagle , the Charlotte Observer , and "We
the People" in Madison, WI . There are numerous case studies of other
cities and regional efforts: Akron, OH (; Anniston, AL ; Binghamton, NY ;
Boston ; Dayton, OH ; Florida, ; New Jersey ; Norfolk, VA ; Sirianni &
Friedland, 2001); Rochester, NY ; San Francisco (Thorson, 1996); Spokane,
WA ; and Tampa, FL . However, these cases have not been synthesized in a
systematic comparative framework.
Our discussion is limited to the literature on newspapers. First, we
examine the organization of newsrooms and their effects on individual
journalists' values, norms, and behavior. Second, we discuss how changes in
news content and framing are linked to public journalism efforts. Finally,
we examine how these public journalism practices coupled with efforts to
involve and activate the community affect electoral knowledge and behavior,
and whether and how they engage citizens in civic problem-solving and
public life.
Media Organizations and News Values
Public journalism has changed the way the production of news is organized
in some newspaper newsrooms, but the findings on the degree of change and
the depth at which change has been institutionalized vary widely. There
are, broadly, two types of newsroom studies, studies of newsroom
organization as a whole and studies of journalists' beliefs and attitudes.
While there are no inherent contradictions between these two levels
methodologically, each tends to have different theoretical orientations,
and different understandings of how public journalism is established in
newsrooms. Organizational proponents tend to see the adoption of public
journalism by publishers and editors as the most important predictor of
developed public journalism practice over the long-term . This investment
by news organizations in public journalism shapes reporting routines and
story content, and also accounts for practice over longer periods of time
(3 years or more), in contrast with the episodic coverage of elections or
single issues by newsrooms experimenting with public journalism. In this
view, organization shapes journalists' routines and behavior, which, in
turn, may affect the adoption of public journalism beliefs and values.
However, the critical variable is institutional adoption, not the beliefs
of individuals. The long-term effects on community are likely to be
stronger with organizational investment as well.
An alternative approach looks at the values of journalists, and the effect
of values on attitudes and behavior. Here there are two variants. The
"strong values" approach tends to see the adoption of public journalism
values by journalists as a prerequisite for genuine individual normative
transformation that leads to newsroom change. Proponents, both editors and
scholars , tend to be ardent supporters of public journalism and base their
views on case observation and normative argument. The behavioral approach
holds that positive attitudes toward public journalism in individual
journalists should precede behavioral change. This view is adopted by
Massey and Haas (2002), following Chaffee and McDevitt . Both the
"normative' and "behavioral" variants are consistent in their understanding
of the order of change: change in beliefs and attitudes generates new
public reporting behavior, and it is this behavioral change that underlies
the transformation of news organizations.
Massey and Haas (2002), reviewing 11 studies of the "behavioral" type,
found mixed effects. They found that journalists were most comfortable with
the more "traditional" shadings of public journalism – e.g. helping
communities solve civic and public problems (although arguably, in contrast
with the standards of national flagship papers like the New York Times and
Washington Post, this is already a major movement toward a public stance).
On the other hand, they also found some support for more "activist" roles,
like convening public meetings. It seems that traditional and public
journalism beliefs coexist in many newsrooms as a whole, and within
individual journalists, suggesting an "occupational pragmatism" (pp.
564-565). This is consistent with in-depth case evidence found by Friedland
(2003). But it leaves a major question unresolved. Proponents of both
values and behavioral change suggest that a genuine change of attitudes,
beliefs and behavior at the individual level is necessary for public
journalism news to be produced, which in turn is necessary for civic and
public effects.
Not surprisingly, "mixed-change" characterizes virtually every case, even
the flagship cases of public journalism. This posits a kind of "tipping
point" within individuals and within newsrooms. So newsrooms have either
tipped or not – if not, public journalism practice should be weak. The
macro-view sees institutionalization as a property of the organization.
Organizations make decisions that then mold the actions and routines, if
not the beliefs, of individuals within them. If so, we should see stronger
public effects in newsrooms that have institutionalized public journalism,
here measured by corporate decision to do public journalism and length of
practice. Loomis and Meyer (2000) have found linkages between corporate
culture and civic journalism practice. Our data should shed light on
whether newsrooms that have practiced public journalism three years or more
generate positive changes in content (measured by story frames) and
community level deliberation and problem-solving, regardless of the degree
of value, attitude, and behavioral change among individuals.
News Coverage and Framing
Research on changes in reporting spurred by public journalism is sparse,
with most studies focusing on content or sources, and very few on framing.
The studies on content are contradictory. Blazer and Lembert found very
few changes in content of the Seattle Times deliberative electoral
coverage, with the exception of project logos. Coleman found that public
journalism newspapers provided more opportunities to contact the media and
framed common ground and possible solutions more often than traditional
papers.
For sourcing, Massey found that citizen voices increased to numerical
parity with elite sources in the Tallahassee Democrat, but the
amplification of news voices as measured by directness of quotes was
largely unchanged. Haas found that a project on race in Akron, OH,
included more citizen sources, but used them as background for framing the
perspectives offered by local political elites. As this suggests, citizen
sources may not always reframe the issue that is the focus of public
journalism coverage, especially if elite voices remain dominant.
Using the Meyer and Potter data on 20 cities in the 1996 election,
Loomis found that citizen-based-journalism newspapers produced more
staff-written copy than did traditional papers, suggesting greater focus on
community concerns. Our research may begin to address whether there is a
numerical increase in citizen voices in newspapers, and may also begin to
show what types of stories more clearly show citizen voice. We would expect
that citizen voices would be strongest in those projects engaged in civic
problem-solving, and that they would tend to be used as background in
electoral coverage, except in those electoral projects explicitly designed
to introduce a citizens' agenda.
Effects of Public Journalism
The evidence for public journalism project effects on civic life and public
life is partial and incomplete. Studies divide, broadly, into those
investigating the effect of public journalism on electoral outcomes,
including voter turnout and citizen knowledge and those addressing civic
and public problem-solving. In the largest study of the relation of public
electoral coverage to outcomes, Meyer and Potter research 20 news markets
across the United States surveying citizens before and after the 1996
election and also drawing content samples. They created an index of
"citizen-based journalism" composed of the following elements: sponsoring
public forums, forming citizen panels, conducting focus groups, soliciting
reader questions for candidates, and providing information to help citizens
vote and get involved in the electoral process. For the counties in which
newspapers ranked high on the citizen-based index, they found that citizens
had significantly more knowledge about the election and higher trust in the
media, but there was no significant relation between high civic rankings
and trust in government and political participation.
Similarly, in a study of a constitutional referendum in upstate New York, a
three-way partnership between a Rochester newspaper, a commercial and a
public television station, Bowers et al. found that residents of a
six-county area in which civic coverage was conducted participated in the
state referendum at rates 11% higher than other upstate regions and 46%
higher than New York. Likewise, Denton and Thorson found that the "We the
People" project conducted in Madison, WI – which involved four media
partners coordinating town hall meetings and debates as well as media
coverage – had widespread public awareness and prompted more interest in
political participation, although they caution that untangling political
cause and effect was difficult. Nonetheless, these results suggest that
public journalism partnership can successfully encourage civic volunteerism.
In addition, Thorson et al. found that public journalism electoral
projects increased voter awareness, knowledge, and self-reported
deliberation in San Francisco, Binghamton, Charlotte, and
Madison. However, Blomquist and Zukin , in a study of civic coverage of
national elections in central New Jersey, found few or no effects. On the
whole, studies of electoral effects support a moderate effect of civic
electoral coverage in the areas of voter awareness of issues and
traditional forms of political participation.
There is some evidence that public electoral coverage increases
deliberation and civic problem-solving. Riede found that the 1992 Wichita
Eagle "People Project," one of the earliest public journalism projects to
develop a community-based approach to election coverage, reduced
conflict-laden coverage and adversarial tone, although source diversity was
not significantly expanded. Friedland (2003), however, found significant
long-term effects of the People Project on citizen activism in two Wichita
neighborhoods, and on citywide neighborhood-based deliberation five years
after its publication, with both neighborhood and citywide leaders
crediting the project with opening substantial space of public deliberation.
There is also substantial evidence for increased civic problem-solving in
Charlotte, NC, growing from the year-and-a-half-long "Taking Back Our
Neighborhoods" project. City leaders, association heads, neighborhood
leaders and citizens all reported an increase in public awareness of crime
issues, of public deliberation, of boundary crossing among citizens, and of
increased government effectiveness. Also, all community sectors reported
greater awareness of the problems caused by racial boundaries and
segregated housing patterns .
In Madison, WI, Friedland et al. (1998) found evidence that because of its
longevity, the "We the People" project had substantial cumulative impact on
opinion leaders, media cooperation, institutional effectiveness, and
increased political engagement (although on the latter the evidence is
mixed). However, the episodic coverage with little follow-up from each
project meant that the cumulative problem-solving effects of the project
were attenuated. In all, the case-based evidence shows clear and strong
effects of increased civic and public problem-solving in the limited areas
studied. We would expect that our data show some broader and more
generalizable effect of public journalism in this area.
Research Questions
Past theorizing and case studies concerning the practices and effects of
public journalism do not present a clear picture of the consequences of
this shift in coverage for civil society. Scholars have focused on
different levels of analysis – organization, newsroom, story, and citizens
– and attended to a wide range of outcome variables. Results have not been
consistent, though this inconsistency may be a function of community and
organizational factors that are not the focus of a particular case study.
Accordingly, we intend to examine the effects of a wide range of
potentially explanatory variables on assessments of the outcomes of a wide
cross-section of public journalism efforts between 1994 and 2002. Our
research will attempt to answer how various features of news organizations,
different approaches to public journalism projects, the emphasis on
different story frames, and the involvement of the public influence three
broad goals of public journalism: (1) to improve civic skills among
citizens, (2) to influence the policy making process, and (3) to increase
levels of civic volunteerism. Accordingly we offer the following research
questions to guide our analysis:
RQ1: What features of news organizations involved in public journalism
projects appear to influence whether sponsored efforts increase civic
skills of citizens, public input on policy-making, and levels of civic
volunteerism?
RQ2: What features of public journalism projects appear to influence
whether sponsored efforts increase civic skills of citizens, public input
on policy-making, and levels of civic volunteerism?
RQ3: What types of story frames used by journalist involved in public
journalism projects appear to influence whether sponsored efforts increase
civic skills of citizens, public input on policy-making, and levels of
civic volunteerism?
RQ4: What types of involvement by citizens in public journalism projects
appear to influence whether sponsored efforts increase civic skills of
citizens, public input on policy-making, and levels of civic volunteerism?
Methods
Data
This study is based on data collected by the Center for Communication and
Democracy (CCD) at the University of Wisconsin-Madison from the archives of
the Pew Center for Civic Journalism (Pew Center). Since its inception in
1993 as an incubator for the emerging movement, the Pew Center has
collected examples of civic journalism projects that were submitted by U.
S. newsrooms seeking funding for innovative experiments, competing for the
Center's Batten Awards for Excellence in Civic Journalism, and/or for
informal recognition, advice or assistance. While the Pew Center has
supported both print and broadcast civic journalism, and more recently
experiments using Internet technologies, we examined only those projects
submitted as print and Internet projects. Broadcast projects are currently
being evaluated in a separate study.
Over the course of five two-week visits to the Pew Center's offices between
January 2000 and May 2001, CCD researchers systematically examined the
archive for all evidence of civic journalism experiments. Data collection
began by structuring the archive into a set of cases, organized by discrete
projects and year of publication. Each case generally provided descriptions
of the civic journalism experiment in the form of the news organization's
transmittal letters, formal proposals for funding and/or applications for
reward recognition, as well as examples of the published projects. A coding
scheme was developed to capture a rich account of the attributes and
elements of each project, including: information on the primary news
organizations responsible for the project (i.e., circulation, population
served, partnerships, length of involvement with public journalism);
information on the project itself (i.e., length and format of publication,
topic covered); information about the stories used in the project (i.e.,
frames, sources, presentation format, links to civic resources); the civic
tools and practices used to develop the project and involve citizens (i.e.,
focus group and survey research, public deliberative events,
problem-solving activities); and project outcomes (i.e., improvement in
citizenship and deliberative skills, increase in funding and voluntarism,
effects on public policy formation, and the development of new civic
organizations). From this qualitative coding scheme, we gathered all
available data on each project in the archive using a FileMaker database.
After the completion of data collection, CCD researchers developed a
quantitative coding guide to more systematically organize the data into a
SPSS database for analysis. The study sample contains a total of 651 cases
of public journalism conducted between the years 1994 and 2002, involving a
total of 322 media or civic organizations that took lead role in one or
more of the projects.
In the fall of 2002, the CCD, in collaboration with the MCRC at the
University of Wisconsin-Madison, began exploring the relationship, if any,
between public journalism efforts and improvements on citizenship,
political processes, and volunteerism. The unit of analysis for this work
is the civic journalism project undertaken by news organizations nationwide
(N=651). Reductions in sample size will be noticed in the following
analyses due to missing data for some variables included in statistical
models.
Measures
Items coded from the civic journalism projects were used to operationalize
six general clusters of variables: (1) outcomes of the civic journalism
project, (2) features of the news organization, (3) features of the
project, (4) features of the stories, (5) citizen feedback, and (6) citizen
opinion. In the analyses reported in this paper, the outcomes of civic
journalism were used as dependent variables predicted by the other five
sets of variables.
Outcomes of civic journalism. Initially, six items were identified as
outcomes of civic journalism projects: improved citizenship skills,
improved public deliberative processes, raised private funds and donations,
increased level of volunteer efforts, changed public policy, and formed new
public or civic organizations (See Appendix for a detailed coding scheme).
These items were dummy coded with "present" coded as 1, "absent" as 0. The
six items were then examined through factor analysis and three factors
emerged. Based on the results of exploratory factor analysis, three
measures for civic journalism outcome were constructed. First, improved
citizenship is an additive index consisting of two measures of civic
journalism outcomes: improved citizenship skills and improved public
deliberative processes (inter-item correlation=.33). Improved political
process was constructed by adding up two items: changed public policy, and
formed new public or civic organizations (inter-item correlation=.33).
Finally, improved volunteerism is also a two-item additive index consisting
of measures of raised private funds and donations, and increased level of
volunteer efforts (inter-item correlation=.34).[2]
Features of news organization. For news organizations' publication
schedule, daily publication was coded as one and all other scheduling
formats as zero. Level of circulation was measured using a six-point scale
with one representing circulations under 50,000 and six representing
circulations over 5 million. Type of population served was measured on a
four-point scale with one for small to medium size populations and four for
a national audience. Level of involvement in civic journalism represents
the length of time in years the news organization experimented with civic
journalism practices. Partnerships was coded for evidence of other media or
civic organizations involved in the project with "none" coded as 0, "either
media or civic" as 1, and "both civic and media" as 2.
Features of project. Eleven specific categories were identified to code
each project according to the primary topic covered, including community,
crime, diversity, economy, education, environment, health, poverty, youth,
election, and government (See Appendix for detailed definitions). These
items were dummy coded with "present" coded as 1, "absent" as 0. Project
branding was constructed with three items, each used to develop a unique
project identification, including evidence of a formal presentation format,
clearly stated aim of the projects, and guide for reader comprehensiveness
(reliability a=.85). Each item was dummy-coded with 1 being "present," 0
being "absent." Mobilizing information consisted of two items: empowerment
information (to help citizens engage in civic activities) and civic
linkages (contact information for public officials and civic leaders), each
item was dummy-coded with "present" coded as 1, "absent" coded as 0
(inter-item correlation=.73).
Features of stories. Each project was coded according to six frames widely
used in journalistic reporting: investigative frame, conflict frame,
issue-oriented frame, problem-solving frame, human-interest frame, and
historical frame (see Appendix for detailed definitions). These items were
dummy-coded with 1 representing a frame being "present," 0 for "absent."
Citizen involvement. To measure a project's effort to include citizens'
input in the project, each project was coded for evidence of (a) inviting
citizens' to provide feedback on the project and issue, and (b) giving them
a voice in the publication of their community's conversation. These two
variables were dummy-coded with 1 for "present," 0 for "absent." Each
project was also coded for evidence of the news organization's effort to
determine citizen opinion on the project and/or issue including (a) survey
and (b) focus group research. Surveys included conducting scientific or
informal surveys, and the use of surveys provided by other sources. Each
was dummy-coded with 1 for "present," 0 for "absent."
Results
In order to examine the relationships between civic journalism efforts and
potential improvements on citizenship, political process and volunteerism,
we performed hierarchical multiple regression analyses in which features of
the news organization involved in the project, features of the project
itself, features of the stories developed for the project, and the role
played by citizens in the project served as independent variables
predicting our three criterion variables.
Improved Citizenship
Shown in Table 1, the regression model predicting improved citizenship
performed quite well, as it accounted for a total of 52.9% of variance. The
features of the news organization were substantial predictors (20.4% of
variance), with news organizations that served small or medium communities
(_ =-.123, p<.05) and that partnered either with civic organizations or
other media as the most successful (_ =.100, p<.01).
Improved citizenship was also anchored in the actual features of the civic
journalism project (16.4% of incremental variance). The focus on certain
social problems such as poverty (_=.148, p<.001 in our final model) seems
well suited for the purpose of improving citizenship. On the other hand a
focus on negative issues as crime (_ =-.079, p<.05) or individualized
concerns, as health (_ =.117, p<.001), seem to run counter to this objective.
The features of the news stories or prevalent frames accounted for 10.3% of
the variance in the model. Among the frames is seems clear that the
problem-solving frame was the most appropriate to elicit improvement in
citizenship (_ =.259, p<.001). Conversely, the human-interest frame
appeared to produce the opposite effects, reducing the perception of an
increase in civic skills among the audience exposed to public journalism
efforts (_ =-.108, p<.01).
Above and beyond characteristics of the news organization, features of the
project, and selection of story frames, enhanced citizenship seemed to be
contingent on involving citizens in the process of public journalism. In
our model this block accounted for 4.6% of the final variance, with
inviting audience/reader feedback (_ =.146, p<.001) and giving citizens an
actual voice (_ =.213, p<.001) key contributors. In addition, administering
surveys in the community was positively related (_ =.108, p<.01) to
improving citizenship in the community.
Improved Political Process
The regression model to gauge political process improvements – as shown in
Table 2 – accounted for 22.6% of the variance. In this model the features
of the news organization where not as critical as in the previous model,
accounting for 4.2% of the total variance explained. In the final model
only having established a partnership was a significant predictor of
success in this area (_ =.096, p<.05).
The features of the project were substantial predictors of an improved
political process, with 10.8% of the variance explained by this block.
Among this group of variables, community (_ =.124, p<.05) and education (_
=.181, p<.001) topics are positively related with the dependent variable,
while concentrating on the poverty topic is negatively related (_ =-.120,
p<.05).
The features of the stories where also important contributors to the
explanation of an improved political process, with the investigative frame
(_ =.215, p<.001) as the strongest predictor, followed by a problem-solving
frame (_ =.119, p<.05). Conflict, explanatory and human-interest frames
appear to be irrelevant for this purpose, while the use of a historical
frame appears as negatively related (_ =-.120, p<.01).
Engaging citizens with the project or seeking their opinion does not seem
to be particularly consequential for improvements to the political process.
These two blocks only account for .5% of the incremental variance and yield
no significant predictors.
Improved volunteerism
The model to predict levels of volunteerism in the community explained
22.7% of the variance, as shown in Table 3. As was the case for improving
the political process the features of the news organization do not play as
big a role as they do to improve citizenship explaining 3.1% of the
variance. However, establishing a partnership with another news or civic
organization is positively related to increased volunteerism (_ =.108, p<.05).
Features of the project were a substantial predictor of improved
volunteerism, accounting for 13.5% of the variance. A focus on community (_
=.339, p<.001), crime (_ =.204, p<.001) and education (_ =.172, p<.001)
topics worked to increase volunteerism.
From the perspective of the features of coverage, concentrating on
human-interest frames (_ =.154, p<.01) and avoiding historical frames (_
=-.157, p<.001) seems to increase civic volunteerism. This block explains
5.0% of the incremental variance.
Finally it seems that citizens' engagement with the project and seeking
citizen's opinions are not necessary to improve volunteerism. The model
shows a very small contribution of these two blocks (incremental variance
explained 1.1%) with the use of surveys actually being negatively related
to volunteerism (_ =-.124, p<.05).
Discussion
This study is the first to explore a broad range of public journalism
projects, incorporating the entire field of efforts between 1994-2002 and
tracing the effects on civil society. Further, it incorporates multiple
levels of specificity, examining the effects of organizational features,
particular projects, and the story frames favored by journalists on civic
and public life. Our findings, thus, provide the first holistic assessment
of the impact of public journalism on civil society in America and provide
critical insights for future research and practice involving public
journalism efforts.
In particular, we find that the features of news organizations have
considerable implications for the success of public journalism efforts, but
most of these effects are mediated through the structure of the civic
journalism project and the journalistic framing of the issues that are the
focus of these efforts. Two features of the news organization are
repeatedly found to shape the success of these efforts: the publication
schedule of the news paper and partnerships with other news and community
organizations. Of these two only partnerships retained significance in our
final models predicting improved citizenship, political processes and
volunteerism, indicating the centrality of institutional connections
between and among news organizations and civic groups. This seems most true
of public journalism to improve citizenship where these institutional
connections may provide the basis for civic recruitment and a broader
project scope.
In addition, the focus of public journalism projects also appear to hold
some sway over their success. Projects focusing on education, community,
crime, and poverty were found to be related to the achievement (or
failure) of civil society goals. The effects for a particular issue varied
across these goals. For example, focusing on community and education was
related to positive effects on both the political process and civic
volunteerism, whereas a focus on crime was found to have a negative effect
on citizenship, but a positive effect on volunteerism. Likewise a focus on
poverty was found to have a positive effect on the development of civic
skills among citizens, but found to decrease their perceived involvement in
the political process. Future research must work to disentangle these
effects in order to come to terms with the types of issues that produce
outcomes desirable to specific community needs and problems.
Particularly notable are the results regarding the emphasis on certain
story frames in public journalism projects. Our results suggest that
problem-solving frames have the most pronounced effects on efforts by media
organizations to improve citizenship and the political process. In
addition, investigative news frames were also found to be linked with
improvements in the political process. In sharp contrast, however,
human-interest and historical news frames appeared to generally reduce the
success of public journalism efforts at achieving civil society goals,
particularly in relation to citizenship. In the case of human-interest
frames, our results show a reduction in the development of civic skills,
yet an increase in civic volunteerism. In the case of historical news
frames, we observed negative effects on both improvements to the political
process and civic volunteerism. This pattern of results points to the
importance of journalistic choices in framing news stories around certain
persistent themes. Investigative and problem-solving frames would appear
to spur involvement and action, whereas human-interest and historical
frames appear to reduce citizens' sense of involvement with problems
confronting their communities, and potentially, their sense of efficacy for
solving them. This is not to say that there is not a need for frames that
spur long-term reflection on deep seated problems, though this does call
into question the way such stories are organized and presented to the
public. Further, it may be that the cross-sectional nature of most these
case assessments do not allow us to observe the effects of certain story
frames over time.
Finally, efforts to involve citizens in public journalism efforts were
found to have effects on the improvement of civic skills, in particular,
inviting feedback from citizens and giving them a voice in the
community. Further, the use of surveys to gather citizen opinion were also
related to improved citizenship. In total, these results suggest that,
when journalists understand the perspectives of citizens, they are more
able to construct projects that will both involve citizens and develop
their civic skills and capacities. It also may be that the simple act of
asking citizens for their perspective and giving them public forums is
fundamentally mobilizing. We might term this a type of "civic Hawthorne
effect."
This study based on 651 case studies of public journalism efforts, while
comprehensive, is not without limitations. Most notably, the case studies
that function as our units of analysis rely on some self-assessment of
effects by the editors and journalists involved in the civic journalism
efforts for certain variables. This may create some biases. However, these
would seem to be equivalent across all news organizations involved in the
study, thereby rendering the differences we observed meaningful. In
addition, members of the CCD made an effort to validate the outcome
variables in this study which were often self-assessed against real world
indicators of change. The reliability of these self assessments were found
to be well above the threshold for acceptability, further suggesting the
validity of the data.
The implications of this study, even with these limitations, are broad
reaching. They can be used to inform the next generation of public
journalism efforts and to structure research efforts to evaluate their
success of failure. Future research should certainly consider the
longitudinal effects of public journalism projects on civic life. Using
small-N comparative historical methods, researchers might more closely
explore the precise configurations of organization, project and story frame
that lead to the most effect civic and public outcomes. This would
further test the general claims we are making. Finally, there are broad
theoretical implications of our study. If historical frames are
demobilizing, for example, can long-term, deep-seated community problems be
addressed by journalism interested in democracy?
References
Table 1 - Improved citizenship
Block 1
Block 2
Block 3
Block 4
Block 5
Block 1 – Features of news organization
Publication schedule
.145**
.084
.040
.037
.033
Circulation level
.037
.005
.039
.033
.032
Population type
-.205**
-.151*
-.133*
-.122*
-.123*
Involvement with CJ
.088
.043
.041
.055
.056
Partnership
.329***
.214***
.138***
.124***
.100**
Incremental R2 (%)
20.4%***
Block 2 – Features of project
Community topic
.019
-.029
-.040
-.044
Crime topic
-.093*
-.079*
-.082*
-.079*
Diversity topic
.011
-.019
-.053
-.058
Economy topic
-.041
-.058
-.067
-.068
Education topic
.010
-.005
-.017
-.027
Environment topic
-.043
-.039
-.042
-.036
Health topic
-.181***
-.145***
-.124***
-.117**
Poverty topic
.266***
.181***
.158***
.148***
Youth topic
-.026
-.040
-.050
-.044
Election topic
-.010
-.004
.003
.001
Government topic
-.058
-.051
-.054
-.054
Project Branding
.121**
.054
-.036
-.045
Mobilizing Information
.164***
.052
-.040
-.056
Incremental R2 (%)
16.4%***
Block 3 – Features of stories
Investigative frame
-.057
-.029
-.025
Conflict frame
.011
.010
.014
Explanatory frame
.023
.010
.009
Problem-solve frame
.359***
.269***
.259***
Human-interest frame
-.083*
-.121**
-.108**
Historical Frame
-.021
-.003
-.002
Incremental R2 (%)
10.3%***
Block 4 – Citizen engagement
Invite feedback
.163***
.146***
Citizen voice
.213***
.210***
Incremental R2 (%)
4.6%***
Block 5 – Citizen opinion
Surveys implemented
.108**
Focus groups present
.043
Incremental R2 (%)
1.2%**
Total R2 (%)
52.9%
Cell entries are standardized regression coefficients. * p < .05, ** p <
.01, *** p < .001
Table 2 - Improved political process
Block 1
Block 2
Block 3
Block 4
Block 5
Block 1 – Features of news organization
Publication schedule
.106*
.058
.037
.035
.035
Circulation level
-.114
-.096
-.080
-.077
-.073
Population type
.001
-.012
-.019
-.022
-.024
Involvement with CJ
.005
.010
.015
.016
.017
Partnership
.120**
.085
.084
.081
.096*
Incremental R2 (%)
4.2%**
Block 2 – Features of Project
Community topic
.106
.122*
.123*
.124*
Crime topic
.038
.023
.019
.018
Diversity topic
-.079
-.048
-.057
-.053
Economy topic
.092
.055
.053
.056
Education topic
.193***
.176***
.175***
.181***
Environment topic
.119*
.082
.081
.080
Health topic
.060
.041
.044
.042
Poverty topic
-.093
-.122*
-.125*
-.120*
Youth topic
.071
.037
.037
.035
Election topic
.012
.031
.034
.036
Government topic
.095
.035
.038
.039
Project Branding
.013
-.009
-.023
-.021
Mobilizing Information
.119*
.106
.096
.105
Incremental R2 (%)
10.8***
Block 3 – Features of stories
Investigative frame
.209***
.216***
.215***
Conflict frame
.035
.033
.031
Explanatory frame
.044
.041
.041
Problem-solve frame
.126*
.115*
.119*
Human-interest frame
-.070
-.078
-.083
Historical Frame
-.124**
-.120**
-.120**
Incremental R2 (%)
7.1%***
Block 4 – Citizen engagement
Invite feedback
-.005
.005
Citizen voice
.055
.057
Incremental R2 (%)
0.2%
Block 5 – Citizen opinion
Surveys implemented
-.066
Focus groups present
.003
Incremental R2 (%)
0.3%
Total R2 (%)
22.6%
Cell entries are standardized regression coefficients.* p < .05, ** p <
.01, *** p < .001
Table 3 - Improved Volunteerism
Block 1
Block 2
Block 3
Block 4
Block 5
Block 1 – Features of news organization
Publication schedule
-.003
-.013
-.006
-.006
-.005
Circulation level
.159*
.154*
.125
.124
.133
Population type
-.112
-.100
-.102
-.101
-.105
Involvement with CJ
.088
.059
.057
.058
.059
Partnership
.090
.063
.084
.083
.108*
Incremental R2 (%)
3.1%*
Block 2 – Features of Project
Community topic
.296***
.339***
.338***
.339***
Crime topic
.198***
.205***
.205***
.204***
Diversity topic
-.022
.022
.019
.027
Economy topic
-.037
.030
.029
.035
Education topic
.121*
.164***
.163***
.172***
Environment topic
-.019
.032
.032
.030
Health topic
.037
.035
.037
.032
Poverty topic
-.017
.017
.015
.023
Youth topic
-.020
-.002
-.003
-.006
Election topic
.049
.055
.056
.059
Government topic
-.006
.024
.024
.027
Project Branding
-.002
.052
.044
.047
Mobilizing Information
.051
.052
.044
.060
Incremental R2 (%)
13.5%***
Block 3 – Features of stories
Investigative frame
-.017
-.014
-.015
Conflict frame
.021
.021
.017
Explanatory frame
.008
.006
.007
Problem-solve frame
-.030
-.038
-.031
Human-interest frame
.164***
.162***
.154**
Historical Frame
-.159***
-.157***
-.157***
Incremental R2 (%)
5.0%***
Block 4 – Citizen engagement
Invite feedback
-.017
.036
Citizen voice
.016
.020
Incremental R2 (%)
0.1%
Block 5 – Citizen opinion
Surveys implemented
-.124*
Focus groups present
.019
Incremental R2 (%)
1.0%
Total R2 (%)
22.7%
Cell entries are standardized regression coefficients. * p < .05, ** p <
.01, *** p < .001
Appendix
Coding Guide
Project
The project involves an analysis of the civic journalism (CJ) projects in
the archives of the Pew Center for Civic Journalism (Pew Center) undertaken
by news organizations nationwide and submitted to the Center for a Batten
Award (BA), project funding, or any other more informal purpose.
Unit of Analysis
The unit of analysis is the public journalism project undertaken by the
news organization. A project might involve one article or a series of
articles published over any length of time appearing under the same project
name.
BLOCK 1 - FEATURES OF NEWS ORGANIZATION
Publication Schedule: Use the frequency with which the series was, or parts
of the series were, published.
1. Daily consecutive series
0. Other
Circulation Level: Use the circulation figure for each project's primary
news organization according to the Bowker Newspaper Directory, or as given
by news organization in a BA application or project funding proposal.
1. <50,000
2. 50,000 - 200,000
3. 200,00 - 500,000
4. 500,000 - 1 million
5. 1 million – 5 million
6. >5 million
Population Type: Use the "population served" information from the Bowker
Newspaper Directory. If unavailable, use the designated market or city
zone/target market data from Editor & Publisher directory.
1. Small/medium size population
2. Major metropolitan population
3. Regional
4. National
Involvement in Civic Journalism: Measure the degree to which the news
organization has been involved in civic journalism, using number of years.
1. Low – 1 to 2 years
2. Medium – 3 to 4 years.
3. High – 5 plus years.
Partnerships: What type of organization was involved in the partnership
with the primary news organization?
1. None present
1. Media or civic
2. Both
BLOCK 2 - FEATURES OF PROJECT
Topics: Evaluate each project according to the primary topic covered,
evidenced by the primary subject matter, or theme, of the published
project. Although a project might address several themes, use only the most
prominent topic.
Community: Community projects involve efforts to improve the quality of
community or civic life, including efforts to address both current and
anticipated problems. Projects might involve efforts to identify issues and
solutions to current problems, strategic planning for the future, or
reclaim neighborhoods from the grip of crime or poverty. They might also
involve civic renewal efforts, most often through re-engaging citizens in
public life, promoting community conversations, and improving the quality
of civic leadership. Others might be human-interest stories of exemplary
citizens in their community.
Crime and safety. These projects concentrated on efforts to address a crime
and its impact on a community. Some deal with issues of adult violence and
crime and explore ways to stop crime (i.e., community policing programs).
Others deal with public safety issues, such as quality of rental
properties, fire safety and promotion of safety programs. Still others deal
might deal with gun safety and the right to bear arms.
Diversity. Projects covering diversity address race and ethnic relations
and issues of inequality. Some might explore immigration and its impact on
communities, while others examine a community's growing diverse population
and ways to appreciate different cultures.
Economic. These projects explore ways to improve an area's economy,
exploring ways to improve a lagging economy, poor employment conditions.
Some might address land development issues, such as urban growth or
suburban sprawl, and planning strategies for solutions. Others might
address transportation, such as issues of traffic safety and growth-related
congestion. Another economic topic might involve investigations into local
industry and labor relations.
Education. Here projects address the quality of a community's educational
system, while focusing on such issues as overcrowded schools, under-funded
school districts and poor academic performances of students. They might
evaluate the status of the community's educational system by comparing it
to a known successful system elsewhere in the country. Or they might focus
on specific educational policies, such as school choice policies,
evaluating performance of specific schools, and building values and
character development into curricula.
Environment. These projects address community life as it relates to the
quality of the environment, including pollution-related health issues,
public health policies and ways to clean of the community.
Health. These projects focus attention on the health care system,
addressing issues such as insurance, HMOs, healthcare for minority groups.
Some might Another address specific diseases, while others address adult
addictions, from alcohol, drug and gambling abuse, and their social
consequence (i.e. drunk driving and violent crimes associated with such
addictions).
Poverty. Poverty projects address conditions of poverty and the experiences
of those living in poverty. Some might focus on conditions of public
housing or the quality of health care and education for the poor. Others
might explore the quality of social services, conditions of homelessness.
They might also examine welfare programs and the experience of being on
public assistance.
Youth. Youth projects focus on issues specific to young people. They
generally address such youth-related issues as: a range of general social
issues facing youth today, often from the perspective of young people
themselves; juvenile crime, including gang violence; child care issues,
including foster care and adoption ; school violence ; teen sex, including
teen pregnancy and sex education; and alcohol and drug abuse among teens.
Elections. These projects primarily address upcoming local and national
elections, including attempts to move away from traditional election
horserace coverage to more citizen-driven journalism. Here, they might ask
citizens what they consider the most relevant issues impacting their lives,
educate them on the issues and candidates, stimulate public discussion of
the issues and facilitate dialogue between citizens and candidates. Some
might focus on ways to improve their community's electoral processes,
including improving the manner in which communities elect their national,
regional and local representatives.
Government. These projects examine governmental policy and agencies,
examining issues such as general government spending, military spending,
prison spending and conditions, capital punishment and racial profiling.
Often they are designed as formal investigative projects.
Project Branding:
Project presentation: Did the news organization use a structured format for
the presentation of the project that separated it out from the rest of the
news?
1. No, not structured
2. Yes, structured
Stated aim: Is there evidence that the news organization clearly stated the
aim of the project for the reader?
1. Not present
2. Present
Project guide: Did the news organization help readers understand the
evolution of the project through the use of a project guide and/or was
there a schedule of upcoming stories in the series?
1. Not present
2. Present
Mobilizing Information:
Empowerment information: Did the news organization give readers information
that would help them understand the issue? For instance, did it give them
self-help information, how-to-get involved, community resources, reading lists?
1. Not present
2. Present
Civic linkages: Did the news organization give readers the contact
information for civic leaders, officials, experts and civic organizations?
1. Not present
2. Present
BLOCK 4 – FEATURES OF STORIES
Frames:
Evaluate each project for frames used for purposes of telling the story. In
this context, a frame represents an organizing theme around which the story
is told to help the reader make sense of it. A project might involve a
combination of frames to tell the story.
Investigative Frame. An investigative frame is used to tell the story of an
in-depth inquiry into an issue and its results, usually involving a
systematic search for information previously held from the public eye. This
frame is either used as the primary frame to tell an investigative story or
used to support explanatory and problem-solving frames.
Conflict Frame. The conflict frame is used to tell a story from two or more
points of view, emphasizing the oppositional positions of each. This frame,
as opposed to the explanatory frame, typically emphasizes the conflict over
contextual information to help readers understand the complexity of an issue.
Explanatory Frame. An explanatory frame, or issue-oriented frame, is used
to present the story in ways to increase readers' relationship to and
understanding of the issues. As opposed to the conflict frame, which tends
to narrow the story to two or more opposing viewpoints, the explanatory
frame explores an issue, in all its depth and complexity, to provide
readers with the information necessary to thoroughly grasp the issue's
scope, relevance and potential impact on their community and personal lives.
Problem-solving Frame. The problem-solving frame is one that engages the
reader in the process of identifying potential solutions to problems and
issues being explored and, perhaps, participating in the implementation of
solutions. The solutions are usually drawn from either the news
organization's research, the citizens themselves or from other communities
with experience in addressing a similar problem.
Human-interest Frame. The human-interest frame is used to present the human
face of civic and social problems. More specifically, it is used to
increase issue relevance by telling the story through perspective of those
individuals with personal experience with the issue-related problems and
potential solutions. Although it is used in a small number of cases as the
sole frame, it was most often used to support explanatory and
problem-solving frames.
Historical Frame. The historical frame is used to present the issue in
terms of its historical significance, often involving comprehensive
portraits of historical figures, places and events, and elaborate timeline
graphics to demonstrate evolutionary trends.
BLOCK 4 – CITIZEN ENGAGEMENT
Feedback: Did the primary news organization invite any type of citizen
feedback on the issue and/or project through newspaper publication,
surveys, focus groups?
1. No
2. Yes
Citizen voice: Is there evidence that the news organization gave citizens a
voice?
1. Not present
2. Present – but limited use of citizens voices
3. Present – gave citizens a strong voice in story
4. Other
BLOCK 5 – CITIZEN OPINION
Surveys: Is there evidence that the news organization conducted any type of
survey (or used an existing survey) for public opinion purposes?
1. None
2. Yes
Focus groups: Was there any evidence that the project used focus groups to
help identify issues relevant to the community and/or public opinion?
1. None
2. Yes
[1]
Michael Schudson, "What Public Journalism Knows About Journalism, But
Doesn't Know About 'Public'," in Theodore Glasser, ed. The Idea of Public
Journalism (New York: Guilford, 1999), 118-133. The terms "public
journalism" and "civic journalism" evolved from work supported,
respectively, by the Kettering Foundation and the Pew Charitable Trusts.
While most practitioners and critics use them interchangeably, as do we
ourselves, the terms do have somewhat different resonances. The Kettering
Foundation's term is more explicitly grounded in the theoretical traditions
of Dewey and Habermas. Other terms have also emerged, such as
"solutions-based" journalism.
[2] . For variables constructed from less than three items, alpha is not
reported due to its sensitivity to small item counts. In its place, we
report the inter-item correlation, which provides evidence of internal
consistency among items used to construct a scale.
|
