This paper was presented at the Association for Education in Journalism and
Mass Communication in San Antonio, Texas August 2005.
If you have questions about this paper, please contact the author
directly. If you have questions about the archives, email
rakyat [ at ] eparker.org. For an explanation of the subject line,
send email to
[log in to unmask] with just the four words, "get help info aejmc," in the
body (drop the "").
(Feb 2006)
Thank you.
Elliott Parker
====================================================================
A Benchpoint Global Analysis of How Research is
Used in Public Relations Throughout the World
Donald K. Wright, Ph.D.
Professor of Communication
University of South Alabama
1000 University Commons
Mobile, AL 36688-0002
Tel: 251-380-0850
[log in to unmask]
and
Michelle Hinson, M.A.
Director of Development
Institute for Public Relations
School of Journalism and Communications
University of Florida
PO Box 118400
Gainesville, FL 32611-8400
Tel: 352-392-0280
[log in to unmask]
A paper
Submitted for Consideration by the Research Committee
Public Relations Division
Association for Education in Journalism and Mass Communication
March 2005
A Benchpoint Global Analysis of How Research is
Used in Public Relations Throughout the World
Research has been an important part of public relations for more
than half a century. In many ways, it's the research component that
sets strategic public relations apart from simplistic, publicity or
press agentry.
Most effective public relations campaigns begin with research, end
with evaluation, and are tweaked along the way with various aspects
of measurement. The research provider industry today claims a large
number of corporate public relations clients, the most successful
public relations agencies have in-house research departments, and
most university-based public relations degree programs require a
research methods course of all graduates.
PURPOSE OF THE STUDY
Although public relations scholars have insisted for decades that
research is an important part of the public relations process, in
reality the use of research, measurement and evaluation in public
relations practice varies dramatically. While some public relations
campaigns do begin with research, followed by strategic planning and
communication, and end with measurement and evaluation, there still
are a large number of situations where serious aspects of research,
planning and evaluation are virtually non-existent or are replaced by
fly-by-the-seat-of-your-pants publicity or press agentry. Even when
measurement and evaluation are used, the research methods employed
frequently focus more upon message outputs than upon program outcomes.
Most of the decades-long academic insistence about this importance
of research is found in text books. Ironically, the scholarly
journals in the public relations field contain a huge void in terms
of studies that examine when and how research actually is used by
practitioners. The purpose of this study is an attempt to change that
by examining how public relations practitioners throughout the world
use research, measurement and evaluation. With the cooperation of the
International Public Relations Association (IPRA), the Public
Relations Society of America (PRSA), and the major public relations
professional societies in a dozen other countries, the survey
reported on in this paper was distributed to a large, international
sample. More than a thousand (n=1,053) useable responses were
received from public relations people in 48 different countries.
LITERATURE REVIEW
One thing virtually every public relations scholar agrees upon is
that research is essential in public relations. If public relations
people really do function as counselors in the arena of public
opinion similar to the way lawyers operate in the court of law, then
it's essential those who practice public relations conduct research
about their strategic publics as attorneys do when they prepare legal
cases. All serious public relations activities must begin with
research, end with evaluation and frequently employ measurement along
the way. Without research, public relations cannot possibly be
practiced as strategically, and becomes relegated to simplistic
one-way communication.
Cutlip and Center (1952) initially advocated research more than half
a century ago in the first edition of their well-known book,
Effective Public Relations. Their four-step model of the public
relations process included a research stage (defining the problem), a
planning stage (deciding what to do about the problem), an action or
communication stage, and, finally, an evaluation stage. This model
has withstood the test of time and considerable growth in public
relations practice and remains virtually unchanged five decades later
(Cutlip, Center & Broom, 2002).
Research also is an important component of other models that describe
the process of public relations including Marston's (1963) RACE model
(research, action, communication, evaluation), the RPIE model
(research, planning, implementation and evaluation) explained by
Seitel (2004), Kendall's (1997) RAISE model (research, adaptation,
implementation, strategy, evaluation) and Hendrix's (2000) more
recent ROPE model (research, objectives, program, evaluation). Even
models that consider a more managerial approach to public relations –
such as the ROSIE model of Sheila Clough Crifasi continue to insist
that research and evaluation are important components of the public
relations process (Seitel, 2004).
The Importance of Research: Virtually every public relations
textbook has something to say about the importance of research.
Baskin, Arnoff and Lattimore (1997) claim, "Research is a crucial
part of the public relations process" (p. 106). Seitel (2004) says,
"Every public relations program or solution should begin with
research" (p. 181). Newsom, Turk and Kruckeberg (2000) indicate, "The
foundation of good public relations strategy is research" (p. 5).
Lattimore, et al., (2004) say, "Research is a vital function in the
process of public relations" (p. 92). Smith (2005) suggests research
has increased in importance because, "public relations and marketing
communications are becoming more strategic, more scientific" (p.1).
Although not found in later editions, the fourth edition of Wilcox,
Ault and Agee (1995) claims research is necessary in public relations
for these six reasons: (a) Increasing audience fragmentation makes it
essential that public relations people understand the specific
interests and concerns of each strategic target audience – including
attitudes, hopes, fears, concerns and frustrations. (b) Frequent
isolation of senior management from rank-and-file employees and
customers necessitates research that can inform these executives
about important strategic audiences. (c) Monitoring the money and
energy frequently spent on public relations activities that interest
neither the mass media nor the organization's strategic publics. (d)
Providing facts on which a successful public relations program can be
based. (e) In addition to gathering information organizations can
use, surveys frequently can generate information that can lead to
significant news coverage. And, (f) establishing baseline and
benchmark measures that can be used to determine if future public
relations programs are effective.
Even though some practitioners continue to question the value of
measurement and evaluation, within the past two decades published
research has done a better job of providing them with more compelling
reasons to incorporate it into their work. For example from 1980 to
1989 the percentage of the Public Relations Society of America (PRSA)
"Silver Anvil Award" winners using formal research in their campaigns
rose from 25% to 40%. A decade later, in 1998, this figure had
increased to 75%.
What Kind of Research? Although the academic literature doesn't
contain published studies of how research actually is used in public
relations, several non-published studies have been conducted on this
topic by Walter Lindenmann, formerly senior vice president and
director of research at Ketchum, one of the world's largest public
relations firms. The most recent of these studies was conducted in
1994 and found that 75% of those practicing the field in the US said
they used research to plan public relations programs, 58% used it to
monitor program progress and make mid-campaign revisions and 58% said
they measured outcomes with research (Jackson, 1994). Lindenmann
(1997a) also reported practitioners were doing more research in 1994
than they had been doing previously.
Bisland (1990) was one of the first to articulate the difference
between public relations research that focused on outputs and the
measures that concentrated on outcomes. Within the past 15 years
these have become two of the most significant words in public
relations measurement and research. As Lindenmann (1997b) explains,
outputs are short-term measures of public relations success – such as
the number of column inches written about a company in a newspaper,
the number of times an organization's official is quoted in news
stories, the number of people who attended a special event, and the
number of visits to an organization's website. Outcomes, on the other
hand, are far more important and measure the long-term success of a
public relations program. Outcome examples could include the amount
of money raised in a fund-raising campaign, whether or not public
opinion and behavior were changed on an issue, if the legislation
campaigned for was passed, if public relations efforts led to an
increase in the stock price, etc. Essentially, outputs measure
information that has been sent or distributed and outcomes measure
information that has been received and acted upon.
In some ways, the differences between output and outcome research
parallel the differences between public relations that is practiced
as a creative art by "journalists in residence" who focus on the
"means" of a public relations program, and public relations practiced
as an applied social and behavioral science by more strategically
oriented practitioners focusing more on the "ends" of a campaign.
Robinson (1969) called the former the "flying by the seat of the
pants" approach and the latter the "scientific" or
"best-available-evidence" approach. Robinson also stressed those
practicing public relations as an applied social and behavioral
science use research in the planning process.
Stacks (2002) points out the importance of research and measurement
in public relations practice has increased as the field has
transitioned from a technical to a managerial function. He says
management decisions are influenced by many factors and that public
relations people use research in many ways, "to monitor and track,
measure and assess, and finally evaluate public relations actions."
(pp. 4-5) Essentially, technicians are more focused upon
communication outputs while managers are more likely to focus upon
the broader perspective of communication outcomes.
Several Important Initiatives: The academic literature of the field
has encouraged public relations people to conduct research,
measurement and evaluation for half a century. However, most of the
early articles advocating these practices essentially said using
research was the right and proper thing to do, but didn't provide
compelling or specific evidence showing how performing measurement
and evaluation could contribute effectively to the organizational
bottom-line. This began to change during the 1980s thanks to two of
the most significantly funded and most ambitious projects in the
history of public relations scholarly research. And, progress would
continue into the 21st Century thanks to two other important initiatives
It's a huge understatement to suggest that what has become known as
the "Excellence Project" of the International Association of Business
Communicators (IABC) has had a huge impact upon research and
measurement in public relations. Thanks to a $400,000 grant from the
IABC Foundation, a team of public relations scholars conducted an
international research project designed to determine how public
relations contributes to organizational effectiveness and what such a
contribution is worth. The major results of this research were
published in 1992 (J. Grunig, 1992).
Also extremely important in advancing the cause of measurement and
evaluation was a project sponsored by the Arthur W. Page Society to
produce a "reader friendly" book about practical applications of how
research was being implemented in the management of public relations
programs (Broom & Dozier, 1990). A decade later, the Institute for
Public Relations commissioned publication of a somewhat similar book
– a self-described "research for dummies" volume – that many
consider to be even more "reader friendly" (Stacks, 2002). The
Institute for Public Relations also initiated another significant
contribution to impacting public relations research when it created
the Commission on Public Relations Measurement and Evaluation in 1997.
Broom and Dozier (1990) and Stacks (2002) were the first books about
public relations research written for audiences of practitioners as
opposed to students. Their contributions in advancing the cause of
research, measurement and evaluation within the field have been
massive. The Broom and Dozier book includes many quotes from leading
public relations experts discussing how and why they have used
research. Although both books do a good job of walking the reader
through various steps necessary for many different methodological
approaches, the later publication date of the Stacks book makes it
particularly useful for those who want to conduct e-mail or web-based surveys.
The Institute's Commission on Public Relations Measurement and
Evaluation has had a major impact in bringing together thought
leaders from the various pillars of public relations measurement.
With representation from the client/corporate sector, agencies,
research providers and academics, this Commission has helped
practitioners gain a better understanding of basic and theoretical
research from academics. It also has helped those who teach public
relations research learn more about applied measurement from those
who actually practice it. The Commission has produced a number of
papers about measurement and evaluation, all of which can be
downloaded free of charge from the Institute's website www.instituteforpr.com.
More on the Excellence Study: The "Excellence Study" also attempted
to identify the specific characteristics that make public relations
programs successful. This major research project led to a much
clearer theoretical understanding of public relations with relevance
to various kinds of organizations across cultural, national and
societal boundaries. Prior to the "Excellence Study" research, there
was concern that public relations research was not overly conceptual
(Pavlik & Salmon, 1983), and that the field lacked a theoretical
relationship between organizations and publics (J.Grunig, 1984). The
general consensus is that things were highly improved in these areas
once the" Excellence Study" research had been completed and published.
Although there were some exceptions, prior to the "Excellence
Project" a majority of the scholarly research in public relations was
atheoretical and, as L. Grunig (1996) has articulated, most of the
theory-based studies involving public relations focused on a single
public relations campaign. The "Excellence Project" took a more
comprehensive approach and attempted to take into account, "the
complex interplay of environment, organizational structure, corporate
and societal culture, power, and the different approaches to public
relations that are possible" (L. Grunig, 1996, p. 459).
Prior to this work, J. Grunig (1984) outlined four models of the
public relations process. Two of these were one-way communication
models – press agentry and publicity. Both disseminated information
through the mass media to various publics, and usually did not
include any research. The other two models were based upon two-way
communication, and were known as two-way asymmetrical and two-way
symmetrical. Both relied upon research in different ways. The
asymmetrical, or imbalanced, model used research to help determine
which messages would be the most likely to persuade publics. The
symmetrical model used research to help foster mutual understanding
between organizations and their publics. A major thesis of the report
of the "Excellence Study" was the encouragement of the practice of
two-way symmetrical communication (J. Grunig, 1992). The report also
said that "excellent" companies conduct research, participate in
strategic management, and have a seat at the organization's
decision-making table (J.Grunig, 1992). Commenting on these
recommendations, Anderson (2002) says research plays a major role in
giving public relations a seat at the decision-making table.
Other Reasons to Conduct PR Research: In addition to the work of J.
Grunig (1992) and Broom and Dozier (1990), a variety of other studies
have played significant roles in encouraging public relations
practitioners to use research, measurement and evaluation. These
include studies that say measurement preserves organizational
autonomy, especially during unstable times (Kelly, 1994), fends off
encroachment of the public relations function by other organizational
departments (Spicer, 1991; Lauzen, 1991 & 1992), and that research
can help demonstrate competency (Ferguson, 1998; Broom & Dozier, 1983).
Obstacles to Conducting Research: Hon (1997 & 1998) says not all
practitioners conduct research. She points out that when research is
conducted it frequently focuses on the impact on specific publics
rather than what value public relations brings to an organization
(1997). Hon (1998) also says public relations research needs to
concentrate more on whether or not programs are achieving specific
outcomes. Cutlip, Center and Broom (2002) say, "Few practitioners
subscribe to or read the field's research journals. Even some board
members of foundations and institutes ostensibly established to
encourage basic research talk about scholarly research with disdain
and try to divert resources to producing 'news you can use'" (p.152).
Other than the reality that a large number of practitioners either
avoid research or are uncomfortable with it, the published literature
doesn't contain any articles that explore possible obstacles such as
how much research might cost, how long it could take to conduct a
research study, the uncertainty of what one might discover, and so
forth. Lindenmann (2002) has explained how public relations research
can be conducted without exorbitant cost.
Support for Research: Although never supported by any published
studies, the common assumption is support for public relations
research pales in comparison to support for other professional fields
of study. While it is not uncommon for medical schools, engineering
programs, architectural design projects, etc., to receive research
grants totaling more than a million dollars, few grants for public
relations research ever total more than a few thousand dollars.
The largest funded support for public relations research involved
the aforementioned Excellence Study made possible through a $400,000
grant from the IABC Research Foundation (Cutlip, Center & Broom,
2002). Since awarding this very large grant in 1987, the IABC
Foundation has continued to support academic research but with much
more modest awards.
The first foundation established to support public relations
research and education was the Foundation for Public Relations
Research and Education, established by PRSA in 1956 with the goal of
advancing public relations professionalism by funding research,
disseminating scholarship and promoting professional educational
development. This foundation established an annual lecture series in
1961 and subsidized publication of Public Relations Review, the first
scholarly journal devoted exclusively to public relations, in 1975.
A variety of philosophical differences between PRSA and members of
its foundation board led to the foundation separating from the
association in 1989 and creating the Institute for Public Relations
Research and Education. Now based in the College of Journalism and
Communications at the University of Florida, the Institute is the
nation's only independent foundation dedicated to improving public
relations research, training, scholarship and professional
development. It has distinguished itself by pioneering publications,
lectures, awards and competitions to promote academic and
professional excellence in the field. Among the Institute's many
activities is sponsorship of the annual International Public
Relations Research Conference, the nation's only scholarly conference
dedicated solely to public relations research. This conference
annually attracts leading public relations scholars and a good number
of professionals from corporate, agency and research provider
organizations. The Institute's Pathfinder Award holds an
international reputation as the most prestigious award presented
annually for excellence in public relations scholarship. Since 1997,
the Institute has sponsored the world's only Commission on Public
Relations Measurement and Evaluation that brings together
representatives from the four pillars of public relations research –
corporate and non-profit clients, agencies, academics and research providers.
Soon after what is now known as the Institute separated from PRSA the
professional society formed another foundation that attempts to
advance the field through career and professional development,
diversity, research and scholarships. The PRSA Foundation's early
contributions focused upon reexamining and articulating the body of
knowledge in public relations. Although this foundation has funded
several small research projects during the past decade, most of its
recent efforts have concentrated more on scholarships and on
advancing diversity within public relations. Few consider it to play
as significant a role in the support and development of public
relations research as the Institute for Public Relations or the IABC
Research Foundation.
Academic Emergence: A variety of academic issues have had a major
impact upon the development of public relations research. With a few
notable exceptions, university-based public relations education in
the US began shortly after World War II and was concentrated in
journalism programs. Former journalists who had an antipathy for both
behavioral research and public relations dominated faculty ranks at a
good number of these journalism programs. As Weaver and Gray (1980)
have pointed out, a majority of journalism faculty thought scientific
study of mass communication "to have no practical value whatever, in
part because few effects researchers bothered to expound on the
implications of their studies for journalists, and also because many
such researchers wrote the results of their studies in barely
comprehensible language" (p. 142). Scholars from other disciplines
also complained that journalism and mass communication (including
public relations) research was inappropriate for the academy because
so much of it was atheoretical (Greenberg & Salwen, 1996).
Wright (2004) studied the prejudice many public relations educators
face from their journalism colleagues and reported, "This
professional prejudice and discrimination ranges from the subtle and
obscure to the much more observable and obvious. While virtually
every person teaching today in an academic unit that houses a public
relations program would be appalled if they heard one of their
colleagues use discriminatory terminology to describe minorities,
women, members of a religion, homosexuals, etc., most of these same
faculty members would not get the slightest bit upset if they heard
that same colleague use derogatory terms, such as "flack," to portray
public relations people" (p. 2). Newsom (1984) echoed similar
concerns when she claimed that many journalism and mass communication
faculty members considered public relations educators to be the
"bastard stepchildren" of journalism schools.
Although things are different today, many of the nation's most
notable journalism schools emphasized vocational training more than
social science study throughout the early years of public relations
education – i.e., from 1950 through 1975. This vocational emphasis
coupled with the antipathy for behavioral research resulted in many
journalism schools hiring public relations faculty members who were
neither interested in nor had the ability to conduct scholarly
research. Consequently, the contributions to the development of
public relations research and theory were minimal during this time
period. Much of this has changed during the past 25 or 30 years.
Except in rare instances, it is extremely difficult today for someone
without a doctoral degree and a committed plan for scholarly research
development to get hired as a public relations faculty member at most
universities.
In addition to the development of public relations study in many
journalism and mass communication schools, public relations courses
began to surface in some speech-communication departments and a small
number of business schools. This added to the confused identity of
public relations education, but has stimulated the development of
scholarly research. Given their roots in rhetoric and persuasion,
speech-communication faculties tend to be more scholarly and
theoretical than those in journalism and mass communication. The
speech-communication programs, many of which are now called
communication programs, have significantly facilitated the
development of scholarly research in public relations. Meanwhile,
business school researchers have contributed to adding a number of
case study analyses to the public relations body of knowledge.
METHOD
The study's measuring instrument was a web-based, Benchpoint™ survey
that was translated into five different languages. The Benchpoint™
product is a fully integrated, multi-lingual, interactive opinion
measurement and polling system that has acquired considerable
credibility in the marketplace. It was named the "Best New
Measurement Product of 2004" by The Measurement Standard (2005), a
monthly web publication specializing in public relations and
communications research.
Invitations to participate in the study were extended via e-mail
messages to random samples collected from a variety of membership
organizations, including the principal professional membership
societies in thirteen different countries plus a number of other
professional organizations. In the US, participation invitations were
extended to samples of members of the Public Relations Society of
America (PRSA), the Arthur W. Page Society and to donor, task-force
and commission membership lists of the Institute for Public Relations
(IPR). Invitations also were extended to samples selected from
membership lists of these organizations: Public Relations Institute
of Australia (PRIA), Belgian Public Relations Centre, Belgian
Association of Public Relations Consultancies, German Public
Relations Society (DPRG), Public Relations Association of Indonesia
(PERHUMAS), Public Relations Institute of Ireland (PRII), Italian
Federation of Public Relations (FERPI), Public Relations Institute of
New Zealand (PRINZ), Polish Public Relations Consultancies
Association, Public Relations Institute of South Africa (PRISA),
Spanish National Council of Public Relations, Institute of Public
Relations (IPR-UK), Public Relations Consultants Association-UK
(PRCA), and to a list of practitioners from various states in the
former Yugoslavia. In an attempt to enhance the global scope of this
project even more, additional invitation distribution names were
selected from membership rosters of the International Public
Relations Association (IPRA) and the International Association of
Business Communicators (IABC).
The study's measuring instrument contained 82 questions, including
"builds" designed to ask follow-up questions to certain groups of
respondents. For example, one question asked subjects if they
currently measure or evaluate their public relations work. Builds
following this question directed those who answered "yes" to a series
of questions asking what types of tools they used to measure and some
questions about how effective they perceived these tools to be. Those
responding "no" were taken to a page that inquired about factors that
are impediments to carrying out evaluation and measurement. In an
attempt to encourage participation from practitioners in non-English
speaking countries, the questionnaire was available in French,
German, Spanish and Portuguese in addition to English.
The questionnaire was designed by the senior author of this paper in
close cooperation with Richard Gaunt, the President and CEO of
Benchpoint™ who played a significant role in all aspects of the project.
Most (96%) of the study's respondents were public relations or
communications professionals with four percent indicating they were
clients. A large majority (94%) claimed to be personally involved,
one way or another, in measurement and evaluation. This included 24%
who were involved "very much," 34% who were involved "quite a lot,"
and 36% who said they were involved "a little." Eighty-eight percent
said their responsibilities included external communication and 81
percent claimed to also have internal communication responsibilities.
The greatest number of usable responses were received from the United
States (n=400), Germany (n=116), Canada (n=108), and the United
Kingdom (n=93). The total number of subjects does not equal 1,053 in
regional and international analyses because two subjects did not
answer the location question.
Data were analyzed via various measurement capabilities built into
the Benchpoint™ survey program. All analyses also were conducted via
the traditional SPSS (Statistical Package for the Social Science)
program for reasons of statistical consistency.
OVERALL RESULTS
Results gathered from this fairly large set of respondents (n=1,053)
from 48 countries suggest research, measurement and evaluation are
considered to be important components of the public relations process
regardless of what part of the world public relations is practiced.
As Table 1 shows, 88% of the study's respondents said research,
measurement and evaluation were important in public relations. More
than two-thirds (69%) said their organizations currently measure the
effectiveness of public relations and 78% of those not currently
measuring said their organizations plan on doing so in the future.
Table 1
Analysis of the Perceived Importance of Research and Measurement
Strongly
Disagree
Disagree
Uncertain
Agree
Strongly
Agree
Mean
Score
Research, measurement and evaluation are important components of the
public relations process.
9%
2%
1%
27%
61%
4.3
Is it possible to calculate ROI in public relations?
2%
13%
20%
52%
13%
3.6
Each type of business needs different measures of ROI on communication.
1%
10%
14%
52%
23%
3.0
ROI measurement is the key to achieving better recognition and
budgets for public relations.
2%
11%
18%
45%
25%
3.8
Communications measurement is impossible because you cannot measure
the intangible
25%
46%
12%
14%
3%
2.2
I don't need measurement because I'm a professional and know what
works and what doesn't work.
55%
32%
6%
5%
2%
1.7
Measurement is essential to demonstrate the value of the
communications function to my organization
2%
6%
8%
47%
37%
4.1
Measurement helps us to inform our constituents about communications
policies and programs.
2%
6%
9%
49%
35%
4.1
Measurement enables us to assign appropriate budgets to programs.
2%
8%
12%
50%
28%
3.9
Measurement is a good idea, but can be impractical and expensive.
9%
30%
16%
36%
9%
3.1
I think we will be doing more measurement and evaluation in the future.
2%
8%
20%
44%
27%
3.9
Communications activities in my company represent good value in terms
of budgets and resources.
2%
6%
14%
47%
32%
4.0
Communications could do a better job with a larger budget and more resources.
2%
9%
14%
39%
37%
4.0
If a sensible, practical, cost-effective way of calculating ROI on
communication could be devised, I would be very interested.
2%
2%
8%
32%
56%
4.4
Note: Frequencies (percentages) might not always total 100% due to
rounding. Unless otherwise noted, mean scores reported throughout
this paper are based upon five-point, Likert-type scales and are in a
positive direction (i.e. "5.0" equals "strongly agree," "very
effective," "very major," etc., and "1.0" equals "strongly disagree,"
"very ineffective," "very minor," etc.).
Table 1 also shows nearly two-thirds (65%) of the respondents
believe it is possible to calculate return on investment in public
relations, and an equal number suggest each type of business needs a
different ROI measure for communication. Seventy percent think ROI
measurement is the key to achieving better recognition and budgets
for public relations, and 88% agreed that they would be interested –
including 56% who strongly agreed – if a sensible, practical,
cost-effective way of calculating ROI on public relations and
communication could be devised. A large majority (84%) indicate
measurement is essential to demonstrate the value of the
communications function in their organizations, 75% agree measurement
enables them to assign appropriate budgets to programs, and 71%
anticipate conducting more measurement and evaluation in the future.
Perhaps understandably, many (79%) believe public relations
activities in their companies represent good value in terms of
budgets and resources, while 76% say the communications function
could do a better job with a larger budget and more resources.
The questionnaire purposely contained three questions that provided
respondents opportunities to question the value of research. However,
responses to these questions strongly suggest this study's subjects
do not question the value of measurement. When asked if they thought
communications measurement was impossible because the intangible
cannot be measured, 71% disagreed with only 17% agreeing. Although
45% agreed, 39% disagreed that, "Measurement is a good idea, but can
be impractical and expensive." The third question in this series was,
"I don't need measurement because I'm a professional and know what
works and what doesn't work." Only seven percent of the respondents
agreed with this statement while 87% disagreed.
Although 88% agreed that research, measurement and evaluation were
important, only 69% said their organizations currently measure the
effectiveness of their work. However, as Table 2 shows, 78% of those
who do not now measure have plans to do so in the future. More than
half (56%) of the study's respondents said their organizations
measured the effectiveness of their central functions – such as
public relations and communications, human resources, marketing,
finance, etc. – and three quarters (74%) of those who are not
currently doing so have plans to measure their central functions in the future.
Table 2
Responses to Questions about What is Being Measured
Yes
No
Mean
Does your organization currently measure the effectiveness of your work?
69%
31%
1.7
If not, do you plan to evaluate or measure your work in the future?
78%
22%
1.8
Do you carry out external communications on behalf of your
organization or its clients?
88%
12%
1.9
Do you carry out internal communications on behalf of your
organization or its clients?
81%
19%
1.8
Do you, or do you plan to, measure internal communications effectiveness?
82%
18%
1.8
Do you currently measure the effectiveness of your central functions
(Human resources, Marketing, Finance, Public Relations and Communications?
56%
44%
1.6
Do you plan to measure the effectiveness of your central functions in
the future?
74%
26%
1.7
Note: Mean scores were based upon a two-point scale where "1" equals
"no," and "2" equals "yes," consequently, the higher the mean the
greater the agreement.
Several questions asked subjects what research tools they used to
measure public relations effectiveness. Results in terms of external
communications found 88% used opinion surveys, 87% employed internal
reviews, 85% used detailed media analysis, 82% benchmarked against
competitors, 72% said they used advertising value equivalents, and
70% used dashboards. Measures of the use of internal communications
research tools found that 95% of the respondents used employee
surveys, 92% sought out feedback from fellow professionals in the
organization, 91% measured with internal reviews, 88% used employee
focus groups, 70% benchmarked with competitors and 65% used dashboards.
The study also asked subjects their opinions about the effectiveness
of these various measurement tools for evaluating the effectiveness
of external and internal public relations. Results are displayed in
Tables 3 and 4.
Table 3 shows, as one might expect, that the tools used the most to
evaluate external public relations and communications efforts are the
ones considered to be the most effective. Opinion surveys are used
most frequently and also are considered to be the most effective of
the possible measures of external communications with 70% of the
respondents saying they are effective or very effective. Nearly
two-thirds (65%) say detailed media evaluations are effective and 59%
say benchmarking with competitors is effective. Nearly one-third
(30%) suggest advertising value equivalents are effective. A similar
number (28%) claim they do not use AVEs.
Table 3
Perceived Effectiveness of External Communications Measurement Tools
Tool Not
Used
Very
Ineffective
Ineffective
Uncertain
Effective
Very
Effective
Mean
Score
Advertising Value Equivalents
28%
8%
10%
24%
25%
5%
3.1
Detailed media evaluations
15%
2%
4%
15%
45%
20%
3.9
Opinion surveys
12%
2%
6%
11%
48%
22%
3.9
Dashboards
30%
2%
6%
32%
24%
7%
3.4
Benchmarking with competitors
18%
2%
7%
15%
46%
13%
3.7
Position in published league tables
32%
3%
8%
33%
19%
5%
3.2
Internal reviews
13%
4%
10%
19%
43%
12%
3.6
Note: For purposes of mean analysis, and subsequent ANOVA tests that
follow, the six-point answer scale for this set of questions was
collapsed into a five-point scale. All of the "tool not used"
responses were reclassified as missing values, and analysis was then
based upon a five-point scale ranging from "1" for "very ineffective"
to "5" for "very effective."
As Table 4 shows, respondents considered internal communications
measurement tools to be more effective than tools designed to measure
external communications and public relations programs.
Table 4
Perceived Effectiveness of Internal Communications Measurement Tools
Tool Not
Used
Very
Ineffective
Ineffective
Uncertain
Effective
Very
Effective
Mean
Score
Employee surveys
5%
3%
3%
4%
50%
35%
4.2
Internal reviews
9%
2%
5%
12%
52%
20%
3.9
Employee focus groups
12%
2%
2%
10%
42%
31%
4.1
Feedback from fellow professionals in the organization
8%
2%
6%
16%
51%
17%
3.8
Dashboards
35%
1%
5%
32%
21%
5%
3.3
Benchmarking with competitors
30%
3%
7%
25%
27%
9%
3.5
Note: For purposes of mean analysis, and subsequent ANOVA tests that
follow, the six-point answer scale for this set of questions was
collapsed into a five-point scale. All of the "tool not used"
responses were reclassived as missing values, and analysis was then
based upon a five-point scale ranging from "1" for "very ineffective"
to "5" for "very effective."
Employee surveys, the tool most frequently used to measure internal
communications effectiveness, was considered to be effective by 85%
of the respondents. Employee focus groups (73%), internal reviews
(72%) and feedback from fellow professionals in the organization
(68%) also scored well. Benchmarking with competitors (36%) and
management dashboards (26%) had lower effectiveness scores.
The study also was interested in evaluating the perceived
effectiveness of a number of other factors that frequently are used
to gauge the success of public relations programs. Results of these
measures are detailed in Tables 5 and 6.
Table 5 shows external measures such as desired outcomes,
awareness/image, and hitting target media are considered fairly
important, more so than formal measurements using tested
methodologies and tools.
Table 5
Perceived Effectiveness of Additional External Measurement Factors
Very
Ineffective
Ineffective
Uncertain
Effective
Very
Effective
Mean
Score
Effectiveness serving the needs of senior management
0%
11%
16%
45%
29%
3.9
On time/budget
3%
5%
26%
40%
26%
3.8
Hit target media
3%
0%
13%
40%
45%
4.2
Achieving desired outcomes
0%
8%
5%
40%
47%
4.3
Awareness/image
5%
0%
13%
34%
47%
4.2
Formal measurements using tested methodologies and tools
8%
18%
21%
42%
11%
3.3
An instinctive feeling and knowledge of what works and what does not work
5%
3%
24%
50%
18%
3.7
In terms of additional internal measures, findings displayed in Table
6 show feedback from employees, managers and professional peer groups
and achievement of desired outcomes rank as being the most important.
More than three-quarters (77%) of the respondents considered feedback
from employees, managers and professional peer groups to be effective
or very effective while 72% said they thought achieving desired
outcomes were effective. Unusually high on the scales in these
measures was "instinctive understanding of what is right/not based on
professional knowledge and experience." This included 72% of the
study's respondents indicating they thought this was an effective
measurement tool while only nine percent said they thought it was ineffective.
Table 6
Perceived Effectiveness of Additional Internal Measurement Factors
Very
Ineffective
Ineffective
Uncertain
Effective
Very
Effective
Mean
Score
Feedback from employees, managers and professional peer groups
0%
14%
10%
36%
41%
4.0
Cost-benefit analysis
2%
17%
19%
48%
14%
3.6
Instinctive understanding of what is right/not based on professional
knowledge and experience
2%
7%
19%
60%
12%
3.7
Achieving desired outcomes
0%
5%
24%
43%
29%
4.0
Good results in the employee opinion survey
0%
5%
33%
41%
21%
3.8
Effectiveness of response to the needs of senior management
2%
17%
26%
31%
24%
3.6
Another interest the study had concerned the various impediments to
carrying out measurement and evaluation. With that in mind, subjects
who indicated they do not measure and evaluate either their external
or internal public relations or communications efforts were asked to
tell why. Opinions of these respondents (n=216) on this subject are
shown in Table 7.
Table 7
Perceived Impediments to Public Relations Measurement and Evaluation
Very
Major
Major
Uncertain
Minor
Very
Minor
Mean
Score
Cost
30%
47%
8%
12%
3%
3.9
Expertise
13%
41%
19%
21%
6%
3.3
Some questioning the value of research
11%
42%
22%
21%
5%
3.3
The time it takes to conduct research
19%
41%
14%
20%
6%
3.5
Uncertainty of what one might find
6%
14%
27%
31%
22%
2.5
Other factors
5%
4%
80%
2%
9%
2.9
Results show cost and the time it takes to conduct research are the
major impediments to measurement and evaluation in public relations
and communication. Interestingly enough, the uncertainty of what one
might discover through measurement proved to be only of minor
concern. However, the fact that more than half of these respondents
(53%) listed the point that some people question the value of
research as an impediment, suggests the possibility that more work
might need to be done explaining the potential value of research,
measurement and evaluation in public relations.
MEASURES OF REGIONAL DIFFERENCES
In addition to gaining an understanding of how research, measurement
and evaluation are used in public relations and communication
programs throughout the world, we also were interested in leaning
about any differences that might exist between how research is used
in one part of the world as compared to how it is used in some other region.
All but two of the study's respondents answered the location
question. The greatest number of returns came from the United States
(n=400), Germany (n=116), Canada (n=108), and the United Kingdom
(n=93). For purposes of geographical analyses, respondents were
grouped into three distinct regional categories: North America
(n=508), Western Europe (n=392), and other regions (n=151).
Analysis of Variance (ANOVA) tests were performed on the study's
entire data set as part of the examination of differences among and
between practitioners from various parts of the world. Findings
reported in Table 8 show respondents from outside of North America
were slightly more likely to agree that research, measurement and
evaluation are important components of the public relations process.
This finding was statistically significant [p = <.04].
Table 8 also shows nine other statistically significant differences
between those who practice public relations in different parts of the
world. Western Europeans were considerably less likely than others to
believe that it's possible to calculate return on investment on
public relations and communications functions [p = <.01]. They also
were less likely to agree that measurement enables public relations
people to assign appropriate budgets to programs [p = <.00].
Meanwhile, those from outside of North America and Western Europe
were more likely to agree that each type of business needs different
measures of ROI on communication [p = <.01], that measurement is
essential to demonstrate the value of the public relations function
to an organization [p = <.00], and that measurement helps them inform
their communications policies and programs [p = <.00]. Practitioners
from these other regions also were more likely to think they will be
doing more measurement and evaluation in the future [p = <.00], and
to agree they would be interested if a sensible, practical,
cost-effective way of calculating ROI on communication could be
devised [p = <.00].
Table 8
ANOVA Analysis of the Perceived Importance of Research and Measurement
North
America
(n=508)
Western
Europe
(n=392)
Other
Regions
(n=151)
Total
Mean
(n=1,051)
F =
Sig.<
Research, measurement and evaluation are important components of the
public relations process.
4.2
4.4
4.4
4.3
3.14
.04
Is it possible to calculate ROI in public relations?
3.7
3.5
3.8
3.6
8.53
.00
Each type of business needs different measures of ROI on communication.
3.8
3.8
4.1
3.0
4.87
.01
ROI measurement is the key to achieving better recognition and
budgets for public relations.
3.8
3.7
3.8
3.8
10.2
.00
Communications measurement is impossible because you cannot measure
the intangible
2.2
2.2
2.3
2.2
0.97
.38
I don't need measurement because I'm a professional and know what
works and what doesn't work.
1.7
1.7
1.6
1.7
0.43
.65
Measurement is essential to demonstrate the value of the
communications function to my organization
4.1
4.0
4.5
4.1
20.1
.00
Measurement helps us to inform our constituents about communications
policies and programs.
4.0
4.1
4.4
4.1
11.9
.00
Measurement enables us to assign appropriate budgets to programs.
4.0
3.8
4.2
3.9
14.5
.00
Measurement is a good idea, but can be impractical and expensive.
3.1
3.1
2.9
3.1
1.04
.36
I think we will be doing more measurement and evaluation in the future.
3.9
3.8
4.1
3.9
5.64
.00
I believe communications activities in my company represent good
value in terms of budgets and resources.
4.1
3.9
4.0
4.0
6.72
.00
I believe communications could do a better job with a larger budget
and more resources.
4.0
4.0
4.1
4.0
0.50
.60
If a sensible, practical, cost-effective way of calculating ROI on
communication could be devised, I would be very interested.
4.4
4.3
4.6
4.4
9.24
.00
Although results showed statistical significance, minimal mean
differences between the three groups were found on items indicating,
"I believe communications activities in my company represent good
value in terms of budgets and resources," [p = <.00], and "ROI
measurement is the key to achieving better recognition and budgets
for public relations," [p = <.00].
In an attempt to provide greater explanation to the regional
differences displayed in the ANOVA results presented in Table 8,
crosstabulations and chi-square tests were performed on some
measures. Results found that 89% of the Western European respondents
agreed that research, measurement and evaluation are important
components of public relations as opposed to 86% of the North
Americans and 88% of those from other parts of the world. Results of
the chi square test were significant at p = < .00. Although they were
not statistically significant, somewhat similar results were found
when crosstabulations were performed on the question asking whether
or not "your organization" is currently involved in public relations
measurement. Responses found that 71% of the Western European
organizations measured as compared with 66% of the North Americans
and 70% of those from other regions.
Crosstabulations also were performed on the question asking how
involved respondents were with measurement and evaluation in their
regular employment duties. Results found that 69% of the Western
Europeans answered either "very much" or "quite a lot" to this
question as compared to 52% of the North Americans and 58% of those
from other parts of the world. The chi square test on this question
was significant at p = < .03.
Regional differences in terms of what is being measured were
minimal. These results are displayed in Table 9.
Table 9
ANOVAs on Responses to Questions about What is Being Measured
North
America
(n=508)
Western
Europe
(n=392)
Other
Regions
(n=1,051)
Total
Mean
(n=1,051)
F =
Sig.<
Does your organization currently measure the effectiveness of your work?
1.7
1.7
1.7
1.7
1.35
.26
If not, do you plan to evaluate or measure your work in the future?
1.8
1.7
1.8
1.8
2.00
.14
Do you carry out external communications on behalf of your
organization or its clients?
1.8
1.9
1.9
1.9
8.30
.00
Do you carry out internal communications on behalf of your
organization or its clients?
1.9
1.7
1.8
1.8
11.1
.00
Do you, or do you plan to, measure internal communications effectiveness?
1.9
1.7
1.9
1.8
7.71
.00
Do you currently measure the effectiveness of your central functions
(Human resources, Marketing, Finance, Public Relations and Communications?
1.5
1.6
1.5
1.6
0.19
.82
Do you plan to measure the effectiveness of your central functions in
the future?
1.2
1.8
2.0
1.7
7.98
.00
Since public relations in the US and Canada is more likely to include
both external and internal communication responsibilities, North
American respondents were slightly more likely to be involved in the
practice of internal communications [p = <.00], but slightly less
likely to be involved in external public relations practice [p =
<.00]. Western Europeans were slightly less likely to be involved in
measuring internal communications effectiveness [p = <.00]. When
subjects were asked how likely they were to measure the effectiveness
of central functions in the future, respondents from outside of North
America had the highest mean scores [p = <.00].
Analysis of differences in the perceived effectiveness of external
communications measurement tools are displayed in Table 10. Findings
show public relations people from North America and Western Europe
consider advertising value equivalents to be much less effective than
practitioners from other regions of the world [p = <.01]. Similar
results were found on the measure involving positions in published
league tables [p = <.04]. North Americans also were considerably less
likely to consider detailed media evaluations [p = <.00] or internal
reviews [p = <.00] to be effective ways of measuring external public
relations efforts.
Table 10
ANOVAs of Perceived Effectiveness of External Communications Measurement Tools
North
America
(n=508)
Western
Europe
(n=392)
Other
Regions
(n=151)
Total
Mean
(n=1,051
F =
Sig.<
Advertising Value Equivalents
3.1
3.1
3.5
3.1
5.79
.01
Detailed media evaluations
3.7
4.0
4.1
3.9
13.0
.00
Opinion surveys
3.9
3.9
4.1
3.9
1.97
.14
Dashboards
3.5
3.4
3.4
3.4
1.31
.27
Benchmarking with competitors
3.7
3.7
3.8
3.7
0.66
.52
Position in published league tables
3.2
3.2
3.4
3.2
3.33
.04
Internal reviews
3.4
3.7
3.9
3.6
10.9
.00
Table 11 reports on perceived differences on the effectiveness of
various tools in measuring the effectiveness of internal communications?
Table 11
ANOVAs of Perceived Effectiveness of Internal Communications Measurement Tools
North
America
(n=508)
Western
Europe
(n=392)
Other
Regions
(n=151)
Total
Mean
(n=1,015)
F =
Sig.<
Employee surveys
4.0
4.4
4.3
4.2
11.8
.00
Internal reviews
3.8
4.0
4.3
3.9
11.9
.00
Employee focus groups
4.1
4.1
4.3
4.1
1.99
.14
Feedback from fellow professionals in the organization
3.8
3.8
4.1
3.8
4.77
.01
Dashboards
3.3
3.3
3.5
3.3
2.60
.08
Benchmarking with competitors
3.5
3.4
3.6
3.5
1.43
.24
Results of this analysis show significance on three of the six
items. North Americans are much less likely to consider employee
surveys effective [p = <.00], while those from outside of North
America and Western Europe are much more likely to consider internal
reviews [p = <.00] and feedback from fellow professionals in the
organization [p = <.01] to be effective.
Two items are statically significant on the measures evaluating the
perceived effectiveness of additional external measurement factors.
Table 12 shows that North Americans are considerably less likely to
consider awareness/image measures [p = <.04] or formal measurements
using tested methodologies and tools [p = <.04].
Table 12
ANOVAs of Perceived Effectiveness of Additional External Measurement Factors
North
America
(n=508)
Western
Europe
(n=392)
Other
Regions
(n=151)
Total
Mean
(n=1051)
F =
Sig.<
Effectiveness serving the needs of senior management
3.0
3.91
4.3
3.9
2.97
.06
On time/budget
3.5
3.7
4.1
3.8
0.69
.51
Hit target media
3.5
4.4
4.1
4.2
2.26
.12
Achievement of desired outcomes
3.8
4.3
4.4
4.3
0.74
.48
Awareness/image
3.0
4.4
4.2
4.2
3.49
.04
Formal measurements using tested methodologies and tools
2.0
3.5
3.3
3.3
3.46
.04
An instinctive feeling and knowledge of what works and what does not work
3.5
3.9
3.5
3.7
0.94
.40
As Table 13 indicates, there were no significant results when
comparing regional differences and the study's additional internal
communications measurement factors.
Table 13
ANOVAs of Perceived Effectiveness of Additional Internal Measurement Factors
North
America
(n=508)
Western
Europe
(n=392)
Other
Regions
(n=151)
Total
Mean
(n=1051)
F =
Sig.<
Feedback from employees, managers and professional peer groups
4.0
4.0
4.2
4.0
0.17
.84
Cost-benefit analysis
3.2
3.5
3.8
3.6
0.81
.45
Instinctive understanding of what is right/not based on professional
knowledge and experience
3.5
3.8
3.6
3.7
3.41
.71
Achievement of desired outcomes
3.3
4.0
4.3
4.0
2.53
.09
Good results in the employee opinion survey
3.7
3.9
3.7
3.8
0.82
.45
Effectiveness of response to the needs of senior management
3.0
3.6
3.8
3.6
1.09
.35
Analysis concerning impediments to carrying out measurement and
evaluation in public relations, as shown in Table 14, find two items
of statistical significance. Western Europeans are considerably more
likely to consider cost as an impediment [p = <.05], while North
Americans are much less likely to consider the uncertainty of what
one might find in measurement as an impediment [p = <.01].
Table 14
ANOVAs of Perceived Impediments to Public Relations Measurement and Evaluation
North
America
(n=508)
Western
Europe
(n=392)
Other
Regions
(n=151)
Total
Mean
(n=1,015)
F =
Sig.<
Cost
3.7
4.1
3.7
3.9
2.96
.05
Expertise
3.4
3.3
3.5
3.3
0.41
.67
Some questioning the value of research
3.4
3.3
3.5
3.3
0.34
.71
The time it takes to conduct research
3.7
3.3
3.4
3.5
2.22
.11
Uncertainty of what one might find
2.3
2.7
2.9
2.5
4.69
.01
Other factors
3.0
2.9
2.7
2.9
1.21
.30
Additional Findings: Eighty-five percent of the respondents had
received at least an undergraduate university degree (BA, BSc, BBA,
etc.), while 36 percent reported having a masters degree (MA, MSc,
MBA, etc.) and six percent claimed to have a Ph.D., or some other
doctoral, degree. When asked how much formal education they had in
public relations or communication, 39 percent claimed some kind of
university degree, 12 percent listed a professional association
certificate, four percent said they minored in public relations or
communications at a university, 11 percent claimed to have taken at
least one university-based public relations course, and 23 percent
said they had no formal education in the field. Forty-one percent
reported they had received some formal education in research,
measurement and evaluation.
Cross tabulations were performed in an attempt to check for
differences or similarities between education obtained and region.
Results found more than half (51%) of the North American respondents
had university degrees in public relations or communication as
compared to 26 percent of the Western Europeans and 39 percent of the
respondents from the rest of the world. The chi-square test on this
measure was significant at p = <.00. When asked if they had received
any formal education in research, measurement or evaluation, 40
percent of the North Americans answered "yes," along with 39 percent
of the Western Europeans and 54 percent of the respondents from other
regions. Chi-square results on this measure were significant at p = <.036.
DISCUSSION
This large (n=1,053) international study of communications
professionals from 48 countries finds most (88%) think research,
measurement and evaluation are important components of the public
relations process. More than two-thirds (69%) of their organizations
currently measure.
These results are both encouraging and frustrating. They are
encouraging because a very large number of respondents say research
is important. They're frustrating because almost one-fifth of those
who claim research is important also point out that their
organizations aren't doing any. In regional terms, these results also
are not impressive for North America because respondents from there
are less likely to agree that research is important and less likely
to be involved in measurement than respondents from Western Europe
and other parts of the world. This study's finding that 66% of its
North American respondents say their organizations are involved in
measurement pales in comparison to Lindenmann's (1997a) discovery
that 75 percent of US public relations practitioners used research
(Jackson, 1994).
A number of other findings fall in the encouraging category including
the result indicating nearly two-thirds of the respondents (65%)
agree it is possible to calculate return on investment in public
relations, 84% say measurement is essential to demonstrate the value
of the communications function to their organizations, and 70% agree
ROI measurement is the key to achieving better recognition and
budgets for public relations.
Even though results generally are encouraging, there might be some
cause to worry when 45% agree measurement is a good idea but can be
impractical. The same thing could be said about the 30% agreement on
the question asking if advertising value equivalents are effective.
It's also a little surprising that more than one quarter of the
respondents (26%) thought formal measurements using tested
methodologies and tools were ineffective.
Although those advocating more measurement of outcomes instead of
outputs will be pleased with the fact that opinion surveys were found
to be the most frequently used measurement tool for external
communications effectiveness research, it also needs to be pointed
out that 85% use media evaluations and 72% conduct advertising value
equivalency studies.
Cost and time were found to be the major impediments to measurement
and evaluation with the uncertainty of what one might discover only
being of minor concern. Cost was a much more significant impediment
for Western Europeans than for respondents from North America or
other parts of the world. The uncertainty of what one might find
through measurement was much less likely to be an impediment in North
America than in Western Europe or other parts of the world.
Comparative analysis of results between North Americans, Western
Europeans and respondents from other parts of the world need to be
viewed with some caution. Even though subjects from these other
regions tend to agree research is more important and are more likely
to measure than North American respondents, one must acknowledge
they're also more likely to use – and consider effective – output
measures such as advertising value equivalents and media
evaluations. This same sense of caution needs to be considered while
interpreting the much higher mean scores – often with statistical
significance – subjects from other regions have these questions: if
measurement is essential to demonstrate the value of the
communications function in an organization, if measurement enables
public relations people to assign appropriate budgets to programs,
and if each type of business needs a different measure for gauging
return on investment in communication.
SUMMARY
This study – with more than a thousand respondents from 48 countries
– finds many public relations practitioners (88%) consider research,
measurement and evaluation to be important components of public
relations. More than two-thirds (69%) say their organizations
currently measure communications effectiveness and 78% of those not
currently measuring plan on doing so in the future. Nearly two-thirds
(65%) think it's possible to calculate return on investment in public
relations and 70% think ROI measurement is the key to achieving
better recognition and budgets for public relations. Most (84%) say
measurement is essential to demonstrate the value of the
communications function in their organizations and 71% anticipate
conducting more measurement and evaluation in the future.
Opinion surveys were found to be the most frequently used
measurement tools for external communications, slightly more popular
than media analysis and advertising value equivalents. Employee
surveys were the most frequently used tool in terms of internal
communications measurement. Opinion surveys and media analysis were
considered to be the most effective methods for measuring external
communications. Although advertising value equivalents are used by
72% of the study's respondents, AVEs were perceived to be the least
effective of these external measures. Employee surveys were the most
frequently used internal communications measurement tools. They also
were perceived to be the most effective.
Cost was the most frequently mentioned impediment to public
relations measurement and evaluation, followed closely by the time it
takes to conduct research.
Analysis for differences among and between practitioners from
various parts of the world found that respondents from outside of
North America were slightly more likely to agree that research,
measurement and evaluation are important components of public
relations. Western Europeans were considerably less likely than
others to believe that it's possible to calculate return on
investment on public relations and communications functions. Regional
differences in terms of what's actually being measured were minimal
but respondents from North America and Western Europe considered
advertising value equivalents to be much less effective measures than
subjects from other regions of the world. North Americans were
considerably more likely to have university degrees in public
relations or communication than other respondents.
REFERENCES
Anderson, F. W. (2002). Research in public relations: Strategy and
accountability. A paper published on the website of the Institute for
Public Relations. Available at:
http://www.instituteforpr.com/research_methods.phtml?article_id=2002_strategy_accountability.
Accessed, Feb. 23, 2005
Baskin, O., Arnoff, C., & Lattimore, D. (1997). Public relations: The
profession and the practice. 4th ed. Madison, WI: Brown & Benchmark.
Bisland, J.H. (1990). Accountability gap: Evaluation practices show
improvement. Public Relations Review. Vol. 16, No. 1, pp. 25-31.
Broom, G. M., & Dozier, D. M. (1983). An overview: Evaluation
research in public relations. Public
Relations Quarterly, Vol. 28, No. 3, pp.5-8.
Broom, G.M. & Dozier, D.M. (1990). Using research in public
relations: Applications to program management. Englewood Cliffs, NJ:
Prentice-Hall.
Cutlip, S.M., & Center, A.H. (1952). Effective public relations, 7th
ed. New York: Prentice-Hall.
Cutlip, S.M., Center, A.H. & Broom, G.T. (2002). Effective public
relations, 7th ed. Upper Saddle River, NJ: Prentice-Hall.
Ferguson, S. D. (1998). Constructing a theoretical framework for
evaluating public relations
programs and activities. In M. E. Roloff (Ed.), (1998). Communication
Yearbook (pp. 191-229).
Thousands Oaks, CA: Sage Publications.
Greenberg, B.S. & Salwen, M.B. (1996). Mass communication theory and
research: Concepts and models. In, Salwen, M.B. & Stacks, D.W. An
integrated approach to communication theory and research. Mahwah, NJ:
Lawrence Erlbaum Associates, (pp. 63-78).
Grunig, L. A. (1996). Public relations. In, Salwen, M.B. & Stacks,
D.W. An integrated approach to communication theory and research.
Mahwah, NJ: Lawrence Erlbaum Associates, (pp. 459-477).
Grunig, J.E. (1984). Organizations, environments, and models of
public relations. Public Relations Research and Education, Vol 1, pp. 6-29.
Grunig, J.E. (Ed.). (1992). Excellence in public relations and
communication management. Hillsdale, NJ: Lawrence Erlbaum Associates.
Hon, L. C. (1997). What have you done for me lately? Exploring
effectiveness in public relations.
Journal of Public Relations Research, Vol. 9, No. 1, pp. 1-30.
Hon, L. C. (1998). Demonstrating effectiveness in public relations:
Goals, objectives, and
evaluation. Journal of Public Relations Research, Vol. 10, No. 2, pp. 103-135.
Jackson, P. (Ed.). (1994). Practical, actionable research for public
relations purposes. Exeter, NH: PR Publishing.
Kelly, K. S. (1994). Fund-raising encroachment and the potential of
public relations departments
in the nonprofit sector. Journal of Public Relations Research, Vol.
6, No. 1, pp. 1-22.
Kendall, R. (1997). Public relations campaign strategies: Planning
for implementation. 2nd ed. New York: Addison-Wesley.
Lattimore, D., Baskin, O., Heiman, S.T., Toth, E.L., & Van Leuven,
J.K. Public relations: The profession and the practice. New York: McGraw-Hill.
Lauzen, M. M. (1991). Imperialism and encroachment in public
relations. Public Relations
Review, Vol. 17, No. 3, pp. 245-255.
Lauzen, M. M. (1992). Public relations roles, intraorganizational
power, and encroachment.
Journal of Public Relations Research, Vol. 4, No. 2, pp. 61-80.
Lindenmann, W.K. (1997a). Guidelines and standards for measuring and
evaluating public relations effectiveness. A paper published on the
research website of the Institute for Public Relations. Available at:
www.instituteforpr.com?pdf?2002_Guidelines_Standards_Book.pdf.
Accessed, Jan. 23, 2005.
Lindenmann, W.K. (1997b). Setting minimum standards for measuring
public relations effectiveness. Public Relations Review. Vol. 23, No.
4, (pp. 391-408).
Lindenmann, W.K. (2002). Research doesn't have to put you in the
poorhouse. A paper published on the research website of the Institute
for Public Relations. Available at:
http://www.instituteforpr.com/measurement_and_evaluation.phtml?article_id=2001_poorhouse
Accessed, Feb. 23, 2005.
Lindenmann, W.K. (2003). Public relations research for planning and
evaluation. A paper published on the research website of the
Institute for Public Relations. Available at:
http://www.instituteforpr.com/measurement_and_evaluation.phtml?article_id=2003_pr_planning_research.
Accessed, Feb. 21, 2005.
Marston, J.E. (1963). The nature of public relations. New York: McGraw-Hill.
Measurement Standard, The. (2005). The best measurement product of
2004:The Measurement Standard's third annual product of the year
awards. The Measurement Standard, Vol. 3, No. 11 Available at:
http://www.themeasurementstandard.com/issues/105/eng/contents105.asp.
Accessed, Feb. 23, 2005.
Newsom, Douglas Ann. (1984, March). "Realities, Questions and
Challenges for Public Relations Education," Public Relations Journal,
pp. 15-16.
Pavlik, J.V., & Salmon, C.T. (1983, August). Theoretical approaches
in public relations research. Paper presented to the Public Relations
Division, Association for Education in Journalism and Mass
Communication, Corvallis, Oregon.
Seitel, F.P. (2004). The practice of public relations. 9th ed. Upper
Saddle River, NJ: Prentica-Hall.
Smith, R.D. (2005). Strategic planning for public relations. 2nd ed.
Mahwah, NJ: Lawrence Erlbaum Associates.
Spicer, C. H. (1991). Communication functions performed by public
relations and
marketing practitioners. Public Relations Review, Vol. 17, No. 3. pp. 293-305.
Stacks, D.W. (2002). Primer of public relations research. New York:
Guilford Press.
Weaver, D.H. and Gray, R.G. (1980). Journalism and mass communication
research in the United States: Past, present and future. In G.C.
Wilhoit & H. de Bock (Eds.), Mass communication review yearbook. Vol.
1, pp. 124-155). Beverly Hills, CA: Sage.
Wilcox, D.L., Ault, P.H., & Agee, W.K. (1995). Public relations
strategies and tactics. 4th ed. New York: Harper Collins.
Wright, D.K. (2004, August). We Have Rights Too: Examining the
Existence of Professional Prejudice and Discrimination Against Public
Relations, Paper presented to the Public Relations Division,
Association for Education in Journalism and Mass Communication,
Toronto, Ontario, Canada.
A Benchpoint Global Analysis of How Research is
Used in Public Relations Throughout the World
A B S T R A C T
Although research has been an important part of public relations for
more than half a century, the use of research, measurement and
evaluation in the field varies dramatically. While many have
advocated the use of research through public relations textbooks, the
scholarly literature contains few studies measuring how research
actually is used in public relations.
This study – with more than a thousand respondents (N=1,053) from 48
countries – finds many public relations practitioners (88%) consider
research, measurement and evaluation to be important components of
public relations. More than two-thirds (69%) say their organizations
currently measure communications effectiveness and 78% of those not
currently measuring plan on doing so in the future. Nearly two-thirds
(65%) think it's possible to calculate return on investment in public
relations and 70% think ROI measurement is the key to achieving
better recognition and budgets for public relations. Most (84%) say
measurement is essential to demonstrate the value of the
communications function in their organizations and 71% anticipate
conducting more measurement and evaluation in the future.
Opinion surveys were found to be the most frequently used
measurement tools for external communications, slightly more popular
than media analysis and advertising value equivalents. Employee
surveys were the most frequently used tool in terms of internal
communications measurement. Opinion surveys and media analysis were
considered to be the most effective methods for measuring external
communications. Although advertising value equivalents are used by
72% of the study's respondents, AVEs were perceived to be the least
effective of these external measures. Employee surveys were the most
frequently used internal communications measurement tools. They also
were perceived to be the most effective. Cost was the most frequently
mentioned impediment to public relations measurement and evaluation,
followed closely by the time it takes to conduct research.
Analysis also revealed differences among and between practitioners
from various parts of the world.
|
