Evaluation of Advertising Creativity
Running Head: Evaluation of Advertising Creativity
Qualitative Evaluation of Print Ads by Assessors
Using the Creative Product Semantic Scale
Alisa White Coleman
University of Texas at Arlington
and
Bruce L. Smith (contact)
University of South Dakota
Department of Mass Communication
414 East Clark St.
Vermillion, SD 57069
605.677.6400
[log in to unmask]
Abstract
The purpose of the study was to ascertain whether advertising professionals
judge advertising creativity in the same way as college students who have had no
advertising training, and whether demographic variables significantly affect
judgments about the creativity of advertising. Fifteen print ads were evaluated
using the Creative Product Semantic Scale. The judgments of students and
professionals were significantly different. There were also significant
differences on the basis of demographic variables.
The advertising industry regards creativity as an essential component for
success in the marketplace. The goal is to create "big ideas" that set brands
apart from their competitors. The big ideas transform into advertisements using
art, headlines, and text.
However, what does creativity mean in advertising? Do advertising professionals
agree on measures of creativity? Do advertising professionals judge advertising
creativity in the same way as the public? The purpose of this study was to
measure experimentally how advertising creativity is judged, and whether those
judgments differ between groups.
The purpose of the study was to ascertain:
1. Whether advertising professionals judge advertising in the same way as a
group of college students who have had no advertising training.
2. Whether other factors such as age, gender, education, race, and region where
one lives significantly affect judgments about the creativity of advertising.
Underlying Assumptions of the Study
This research began with a couple of assumptions. One was that people who are
trained and experienced in the creation of advertising are likely to see
advertisements differently from the public, based on their experiences and
prejudices about what constitutes a creative and effective ad. Another
assumption was that demographically different individuals will experience ads
differently-that younger people, in particular, will judge creativity
differently from older people, but that there may be other differences based on
demographic variables.
General Hypotheses
The following null hypotheses were tested:
H1: There will be no significant difference in the evaluation of print ads by ad
professionals and undergraduate students.
H2: There will be no significant difference in the evaluation of print ads based
on demographic variables.
Literature Review
Creativity has no generally accepted definition. It has often been discussed in
terms of the four facets of creativity (Altsech, 1995; MacKinnon, 1987; Kneller,
1965):
1. The product or outcome of creative behavior.
2. The creative process.
3. The creative personality.
4. The environmental and cultural influences on creativity.
Parnes (1975) said the essence of the concept of creativity was the notion of
"aha," which is "the fresh and relevant association of thoughts, facts, and
ideas, into a new configuration which pleases, which has meaning beyond the sum
of the parts, which provides a synergistic effect" (p. 225). Kneller (1965) said
creative thought is innovative, exploratory, and venturesome. Risk and
uncertainty stimulate it. Noncreative thought, he said, is cautious, methodical,
and conservative.
Barron (1988) contributed what he called the "ingredients of creativity," which
include:
1. Recognizing patterns
2. Making connections
3. Taking risks
4. Challenging assumptions
5. Taking advantage of chance
6. Seeing in new ways (Barron, 1988, p. 78).
Creativity in Advertising
Blasko and Mokwa (1986) defined advertising creativity as "mental resolution of
contradictory ideas" (Altsech, p. 14). According to Amabile (1982, p. 1001), a
product is creative to the "extent that appropriate observers independently
agree it is creative." Typically, "appropriate" observers are advertising
"creatives"--copywriters and art directors-- who produce advertisements and vote
to bestow industry recognition such as the One Show, Clio and Addy awards.
Taylor et al's (1996) research comparing advertising strategies common in France
and the United States suggests that definitions of creativity differ across
culture. Film producer David Puttnam says "very good creative work falls into
the area of problem solving" (Fletcher, 1992, p. 13). Flandin et. al (1992)
describe the functional role of creativity in advertising as focusing customer
attention on the brand and ultimately leading to its purchase. Bell (1992)
describes advertising creativity as a "kind" of creativity different from the
"concept" of creativity. He said the creativity needed in advertising is
problem-solving creativity, constrained by marketing objectives, competition,
and the organizational approval hierarchy, among other things.
Measuring Advertising Creativity
Much of the research about advertising creativity has focused on it from the
perspective of the advertising professional. According to Flandin et. al (1992),
professionals conduct research as to whether or not proposed advertisements are
interesting, likable, understandable, and believable. The assumption is that a
creative product will "work" (p. 207). Yet, Altsech (1995) posits the consumers'
judgment of "creative" advertising to be important and relevant as
"domain-specific knowledge." In other words, experts may not have the last word,
consumers do (p. 17). Regardless of the perspective, the judge of the creative
product needs an instrument by which to measure creativity.
Subjects in Altsech's (1995) quasi-experimental study of advertising creativity
rated advertisements on 65 items related to originality, appropriateness or
relevance, liking, and excitement or boredom (p. 29). Altsech's factor analysis
of the 65 items identified 14 terms related to originality and five related to
appropriateness. The following terms are related to originality: different,
unusual, original, unique, rare, innovative, unexpected, imaginative, ordinary,
typical, common, predictable, novel, and nonconformist; the following terms are
related to appropriateness: inappropriate, irrelevant, inadequate, fitting, and
confusing (p. 33). According to Altsech, originality in advertising can be
defined as "novelty, unusualness and unexpectedness; the same terms applicable
to creative products in general" (p. 27). Early studies suggest creativity to be
a function of originality and appropriateness (p. 9). MacKinnon (1970) said
appropriateness "must serve to solve a problem, fit the needs of a given
situation; accomplish some recognizable goal" (p. 24). Altsech concluded from
his study that creativity and originality are synonymous, but that
appropriateness is a "qualifier" for creativity (p. 42). That is, assessment of
originality and creativity may be discounted if an advertisement does not fit
the product or audience.
Method
Population and Sample
The samples were drawn from two populations: college students in introductory
mass communication survey courses, and professionals working in advertising
agencies. A total of 189 students participated in the study, 121 of them from
two classes at the University of South Dakota, and 68 from one class at the
State University of West Georgia. A sample of 43 advertising professionals
consisted of 25 individuals from three ad agencies in Sioux Falls, South Dakota,
and 18 individuals from two agencies in Atlanta, Georgia.
Description of Groups
Twenty-eight percent of the college students were communication majors. The rest
were majoring in a variety of disciplines. None of the students had yet taken an
advertising course. Subjects were demographically diverse. The average age was
20.5 years, 51% were female and 49% male, 88% were Caucasian, 6% African
American and 6% identified themselves as Asian, Hispanic, or multi-racial.
Twenty-one percent of advertising professionals identified themselves as account
executives, 60% work in creative services, 12% in agency management, and the
rest in other functions. The mean age of professionals was 32.5 years, 52% were
female and 48% male, 95% were Caucasian and 5% African American. Sixty-nine
percent of the professionals had bachelor's degrees, 5% had completed some
graduate coursework, and 5% had completed a graduate degree. The rest had
completed some college coursework.
Advertisements
Subjects were given two booklets. One was bound with a red cover and labeled
"Advertisements." The other included the subject consent form, instructions, and
assessment forms. The "Advertisements" booklet contained high quality color
photocopies of 15 ads that were selected from the following 12 consumer
magazines that appeal to various demographics:
Travel America (June 1997)
Redbook (July 1997)
Cooking Light (July 1997)
Today's Homeowner (May 1997)
Family PC (Jul/Aug 1997)
Hunting (May 1997)
Ladies Home Journal (July 1997)
Home (July 1997)
Better Homes & Gardens (May 1997)
Weight Watchers (Mar/Apr 1997)
People (July 14, 1997)
Time (July 14, 1997)
Each issue was searched for full-page general-interest advertisements. Ads were
sought that were not age or gender specific. Ads that promoted a variety of
products, represented a variety of styles, and used a variety of colors, fonts,
and appeals were identified. Approximately 35 ads fitting that description were
chosen, and the list was narrowed to include a range of ads from various
categories (three food, three beverage, three health/personal care, two garden,
two pet, and two automobile). One ad had to be discarded and replaced with a
substitute because it could not be adequately reproduced using
color-photocopying technology.
Assessing the Creativity of Advertisements
The Creative Product Semantic Differential Scale (CPSS) was used to assess
creativity in this study. The scale is based on the standard principles of
semantic differential and was developed by researchers at the State University
College at Buffalo (Besemer & O'Quin, 1986; O'Quin & Besemer, 1989) to judge
creative products. Russell (1991) used the scale to judge creative problem
solving by elementary school students. Smith (1993) used it to judge the
creativity of ideas produced by problem solving groups.
The CPSS is similar in purpose to the instrument used by Altsech (1995) to
measure advertising creativity. The CPSS, however, measures the dimension of how
well crafted the creative product is, or how well executed, as well as
originality and appropriateness. Subjects rated the advertisements by circling a
number ranging from 1 to 7 that best described the ads in terms of bi-polar
adjectives. For example, an ad was judged as to whether or not it was
"appropriate" or "inappropriate." A rating of "4," halfway between 1 and 7,
would indicate a neutral response. A rating of "1," closest to the
"inappropriate" side of the attribute, would indicate the strongest association
with the negative aspect of that attribute. A rating of "6," close to the
"appropriate" side of the attribute, would indicate a strong association with
the positive aspect of the attribute.
The complete CPSS uses 55 items on a 7-point scale. The 55 items are divided
into three dimensions: novelty, resolution, and elaboration and synthesis. The
three dimensions, in turn, are divided into 11 subscales. The subscales
associated with the novelty dimension include original, surprising, and
germinal. The subscales associated with the resolution dimension include
valuable, logical, and useful. The elaboration and synthesis dimension contain
five subscales, which include organic, elegant, complex, understandable, and
well-crafted.
Each ad was printed on a separate sheet and clearly linked with a particular
evaluation sheet that was used to judge it. Ads were printed in random order.
The order of the CPSS scales was also random, using computer-generated random
ordering. Eleven of the 15 evaluation scales were presented to subjects in
positive to negative order. The other four were presented in negative to
positive order to discourage evaluators from going down the list of items and
marking all items with one rating. Before statistical analysis was conducted,
adjective pairs were ordered negative to positive, and corresponding numbers
transformed in the order of 1 to 7.
The following is a list of the dimensions, subscales, and individual items from
the revised CPSS (O'Quin & Besemer, 1989) used in this study:
Novelty Dimension
Original Subscale
over used-fresh
predictable-novel
usual-unusual
unique-ordinary
original-conventional
Resolution Dimension
Logical Subscale
illogical-logical
makes sense-senseless
irrelevant-relevant
appropriate-inappropriate
adequate-inadequate
Elaboration and Synthesis Dimension
Well-Crafted Subscale
skillful-bungling
well made-botched
crude-well crafted
meticulous-sloppy
careless-careful
While only three of the eleven subscales were used, representing a total of 15
items, all three dimensions of creativity were represented. Karen O'Quin, one of
the originators of the instrument, recommends using an abridged version of the
CPSS. The longer instrument, she said, was very fatiguing to evaluators and
yielded little improvement in results over using a shorter version (personal
communication, February 24, 1992). In addition, not all subscales are applicable
to all creative products.
Mean scores were calculated for each item and one-way analysis of variance tests
were calculated to determine judgment differences between students and
advertising professionals. In addition, the 15 items were reduced to three
constructs, termed subscales by the originators of the instrument (original,
logical, and well crafted). One-way analysis of variance tests were conducted
on the mean scores of both the individual adjective pairs and the subscales to
determine differences between students and advertising professionals.
The experimental treatment, CPSS instrument, and instructions to the subjects
were pretested on a group of 14 university students who evaluated them for
clarity of instructions. Slight modifications were made to the questionnaire
before proceeding with the study.
Results
Evaluations by Students and Professionals
There was a significant difference in the evaluation of advertisements by
college students and working advertising professionals. Students gave the Minute
Maid Orange Juice ad the highest rating, but professionals ranked that ad 7th.
Students and professionals agreed on three of the top five highest rated ads,
though they ordered them differently. They agreed on only two of the five middle
ranked ads, and on four of the bottom five. The differences were tested using
one-way analysis of variance, and the difference was found significant (F=10.70,
p=0.001). A chi-square test of the rankings confirmed that there were
significant differences (DF=14, p=0.464).
There were not, however, significant differences for all three subscales.
Judgments of the ads using the original subscale, which measured the attributes
original-conventional, novel- predictable, unusual-usual, unique-ordinary, and
fresh-over used, were not significantly different (F=2.02, p=0.155). The logical
subscale, which measures the attributes appropriate-inappropriate,
logical-illogical, makes sense-senseless, relevant-irrelevant, and
adequate-inadequate, showed significant differences between the two samples
(F=5.68, p=0.017). The greatest differences were for the well crafted subscale,
which measured the execution of the ads with such attributes as well
crafted-crude, meticulous-sloppy, skillful-bungling, well made-botched, and
careful-careless. The differences were very significant (F=16.08, p=0.000).
Professionals and students agreed on the originality of ads, but disagreed on
measures of logic and well craftedness. And, overall, their judgments were
different. Hypothesis H1, which posited that there would be no differences in
the judgments of the two groups, was rejected.
Table 1
Rankings of Ads by Students and Advertising Professionals
Student Mean Prof. Mean
Advertisement Ranking Ratings Ranking Ratings
Minute Maid 1 5.389 4 5.173
Orange Juice
Atrovent Nasal Spray 2 5.254 10 4.722
Off Citronella Candles 3 5.120 2 5.343
Plymouth Breeze 4 5.092 5 5.129
Hershey's Syrup 5 5.028 6 4.897
General Foods 6 4.941 3 5.205
International Coffees
Absolut Vodka 7 4.893 1 5.513
Purina One Dog Food 8 4.801 8 4.691
Ortho Weed-B-Gon 9 4.562 9 4.521
Subaru Outback 10 4.549 12 4.130
Kibbles 'n Bits 11 4.543 10 4.309
Neutrogena Sunblock 12 4.423 13 4.044
Comforel Pillows 13 4.251 15 3.549
Gardenburger 14 4.238 11 4.132
Ro-Tel Diced Tomatoes 15 4.201 14 3.576
Demographic Variables
Significant differences in the evaluations of ads were also found with respect
to age, gender, education, income, race, region the respondents come from,
whether they consider themselves rural or urban, the number of years experience
they have working in advertising, and their advertising job title. How
advertising creativity is judged varies significantly based on all sorts of
factors.
Some of the differences are predictable, and correlate strongly with the
differences between students and advertising professionals. Advertising
professionals are older, have more experience, and higher incomes.
How we judge advertising is clearly a function of who we are, as represented by
various demographic and other descriptive variables. Evaluations differ
significantly based on all sorts of differences between individuals. Hypothesis
H2 , which posited that there would be no differences in evaluations of
advertisements based on demographic variables, was rejected.
Table 2
Evaluation Differences by Demographic Variables
Variable F p
Age 3.28 0.000
Gender 18.04 0.000
Education 3.20 0.012
Income 4.01 0.003
Race 4.44 0.001
Region 3.35 0.001
Rural/Urban 4.46 0.012
Experience (years) 3.87 0.000
Experience (job title) 14.14 0.000
Factor Analysis of the CPSS Subscales
The researchers wondered whether factor analysis of the individual items in the
Creative Product Semantic Scale would confirm the relationships that should
exist between the items and the CPSS subscales. In other words, do the five
items that make up the CPSS original, logical, and well crafted subscales load
as distinct factors? A maximum likelihood factor analysis with a Varimax
rotation was conducted on the 15 CPSS items. The results confirm the
relationships between the five items that make up each subscale. All five items
had high loadings (0.6 and above). The five highest loadings for Factor 1 were
the five items that make up the original subscale. The same was true for the
other two subscales. In each case the five items associated with a subscale had
the highest loadings for a factor, and the loadings were high (0.6 or above) or
moderate (0.4 or above). The validity of the CPSS subscales was confirmed.
Discussion
Different groups of people judge the creativity of print advertisements
differently. That is the general conclusion of this study. The situation is
somewhat more complex than that, however. College students and advertising
professionals judged the originality of ads similarly. Where their opinions
differed, perhaps predictably, was with regard to the logic or appropriateness
of the ads, and how well crafted the ads were.
Professionals, whose experience and training should give them greater insights
about these measures of creativity, judged the ads differently on those scales.
In the case of almost every ad, professionals gave ads lower mean scores for the
logic and well crafted subscales than did students. With regard to originality,
however, the mean scores were quite similar. This is not surprising if one
considers that students as general consumers have considerable familiarity with
advertising. While not schooled in technique, perhaps, students may have enough
experience with advertising to deduce what is original.
People of different ages, gender, professional experience, region of the
country, and other demographic groups judged the ads differently. People of
different ages experience the same ad in different ways. Men and women saw the
ads differently. Midwesterners and Southerners had different judgments. Several
factors apparently affect the experience of looking at an ad and judging its
creativity.
What difference does it make? Perhaps the main lesson that advertising
professionals could take from the study is to not assume that their judgments
about the advertisements they create are congruent with those of the general
public that will subsequently experience the ads. Who you are affects how you
experience an advertisement. Creativity is in the eye of the beholder in
advertising, as well as other spheres of creative endeavor.
Advertising is pervasive in Western society and may be examined in terms of its
content, its vehicle, product, and process, among other things. Industry lore
presumes desirable and effective advertising to be creative. Industry awards
are predicated on this assumption. To date, advertising creativity has been
judged primarily by advertising professionals who have training and experience
in the creation of advertising. One may also assume that advertising
professionals attempt to predict that which will capture the attention of the
consumer. This study demonstrates the differences in judgments made by students
and advertising professionals. Additional research on a sample of general
consumers would provide additional insight into the differences between
consumers and advertising professionals. Also, research on creativity as a
function of advertising content might identify common characteristics that are
likely to be thought creative.
References
Altsech, M. B. (1996). The assessment of creativity in advertising and the
effectiveness of creative advertisements. Unpublished doctoral dissertation,
Pennsylvania State University.
Amabile, T. (1982). Social psychology of creativity: A consensual assessment
technique. Journal of Personality and Social Psychology, 43, 997-1013.
Barron, F. (1988). Putting creativity to work. In R. J. Sternberg (Ed.),The
Nature of Creativity (pp. 43-75). Cambridge: Cambridge University Press.
Bell, J. A. (1992). Creativity, TV commercial popularity, and advertising
expenditures. International Journal of Advertising, 11, 165-183.
Besemer, S., & O'Quin, K. (1986). Analyzing creative products: Refinement and
test of a judging instrument. Journal of Creative Behavior, 20(2), 115-125.
Blasko, V. & Mokwa, M. (1986). Creativity in advertising: A Janusian
perspective. Journal of Advertising, 15, 43-50.
Flandin, M. P., Martin, E. & Simkin, L. P. (1992). Advertising effectiveness
research: A survey of agencies, clients and conflicts. International Journal of
Advertising, 11, 203-214.
Fletcher, W. (1990). The management of creativity. International Journal of
Advertising, 9, 1-37.
Kneller, G. F. (1965). The Art and Science of Creativity. New York: Holt,
Rinehart and Winston.
MacKinnon, D. W. (1987) Some critical issues for future research in creativity.
In S. G. Isaksen (Ed.), Frontiers of Creativity Research (pp. 120-129).
Buffalo, NY: Bearly Ltd.
MacKinnon, D. (1970). Creativity: A multi-faceted phenomenon, in J. Roslansky,
ed. Creativity: A Discussion at the Nobel Conference, p. 24. Amsterdam: North
Holland Publishing.
O'Quin, K., & Besemer, S. P. (1989). The development, reliability, and validity
of the revised creative product semantic scale. Creativity Research Journal, 2,
267-278.
Parnes, S. J. (1975). Aha! In I. A. Taylor, & J. W. Getzels (Eds.), Perspectives
in Creativity (pp. 224-248). Chicago: Aldine.
Russell, S. G. (1991). A comparison of approaches to problem solving with
elementary school students. (Doctoral dissertation, Boston University, 1991).
Dissertation Abstracts International, 52, 2505a-2506a.
Smith, Bruce L. (1993). Interpersonal behaviors that damage the productivity of
creative problem solving groups. Journal of Creative Behavior, 27(3), 171-187.
Taylor, R. E., Hoy, M. G., & Haley, E. (1996). How French advertising
professionals develop creative strategy. The Journal of Advertising, 25(1) 1-14.
|
