Wonder
Sign up
Log in
Research Outline
Prepared for Marsha R. | Delivered May 6, 2020
Ad Testing Pitfalls
Review your project details
Goals
Gain an understanding of potential pitfalls and issues surrounding creative testing such as is used for ads, in order to support a client presentation regarding how testing results and actual market impact are not always aligned.
View less
Early Findings
Data Integrity
One key challenge with testing creative ads is
maintaining data integrity.
One source notes
that it's not typical to have test data come out evenly, with some ads getting more impressions than others, which often creates noise in the data.
The lack of even data makes it challenging to use data sets from testing to
accurately determine the winning ad.
It's also easy for marketers to
overlook data inconsistencies
, and therefore overlook factors that are leading to conversions because of these variances.
History & Selection Effects
Specific challenges to A/B creative testing include
history effects and selection effects.
History effects
occur when testing is run between two different times without accounting for the impacts of timing, such as 20 hours on a weekday one month and then 20 hours on a weekend another month, or during a holiday season peak vs. low season.
Selection effects
involve testing only one element of ads, without accounting for the impact of other marketing strategies being run simultaneously.
For example, testing one ad, while simultaneously running a pay-per-click ad on another platform that is directing traffic to a website, and not differentiating the results could
cause data issues
that make it hard to determine the impact of a campaign.
Sampling distortion
effects can also be an issue, which are primarily caused when the sample size is not large enough to overcome random chance.
Selecting Testing Approach
Because there are so many
potential testing strategies,
it's often challenging to determine which type of tests will yield the most accurate or useful results.
For example, Facebook ads
offer a split-test feature
, but for localized or custom ads, a more complex approach may be more effective.
Knowing what to test
within an ad is also often a challenge, because involves marketers being able to determine which changes will both be easy to implement and create a measurable impact.
Ad testing methods need to be selected in a way that generates the most information on why the customer behaves the way they do that the marketer can apply the knowledge and have a
f
o
u
n
d
a
t
i
o
n
a
l
understanding of
what motivated the behavior.
View less