Types of tests
A/B Test: testing 2 brand new assets (concepts or variations) one against another.
On/Off Test: Test a new concept/variation against an already running asset. Important to turn off the original asset when running the new one.
Insight Validation (Not formal testing)
“Natural Selection”: not a formal test, but a way to validate insights with less precision. Running all assets to be validated together and letting media algorithms choose the winners. Great way to work when agility is a priority
Pros and Cons of Formal Testing
Deciding when to set up a formal test is just as important as the process itself, and it will depend on what you’re prioritizing.
These are some behaviors to map out and make a decision on the type of testing to implement:
Pros Formal Testing:
Has time before airing ads
Produces fewer ads, prioritizing new concepts
Prioritizes accuracy of insights
Cons Formal Testing:
Needs to air assets asap
Produces lots of ads and is constantly making variations
Accuracy not a priority
The duration of a test can vary.
Rule of thumb: leave it on for 5 days and gather results (7 days max).
Minimum investment will also vary.
Rule of thumb: spend $3.00 for every 10,000 people in the potential audience, per creative.
*These are suggestions in order to have an idea of testing size, this does not guarantee statistically relevant results for all situations