Types of tests

Formal Testing

  • A/B Test: testing 2 brand new assets (concepts or variations) one against another.

  • On/Off Test: Test a new concept/variation against an already running asset. Important to turn off the original asset when running the new one.

Insight Validation (Not formal testing)

  • “Natural Selection”: not a formal test, but a way to validate insights with less precision. Running all assets to be validated together and letting media algorithms choose the winners. Great way to work when agility is a priority

Pros and Cons of Formal Testing

Deciding when to set up a formal test is just as important as the process itself, and it will depend on what you’re prioritizing.

These are some behaviors to map out and make a decision on the type of testing to implement:

Pros Formal Testing:

  • Has time before airing ads

  • Produces fewer ads, prioritizing new concepts

  • Prioritizes accuracy of insights

Cons Formal Testing:

  • Needs to air assets asap

  • Produces lots of ads and is constantly making variations

  • Accuracy not a priority

General Recommendations*

  • The duration of a test can vary.

    • Rule of thumb: leave it on for 5 days and gather results (7 days max).

  • Minimum investment will also vary.

    • Rule of thumb: spend $3.00 for every 10,000 people in the potential audience, per creative.

*These are suggestions in order to have an idea of testing size, this does not guarantee statistically relevant results for all situations

A Digital Campaign Testing Structure and Variables

Did this answer your question?