Paid Media Decoded

Module 10 Reporting & testing

A/B vs lift studies

Split people into two groups, or hold a group back and measure what the ads added.

8 minBeginner5 steps

Both designs compare groups of people

An A/B test splits people into two groups and shows each group a different ad. A lift study keeps a group out of the ads on purpose, so you can see what would have happened without them. Both are experiments. The difference is whether the second group sees a different ad, or no ad. One design compares ads. The other compares ads with nothing.

An A/B test splits the audience

In a Meta A/B test each version is shown to a segment of your audience, ensuring that nobody sees both, and the test then decides which version performed best. Google Ads experiments split traffic between a control group or campaign and one or more treatment groups. TikTok Split Testing keeps each audience group to one ad group so the two do not compete. An A/B test answers “which version?”

A lift study holds people back

Meta’s Conversion Lift divides the audience into a test group who have the opportunity to see the ads and a control group who do not, and the difference in conversions between them reveals the lift. Google Ads reports Conversion Lift against an unexposed holdback group. TikTok’s Conversion Lift Study compares a test group and a control group the same way. A holdout measures what the ads added, not which ad won.

Some tests do both at once

Google reports A/B plus Conversion Lift, which measures per-arm holdback incrementality and gives head-to-head winner scores between the arms. Meta says you can add a holdout to an A/B test to compare the cost per incremental purchase of each campaign. Here the two ad groups each shrink so a slice can be held back. You can compare versions and measure lift in one study.

Same design, different names

Google Ads has experiments, Conversion Lift and Brand Lift. Meta has A/B tests, Conversion Lift and brand survey tests that measure Brand Lift. TikTok has Split Testing, Conversion Lift Studies and Brand Lift Studies. Pinterest lets you create an A/B test with up to four versions. Snapchat documents Split Testing and Brand Lift. Translate the test name before you compare the result.

Platform
Interactive diagram
People-dot force field · Default state
SampleValue
People in the sample240
Held back20%
ModesA/B split · Holdout · A/B + holdout

Analytics settings

Allow Google Analytics? It helps us see which lessons people use. You can change this in Analytics settings.

Search the course

Type to find lessons, terms and sources.