Incrementality Testing: How to Know If Your Ads Are Actually Working

Attribution answers "which touchpoint gets the credit?" Incrementality testing answers a completely different, arguably more important question: would this conversion have happened anyway, even without the ad?

The core idea

Every attribution model — first-touch, last-touch, data-driven, anything — assumes that a touchpoint it can see contributed to the conversion. Incrementality testing doesn't assume that; it tests it directly, by comparing outcomes between a group exposed to the marketing and a group that wasn't, and attributing the difference to the marketing itself.

A customer who was already going to buy from you — because they searched your brand name directly, or came back from a previous positive experience — will often still convert even if the ad they technically "saw" had never run. Attribution would still credit that ad. Incrementality testing wouldn't.

The two main testing methods

Conversion lift / holdout tests

A platform (or your own setup) randomly withholds ads from a defined holdout group of users while showing them normally to everyone else, then compares conversion rates between the two groups. The difference is the incremental lift the ads actually caused.

Geo holdout experiments

Instead of holding out individual users, campaigns run normally in some geographic markets ("treatment") and are paused in similar comparable markets ("control"). Comparing sales trends between the two groups of markets over the test period — typically using a difference-in-differences analysis — estimates the incremental effect without needing any individual-level tracking at all, which makes it resilient to the same privacy-driven data loss covered in server-side tracking.

Why this matters more than it might seem

This is especially relevant for channels that are very good at capturing existing demand (branded search, retargeting) versus channels that are better at creating new demand (broad prospecting, top-of-funnel video). Attribution alone tends to systematically favor the former; incrementality testing is one of the few methods that can fairly evaluate the latter.

How this fits with marketing mix modeling and attribution

Incrementality testing, marketing mix modeling, and multi-touch attribution aren't competing choices — they sit at different levels of rigor and granularity. A common pattern: use incrementality tests periodically to calibrate and sanity-check what MMM and MTA are reporting, since incrementality is the only one of the three that measures actual causation rather than inferring it from patterns.

Getting started without a data science team

• Most major ad platforms (Google, Meta, TikTok) offer built-in conversion lift testing tools for accounts above a certain spend threshold
• A simple geo holdout can be run manually — pause a campaign in a handful of comparable regions for a defined period and compare sales trends against regions where it kept running
• Start with your highest-spend, most-uncertain channel rather than trying to test everything at once