Glossary/Incrementality
What is incrementality?
Incrementality is the portion of attributed sales that would not have occurred without the advertising — the causal effect, as opposed to the credit. An ad that appears in front of a shopper who was already going to buy generates attributed revenue and zero incremental revenue. Because attribution assigns credit by proximity in time rather than by causation, reported ROAS is an upper bound on true causal return, and the gap between them is often large. Measuring incrementality requires withholding the ad from a comparable group and comparing outcomes; nothing in a standard ad report can substitute for that comparison.
01/Formula
Formula
Incremental sales = Sales(exposed) − Sales(withheld) Incremental ROAS = Incremental sales / Ad spend Incrementality rate = Incremental sales / Attributed sales
Example
A campaign reports ฿300,000 attributed revenue on ฿50,000 spend — ROAS 6.0. A geographic holdout shows withheld regions ran at 82% of exposed regions. Incremental revenue = 300,000 × 0.18 = ฿54,000 Incremental ROAS = 54,000 / 50,000 = 1.08 At a 20% contribution rate, break-even is 5.0. The campaign is deeply unprofitable on a causal basis while reporting a ROAS of 6.
02/In detail
How do you measure incrementality?
By withholding. The credible designs all rest on the same idea: create a group that does not see the ad, keep it otherwise comparable, and measure the difference. Geographic holdouts split regions; time-based tests alternate on and off periods on a fixed schedule; audience holdouts, where a platform supports them, randomise at the user level. The hard part is statistical power rather than mechanics. Order counts on a single marketplace campaign are small and volatile, so a test needs either a large effect or a long run to distinguish a real lift from ordinary week-to-week variance — which is precisely why so few sellers ever run one, and why the intuition-based alternative persists.
Which campaigns are least incremental?
Brand and exact-model-number search, consistently. In a large field experiment on eBay, Blake, Nosko and Tadelis found that returns to branded paid search were close to zero — shoppers who searched the brand overwhelmingly arrived anyway through organic results when the ads were switched off. The marketplace analogue is a shopper searching your exact product name and clicking the sponsored placement that sits above the organic one you would have won regardless. Retargeting an existing cart is the second usual suspect. Broad discovery terms and genuinely new-audience placements sit at the other end: lower reported ROAS, far higher incrementality.
What can you do without a formal test?
Several cheap approximations catch the worst cases. Compare attributed revenue growth against total shop revenue growth: if attributed revenue rises while the shop total is flat, the campaign is reallocating credit rather than creating demand. Watch what happens to organic orders on a SKU when its campaign scales — a fall that mirrors the ad-driven rise is cannibalisation in plain sight. And pause a campaign deliberately for a defined period and read the shop-level effect rather than the campaign-level one. None of these is a controlled experiment, and all of them are better than assuming the attributed number is causal.
03/Why it matters
The trap, in one paragraph.
Every budget decision made from attributed ROAS implicitly assumes incrementality of one hundred percent. Measured incrementality on mature marketplace accounts is frequently a fraction of that, which means a campaign clearing break-even on reported numbers can be losing money on real ones. This is the largest single gap between what ad reporting says and what a shop’s bank balance does, and it is invisible without deliberately withholding something.
Common mistake
Treating a strong before-and-after result as an incrementality test. Turning a campaign on during a promotional period and observing more sales confounds the ad with the promotion, the season, and the platform’s own traffic. Without a comparable withheld group measured over the same period, a before-and-after comparison measures the calendar.
04/In DataGlass
How Incrementality is used in DataGlass.
DataGlass compares advertised products against a control set of the seller’s own comparable, unadvertised products over the same window, and reports the difference as an observed gap rather than as a proven causal lift. Where the comparison is too thin to be informative, the surface says so instead of producing a number.
05/Sources
- [1] Blake, Nosko & Tadelis — Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment (NBER Working Paper 20171)
The eBay experiment that found returns to branded paid search close to zero once the ads were switched off in randomised markets.
- [2] Lewis & Rao — Measuring the Effects of Advertising: The Digital Frontier (NBER Working Paper 19520)
Documents how large sales volatility is relative to typical advertising effects, and therefore how much data an honest incrementality test requires.