Glossary/Thompson Sampling
What is thompson sampling?
Thompson Sampling is a strategy for making repeated decisions under uncertainty: instead of acting on the single best estimate, draw one plausible value at random from the range of things that could be true, and act as if that draw were correct. Repeat every cycle. Options the system is confident about get chosen consistently, because their range is narrow. Options it is unsure about get chosen sometimes, in proportion to how plausible it is that they are good — so they get tested without anyone reserving a test budget. Applied to advertising, it is how a system keeps funding proven campaigns while continuing to learn about new ones.
01/Formula
Formula
each cycle: draw one plausible parameter set from the posterior choose the action that would be best if that draw were true A narrow posterior → nearly the same action every time (exploit) A wide posterior → varied actions across cycles (explore)
Example
A new Shopee campaign with three days of data. Plausible daily order rates range from near zero to well above the account average. Some cycles draw a pessimistic value and it is funded at the floor. Some cycles draw an optimistic value and it is funded properly. After two weeks the range has narrowed and the funding stabilises around whatever the campaign actually is.
02/In detail
What problem does exploration solve?
The one where a system optimises itself into ignorance. Always funding the currently best-looking option means never gathering the evidence that would reveal a better one, and in advertising this bites hard, because a new campaign, product, or keyword always starts with a worse-looking record than an established one — it has no record. A purely greedy allocator therefore kills new campaigns before they have produced enough data to be judged, and slowly concentrates the entire budget on whatever happened to work first. Exploration is the correction, and the reason to do it by sampling rather than by reserving a fixed test budget is that sampling spends the exploration effort in proportion to how plausible it is that the uncertain option is actually good.
Why can a single draw be badly wrong?
Because a draw from a wide range can land anywhere in it. On a campaign with almost no data, consecutive cycles can produce wildly different implied optima — near zero one day, very high the next — and that is the method working as designed, not a malfunction. The danger is a system that reads one pessimistic draw as evidence and acts irreversibly on it. The design rule that follows is important: while a campaign is still in the exploring regime, a pessimistic draw is a reason to keep sampling at a floor budget, never a reason to shut it off permanently. Hard-stopping an under-observed campaign destroys the very information the method exists to gather.
When should a campaign stop exploring?
When its uncertainty has narrowed enough that the full distribution can be trusted, at which point acting on one random draw wastes information. Mature campaigns are better served by optimising against the whole outcome distribution — the expected result and its downside tail together — rather than against a single sampled scenario. Practically this means a system needs two regimes and a rule for graduating between them, usually based on how many periods of real performance data a campaign has accumulated. Applying the exploration regime to a mature campaign adds pointless variance; applying the mature regime to a new one kills it early.
03/Why it matters
The trap, in one paragraph.
Every ad account faces the same trade-off between funding what is known to work and finding out what else might. Handled by instinct, it usually collapses into one of two failure modes: never testing anything new, or running perpetual tests that never conclude. Sampling in proportion to uncertainty resolves it automatically, and it is the reason a well-built allocator will keep a small budget on a campaign that currently looks mediocre rather than reallocating every baht to the current leader.
Common mistake
Reading day-to-day variation in a new campaign’s budget as instability. On an under-observed campaign that variation is the exploration doing its job. The thing to check is not whether the budget moved, but whether the campaign is being hard-stopped on a single bad draw — that is the actual failure mode.
04/In DataGlass
How Thompson Sampling is used in DataGlass.
DataGlass treats under-observed Shopee campaigns differently from mature ones: while a campaign is still being learned about it is funded from sampled scenarios and floored rather than hard-paused, and it moves to full-distribution optimisation once it has accumulated enough performance history. Which regime a campaign is in is visible on the recommendation.
05/Sources
- [1] Russo, Van Roy, Kazerouni, Osband & Wen — A Tutorial on Thompson Sampling (arXiv:1707.02038)
The standard reference on posterior sampling for sequential decisions, including why sampling in proportion to uncertainty balances exploration against exploitation.
- [2] Thompson sampling — overview
Background on the method, its origin as a solution to the multi-armed bandit problem, and its regret properties.