Let's not forget the 'C'- controlled in RCT.
Randomization alone is not enough. Control is also equally important in Marketing Experiments
A lot of people have the misconception that just because they have done randomization, their experiments are Randomized Controlled Trial grade.
This misconception is more prevalent in Marketing and among marketers.
Firstly lets understand what is Randomization.
Randomization means that every unit (e.g. people) have equal chance of being assigned either to the control arm or the treatment arm.
Randomization ≠ control
If Randomization were to be equal to controlling, we would not be abbreviating RCT as Randomized control trials. We could as well be saying Randomized trials.
Marketing Experiments miss the ‘C’ control
The reason why marketing experiments can’t be called RCT is because in marketing we can’t ‘control’ for a lot of factors.
There will always be spillover. There will always be SUTVA (Stable unit treatment value assumption) violation.
This is a one of the foundational assumption in causal inference that states a unit’s outcome depends only on the treatment it receives and not on the treatment of other units.
But we all know we can’t imprison people in one room and make them watch only TV ads, TikTok ads, META ads or Google ads.
People are free agents and they will get influenced by variety of factors.
I particularly like what Ron Kohavi said in one of his post
“The issue is the name A/B test, because it implies to many that this is a properly *randomized* controlled experiments, which it is not. This misunderstanding has led to multiple peer-reviewed papers published based on this misunderstanding, which we are now attempting to highlight.
Tesla’s FSD (Full Self-Driving) is also useful, but the use of “full” is misleading and it’s important to highlighting to drivers that this is just SAE level 2 driver-assistance, requiring the human driver to watch the road and be ready to intervene.”
Quasi Causal Experiment is not a bad word
Just because we can’t perform RCT in marketing, it doesn’t mean the end of the world. We can still perform Quasi Causal Experiments. And when done well they are almost equivalent to a RCT.
If you wish to develop first principles understanding of causality and causal experiments, you can check out our comprehensive causality course hosted in our LMS: https://mmm-courses.arymalabs.com/causality-and-causal-experiments/
Thanks for reading.
For help with MMM, Causal Marketing Experiments and Experimentation, get in touch with us.
 | A guest post by
|