Omitted Variable Bias (OVB) is not just a MMM Problem - It is a Marketing Reality
OVB is the invisible hand that affects your MMM and Experiments alike.
One of the most common criticisms hurled at Marketing Mix Modeling (MMM) is:
It can suffer from Omitted Variable Bias (OVB).
True, MMM can suffer from OVB, however the word ‘can’ becomes ‘does suffer’ in case of any experiment.
Every time you run multiple campaigns across multiple platforms, often simultaneously (as every real-world brand does), one would also be committing OVB if running individual experiments.
Why?
Because your experiment on Meta doesn’t account for YouTube running at the same time.
or
Your YouTube test can’t truly account for the influencer push or OOH that overlapped.
In short:
You can’t hold the rest of the marketing universe constant unless you shut off everything - which no marketer can or will ever do.
MMM at least acknowledges the possibility of OVB and there are multiple ways to solve that problem.
The simplest way is to include the omitted variable and the more complex way is to do a 2SLS model (more on this in future posts or you can check out our causality course)
Experiments often pretend OVB doesn’t exist.
Can Experiments control for other factors?
Yes, not all is lost. There are ways to ‘control for’ factors even when running experiments.
The Staggered DiD analysis within our product DiDective does exactly this. There is a provision to include factors that affect the KPI.
OVB is the invisible hand that affects your MMM and Experiments alike. But we can make the invisible, visible again.
You can try DiDective for Free - https://did.arymalabs.com/accounts/login/
Thanks for reading.
For help with Causal Marketing Experiments and MMM, get in touch with us.