Claude Code and Jevons Paradox in MMM
How Claude and LLMs could democratize MMM - but at what cost? The hidden risks of AI-driven modeling and why statistical rigor still matters.
There’s a lot of excitement around Claude.
Naturally, the question is: Will this make building Marketing Mix Models (MMM) easier?
Short answer: Yes.
But what happens when MMM becomes easy?
There is a concept of AI Slop. Because it has become easier to generate content, anybody with an access to LLM can write a surface level well sounding article on any subject.
Similarly we will see MMM Slop. MMM that looks good on the surface but misses crucial pieces.
📌 Enter Jevons Paradox
Jevons Paradox states: When efficiency increases, consumption doesn’t go down. It goes up.
If Claude makes MMM - Faster to build, Easier to iterate and Cheaper to deploy; then we won’t see fewer MMMs. We will see a flood of MMMs.
📌 Jevons Paradox - First open-source MMM now Claude
When open source MMM libraries like Robyn and Meridian came onto the scene, many opined that it would be the end of traditional MMM vendors.
But the opposite happened. Firstly, more wrapper MMM companies mushroomed across the globe. And then because these libraries never had exhaustive pieces to make a really accurate and comprehensive models, the clients decided to go back to the traditional MMM vendors either to
1) Fix their open source models
2) Create a mixed approach where the traditional vendor builds their model (acting like a north star) and then the open source models are modeled after it.
3) or adopt vendor built model
2 and 3 is what has happened most in our experience
📌 The Bottleneck in MMM Was Never Code
The bottleneck was never writing code for model fitting, Adstocks, budget optimization etc.
The real difficulty has always been:
- Framing the right problem
- Choosing the right variables
- Handling confounding
- Interpreting outputs correctly
- Convincing clients to act on insights
📌 Claude context window
I came across Svet Semov's interesting post on Claude context window.
Claude context window can encapsulate boiler plate MMM code.
But once real client data are added, context window will not be enough.
Anyhow Claude’s context window coverage doesn’t mean it captures all the underlying know-how !!
📌 Next Token prediction will not lead to Accurate MMM
We leverage AI a lot and have designed some innovative products. But still we build the MMM model itself the good ol statistical way.
Currently, AI will do the following reliably:
- Dissemination of information and report generation
- Mathematical functions like budget optimization (with guardrails)
- 'Reasoning' to a certain extent based on training.
Perhaps sooner we will have a paradigm new architecture that can help build MMM causally and accurately. But we are not there yet. I remain hopeful and my company is researching on it.
Claude will lead to more MMMs. But it will increase the cost of being wrong with confidence too.
The mantra hence should be - MMM model built with statistics and domain knowledge but AI enhancing downstream functions.
The future of AI + MMM is Peripheral Agentic MMM
Thanks for reading.
For help with MMM, Causal Marketing Experiments and Experimentation, get in touch with us.
We also build some pretty cool AI products to aid Marketing Measurements. Check out our products page to know more -
 | A guest post by
|