Multimedia Mix Modeling: How to Tweak It to Maximize Model Accuracy
Multimedia Mix Modeling (MMM) is an awesome holistic attribution method gaining a lot of traction as the viability of traditional click-based online measurement collapses. Meta’s Robyn open-source library, which I’ll discuss below, can help with revenue attribution and optimal budget allocation across channels – but be aware that you’ll need to do quite a lot of tweaking for more accurate results.
What tweaks? Glad you asked. Let’s dive in!
What is model accuracy?
Before we get to the tweaks, let’s set a baseline of understanding. In the field of machine learning, model accuracy is the measurement used to determine which model is best at identifying relationships and patterns between variables in a dataset based on the input. In the context of MMM, we can use the R² coefficient of determination to measure how well the regression predictions approximate the real data points on a percentage scale, that is 0%-100%. An R² of 100% indicates that the regression predictions perfectly fit the data. Model accuracy example with R²=89% indicating strong fit:
Split media channels into sub-channels to gain more variance
It is critical to split media channels into meaningful sub-channels to gain more variance. For example, it probably makes sense to split Facebook into Facebook Retargeting and Facebook Prospecting, and account for differences in historical performance, saturation curves, and incremental impact.
Similarly, it probably makes sense to split Google into Google Brand and Google Non-Brand . But don’t get crazy with segmentation; from my experience, splitting channels really only makes sense up to 10 sub-channels or so, as further splitting is likely to cause overall model accuracy to decline. In short, the way historical channel spend is broken out and used with Robyn matters a lot more than you might think.
Example of channel split into sub-channels:
Add organic variables as inputs
Besides historical paid media spend, I have had a lot of success refining MMM fit through the addition of organic variables. From the Robyn documentation: “Robyn enables users to specify organic_vars to model marketing activities without direct spend. Typically, this may include newsletter, push notification, social media post stats, among others [...] Examples of typical organic variables:
Reach / impressions on blog posts
Impressions on organic & unpaid social media
SEO improvements
Email campaigns
Reach on UGC“
This means that, while not all marketing initiatives have direct spend associated with them, all marketing activities have probably played a role throughout the user journey.
Example of historical paid media spend and organic variables used as inputs:
Add contextual variables as inputs
Similarly to organic variables, contextual variables can be a powerful way to refine MMM fit. More specifically, contextual variables aren’t marketing activities per se. Instead, they reflect externalities or company-wide changes such as:
Competitors
Price & promotion
Temperature
Unemployment rate
Interest rate
Global trends (e.g. COVID)
Adding these contextual variables will therefore further help with MMM model accuracy.
For better results, run your marketing initiatives with MMM in mind
While MMM is a great way to account for many moving pieces at once and handle multicollinearity challenges, I recommend you limit the number of pieces moving at once in order to maximize learnings.
More specifically, ramping up multiple paid media channels at the exact same time may cause multicollinearity challenges to multiply, and model accuracy to decrease. Conversely, dialing paid media channels up or down at different times may help mitigate multicollinearity challenges, hence increased model accuracy.
In this article, we have covered how to best configure MMM for maximizing model accuracy for better attribution of past user activity and smarter predictions for where and how to spend your budget.. As you can tell, there’s a lot going into it, and the stakes are high. At Blackbird, we’re big fans of MMM and have set it up for a variety of accounts across industries.
Get in touch if you’re curious about how to leverage it for your marketing campaigns!
