Causal ai, which explores the integration of causal reasoning into machine learning
Welcome to my series on Causal ai, where we'll explore integrating causal reasoning into machine learning models. Expect to explore a number of practical applications in different business contexts.
In the last article we covered Validate the causal impact of the synthetic control method.. In this article we will move on to improve marketing mix modeling with Causal ai.
If you missed the last article on synthetic controls, check it out here:
Current challenges with digital tracking have led to a recent resurgence of marketing mix modeling (MMM). At the recent Causal ai conference, Judea Pearl suggested that marketing may be the first industry to adopt Causal ai. So I decided it was time to start writing about my learnings from the last 7 years in terms of how MMM, causal ai, and experimentation intersect.
The following areas will be explored:
- What is MMM?
- How can causal ai improve MMM?
- What experiments can we perform to complete the triangulation?
- Featured challenges within marketing measurement.
The complete notebook can be found here:
MMM is a statistical framework used to estimate how much each marketing channel contributes to sales. It is heavily influenced by econometrics and in its simplest form is a regression model. Let's cover the basics of the key components!
Regression
A regression model is constructed in which the dependent/target variable (usually sales) is predicted based on several independent variables/characteristics; These usually include spending on different marketing channels and external factors that can affect demand.
The coefficients of the expense variables indicate how much they contribute to sales.
The PyMC marketing package in Python is a great place to start exploring MMM:
advertising stock
Advertising stock refers to the persistent effect of marketing (or advertising) investment on consumer behavior. Helps model the long-term effects of marketing. It is not common behavior to rush to buy a product the first time you hear about a brand; The idea of advertising stock is that the effect of marketing is cumulative.
The most common advertising stock method is geometric decay, which assumes that the impact of advertising decays at a constant rate over time. Although this is relatively easy to implement, it is not very flexible. It is worth checking out the Weibull method, which is much more flexible. The PyMC marketing package has implemented it, so be sure to check it out:
Saturation
Saturation in the context of marketing refers to the idea of diminishing returns. Increasing marketing spend can increase customer acquisition, but as time goes on it becomes more difficult to influence new audiences.
There are several saturation methods we could use. The Michaelis-Menton function is common. You can also check this out in the PyMC Marketing Pack:
MMM frameworks typically use a flat regression model. However, there are some complexities in the way marketing channels interact with each other. Is there a tool in our Causal ai toolbox that can help with this?
Causal graphs
Causal graphs are great for untangling causes from correlations, making them a great tool for addressing the complexities of how marketing channels interact with each other.
If you're not familiar with causal graphs, use my previous article to get up to speed:
Understanding the marketing graph
Estimating the causal graph in situations where poor domain knowledge is available is challenging. But we can use causal discovery to help us get started. See my previous article on causal discovery for more information:
Causal discovery has its limitations and should only be used to create an initial hypothesis for the graph. Fortunately, there is a wealth of knowledge about how marketing channels interact with each other that we can take advantage of.
Below I share the knowledge I've gained from working with marketing experts over the years…
- PPC (paid search) has a negative effect on SEO (organic search). The more we spend on PPC, the fewer SEO clicks we get. However, we have a major confounding factor…demand! A flat regression model will not capture this complexity which often leads to an overestimation of PPC.
- Social spending has a strong effect on social clicks: the more we spend, the more potential customers click on social ads. However, some potential customers may see a social ad and visit your site the next day through PPC, SEO, or Direct. A flat regression model will not capture this halo effect.
- A similar case can be presented for brand spend, where you target potential customers with longer-term brand messages, but without a direct call-to-action to click. These potential customers may visit your site through PPC, SEO or Direct at a later stage after learning about your brand.
- He clicks They are mediators. If we perform a flat regression and include mediators, this can cause problems when estimating causal effects. I won't cover this topic in too much detail here, but using causal graphs allows us to carefully control for the right variables when estimating causal effects.
Hopefully you can see from the examples above that using a causal graph instead of a flat regression will seriously improve your solution. The ability to calculate counterfactuals and make interventions also makes it very attractive!
It's worth noting that it's still worth incorporating ad stock and saturation transformations into your framework.
When working with observational data, we should also strive to conduct experiments to help validate assumptions and complement our causal estimates. There are three main tests available for use in acquisition marketing. Let's dive into them!
Conversion lift tests
Social platforms like facebook and Snapchat allow you to test conversion increases. This is an AB test where we measure the increase in conversion using a treatment group versus a control group. These can be very useful when evaluating the counterfactual of your causal graph for social spending.
Geographic elevation tests
Geographic elevation tests can be used to estimate the effect of marketing cuts or when starting a new channel. This can be especially useful for digital and TV brands where there is no direct call to action to measure. I cover this in much more detail in the last article:
Try again
PPC campaigns can be scheduled to go on and off every hour. This creates a great opportunity for backtesting. Schedule PPC campaigns to activate and deactivate every hour for a few weeks, and then calculate the difference between the number of PPC + SEO clicks in the activation and deactivation period. This will help you understand how much PPC SEO can capture and therefore evaluate the counterfactual of your causal graphs for PPC spend.
I think running experiments is a great way to modify and then gain confidence in your causal graph. But the results could also be used to calibrate your model. Take a look at how the PyMC team has approached this:
Today I discussed how MMM can be improved with Causal ai. However, causal ai cannot solve all procurement marketing challenges. And unfortunately there are many of them!
- Spending following demand forecast — One of the reasons marketing spending is highly correlated with sales volume may be because the marketing team spends in line with a demand forecast. One solution here is to randomly change spending between -10% and +10% each week to add some variation. As you can imagine, the marketing team is usually not very interested in this approach.
- Demand Estimation — Demand is an essential variable in our model. However, it can be very difficult to collect data. A reasonable option is to pull Google Trends data on a search term that aligns with the product you sell.
- Long-term brand effects — The long-term effects of branding are difficult to capture, since there is usually not much information about it. Long-term geographic elevation testing can help here.
- Multicollinearity — This is actually one of the biggest problems. All the variables we have are highly correlated. Using ridge regression can alleviate this somewhat, but it can still be a problem. A causal graph can also help a little, as it is essential that you break down the problem into smaller models.
- Marketing team buy-in — In my experience, this will be your biggest challenge. Causal charts offer a nice visual way to engage the marketing team. It also creates an opportunity for you to build a relationship as you work with them to agree on the complexities of the chart.
I'll close things there. It would be great to hear what you think in the comments!