Retail Media Measurement: Here Be Dragons
Retail media is now a major global advertising channel. The attractiveness of retail media for advertisers is enhanced by the promise of closed-loop measurement. Retail media networks can directly map ad exposure to shopping and purchase behaviour, which enables continual monitoring and optimisation of ad campaigns. Advertisers can easily navigate their way to success.
It’s marvellous. It’s simple.
Isn’t it?
Unfortunately, it’s not all plain sailing. Indeed, the tailwinds retail media networks can provide to advertising effectiveness, specifically the greater ability to target customers, take us further into the choppy waters of the sea of ad effectiveness measurement. Here be dragons. Retail media adds a layer of complexity which further exacerbates an insidious problem in advertising measurement: endogeneity. It’s a problem that has been steadily growing in-line with the digital share of ad spend, and it is all too often overlooked. If left unchecked, it can cause major problems with analysis, leading to incorrect conclusions and inaccurate predictions; if you see results from advertising effectiveness studies that raise an eyebrow, the chances are that the analysis suffers from endogeneity.
Endoge…what?
Endogeneity occurs when a variable is correlated with the error term in a statistical model, leading to biased or inconsistent estimates. In simpler terms, endogeneity occurs when there is a two-way relationship between the independent and dependent variables in a model.
When analysing the effectiveness of advertising campaigns, endogeneity arises in several contexts. Here are some examples, specific to digital advertising:
Ad placement and user engagement: Ads are placed where they are more likely to attract engaged users or on high-traffic pages. This non-random placement can lead to endogeneity because user engagement with the ad may be influenced by factors other than the ad itself, such as the quality of the content surrounding the ad.
Search engine advertising: Similarly, endogeneity can occur in Search when advertisers bid on keywords based on their expected effectiveness in driving clicks and conversions. However, the observed effectiveness of keywords is influenced by factors such as user intent, search engine algorithms, and competitive bidding strategies.
Behavioural targeting: Advertisers target ads to users based on their online behaviour, such as websites visited, search history, or demographic information. However, users who are targeted with ads may differ systematically from those who are not targeted. For example, users who exhibit certain online behaviours may also be more likely to respond positively to ads, regardless of the targeting strategy.
And the list could go on e.g. Ad retargeting and Ad blocking. In each of these examples, endogeneity arises because the relationship between advertising exposure and outcomes is influenced by factors other than the advertising itself. The inevitable result is that outcomes are falsely attributed to advertising and performance is inflated.
It’s important to note that the measurement of ALL advertising can be affected by endogeneity. For example, the evaluation of Christmas TV campaigns can easily be impacted by endogeneity: ads cause increased sales, but also the (anticipated) increase in sales at Christmas causes increased TV ads. However, the likely influence of endogeneity is more prevalent in digital advertising due to the greater ability to target and personalise ads. The problem of endogeneity is intensified with the growth of retail media as the strength of the signals offered by retail media networks further increase the precision with which advertisers are able to place and personalise their ads. Armed with the results and insights from analysis affected by endogeneity, advertisers can easily make the wrong decisions, wasting money and stifling business performance.
Avoiding endogeneity
There is a simple path to easier measurement of ad effectiveness: random targeting of ad campaigns. Of course, this would be daft (with the exception of relatively small-scale experiments and academic exercises). Better data, enabling better targeting of more personalised advertising inevitably leads to better outcomes. But it’s very hard to know exactly how much better and whether the incremental benefit outweighs the cost premium. It’s very easy to exaggerate the results. So, how can we create better ad campaigns AND better measure the true impact, avoiding the impact of endogeneity?
A key to success is the integration of multiple data sources, bringing together the likes of 1P data with 3P and offline data to create a more comprehensive view of customer behaviour. By combining these data, advertisers can better understand the drivers of customer behaviour and reduce the likelihood of endogeneity by controlling for additional factors that may influence user engagement and outcomes.
A more complete dataset naturally provides a more complete view of performance. It also supports analysts in the use of techniques such as experimental design and Marketing Mix Modelling (MMM); the best approaches to isolate the true causal effect of advertising exposure on outcomes whilst controlling for confounding factors. But, the analysis must be done right. Marketing Mix Modelling based on a perfect dataset can still be riddled with endogeneity. For example, the failure to properly capture seasonality in the evaluation of Christmas TV ads will yield meaningless results. And there are numerous other common MMM mistakes, including:
Price: The inclusion of price as an explanatory factor in a model of sales revenue is very problematic; model results are often not just wrong, but of the wrong sign i.e. they show a negative relationship when the true relationship is positive. A sales volume measure should always be the preferred metric.
Brand Search: Brand Search volumes are not independent as they are a direct function of the rest of the marketing and media plan. These relationships need to be captured in your framework to avoid misattribution.
Impressions, Spend, Clicks: Clicks should not be included in the model. Impressions are the preferred data as it better reflects the reach of the campaign, rather than clicks which partly reflect intent and/or a higher underlying propensity to convert.
Again, the list could go on!
Addressing endogeneity is crucial for obtaining reliable and unbiased results. At Entropy MMM, we use various techniques to mitigate endogeneity, such as instrumental variables, control function approaches, fixed-effects models, and structural equation models. The application of these techniques ensures that our results are more accurate and robust. So, if you have been presented results that are questionable or just plain wrong, it’s highly likely that endogeneity is present. Contact us if you think your ad effectiveness results may be suffering from endogeneity and we’ll be happy to discuss a diagnosis and potential remedies.
