Measurement: From Belief to Evidence

Measure for success

Advertisers plan their media spending down to the last cent. They should plan for the evidence just as carefully.

As we approach the end of the current year, planning for the next begins. Budget season is when companies argue about money in earnest. Sales targets are set, growth ambitions are haggled over and every brand and market pleads its case. Assumptions get challenged. Eventually somebody decides where the money goes.

Then comes the media. How much is needed? Which channels, which markets, and what should change from last year? Agencies recommend, platforms pitch and spreadsheets multiply. Millions of dollars, sometimes billions, are committed.

One question rarely makes the agenda: how much will it cost to find out whether any of it worked? The measurement budget discussion if at all is at the bottom of the agenda.

Firms plan the spending in fine detail but leave the evidence to chance. Every serious advertiser should start the year with a measurement budget and a plan for using it.

Belief-based marketing

Every marketing plan rests on beliefs, such as that the media strategy will work and that certain channels will drive growth. Nothing wrong with that; nobody can invest without assumptions. The trouble starts when assumptions are mistaken for knowledge.

That becomes obvious when sales disappoint. Was the fault in the media, the creative, the targeting, distribution or price? Did one channel flop, or was media innocent all along? Without proper measurement a firm has hypotheses, perhaps well-informed ones, but no evidence.

Measurement cannot banish uncertainty, nor answer every question. What it can do is make the next decision a better-informed one.

What it’s worth

Knowing the return on each channel helps decide where to spend. It also tells you what to pay. A channel that falls short need not be dropped. The numbers give the buyer grounds to demand a lower price from the media owner, which may make the channel worth keeping. Evidence shapes where the next dollar goes and how much it should buy.

Minding the money

There is a broader principle. Marketing is a big investment, and investing responsibly means more than allocating funds. It means establishing what the money achieved and what was learned and letting that shape the next decision. Measurement is therefore part of how marketing spending is governed. Good governance holds people to account for last year’s spending, but its real purpose is to improve next year’s.

Too late to count

There is a practical reason, too, why measurement cannot be an afterthought: by the time someone decides to measure, it may no longer be possible. Measurement may need holdouts to be planned in advance. The infrastructure required for measurement cannot be built overnight. Studies may need a baseline taken before the campaign starts. Data needed tomorrow often has to be collected today.

Looking back at a campaign can be revealing. Planning the measurement only after it has ended is another matter. Leave it until then and the evidence may be impossible to reconstruct.

Elements of the measurement budget

So firms should stop scrabbling for money each time a measurement question comes up. Every media plan has a media budget; every serious advertiser should have a measurement budget, too. 

The measurement budget discussion is not only about funding a few studies - an MMM and or a few experiments. The requirement for measurement is much more beyond the cost of the study.

Where is the data required for measurement going to come from? If you need weekly sales data by geos we need to budget for the data to be recorded and received for analysis. If awareness measures are critical for measurement of media effectiveness, a budget for setting up brand tracking needs to be provided.

Do you have adequate data infrastructure - data pipelines and data repositories that acquire, process and house the data to be ready for measurement. 

The cost of modelling is an obvious item in the measurement budget. But the cost for ingestion of the measurement learning into the companies operating systems has to be considered. It is these tools that actualize the measurement insights for future planning. Without this the best of the insights stay in PPTs and never engage with the real world. 

And most importantly - do we have the budget for a measurement custodian? Anyone whose core KRA is not measurement will not be able to do justice to the function.

The multiple dimensions of the budgeting decision are worth deliberation during the budget season.

How much? 

The good thing is that the cost of measurement does not linearly increase with the size of the brand. Account for brands that constitute 70% to 80% of your revenue. Plan for learning at a national level and for the key markets. Investigate the channels that consume a significant share of the media investment. Map the key decisions that need inputs from measurement. 

I am deliberately not prescribing a percentage. The right figure will vary from firm to firm. What matters is that the budget exists before the year begins, alongside a list of the big decisions that need evidence. Start with those decisions, then pick the method: experiments, MMMs, brand studies or something much simpler.

At the next budget round, give measurement its own session. Decide what needs to be learned, what evidence that will require and what must be in place before the money goes out the door. 

Companies happily spend hours arguing over where each marketing dollar should go. They can spare a little time to work out how they will know what it bought. 

Beliefs can evaporate; Evidence stays and transforms into organization knowledge. Move from belief-based marketing to evidence-based decisions.

Premjeet Sodhi is Global Measurement & Analytics Lead at WPP Media, based in New York, where he builds measurement systems connecting media to business outcomes. The article reflects his personal point of view on the subject.