Why An Index Of 80 Or 90 Isn't Good Enough

While it's good to hear that ARB's Pierre Bouvard is back from his vacation fully-refreshed and enthusiastic for the radio ratings giant's sample improvement goals for the coming year (according to Inside Radio): “2008 is the year of 25-34s.”

After suffering through last Summer’s sagging PPM in-tab rates, IR's Frank Saxe reports he got a phone call from Bouvard yesterday, touring that Arbitron is increasingly confident it has solved the problem of how to hit 18-34 targets. Its performance among adults has been steadily improving, taking its biggest steps forward in recent months due to a focus on the 18-24 demo. It’s an effort that’s included higher sampling rates, richer incentives and more interaction between the company and its panelists.

Arbitron president of sales and marketing Pierre Bouvard says “We’re not done, but we have made a lot of progress in four months.” An upgrade to Arbitron’s software allows its research team to begin honing-in on the next target. Bouvard says “We said we were going to go on a jihad on 18-24s and we did. Now it’s onward an upwards -— 2008 is going to be the year of the 25-34s.”

Since last month they’ve done that by applying many of the same strategies and techniques used on 18-24s. Arbitron’s four-month focus on 18-24s has led to a 13% improvement in that Philadelphia demo — and it’s up 113% over one year ago. Bouvard says “It’s nothing short of spectacular. I’m feeling very good about how the panels are shaping up.”

Here's the problem: as you look at day-by-day, hour by hour and even minute by minute data in PPM, the sample is often four times as big as the daypart, day and even diary page sample. That's because so many diarykeepers 'forget' to write down at least half of their actual listening. The laws of statistical replication state that in order to double the accuracy of a probability sample, you must quadruple the sample size, so in one sense the minute by minute cume a daypart in PPM could potentially be more accurate even than weekly daypart data in the diary, which may be why it looks so reasonable and often passes "the gut test" (does it look believable?). ARB's PPM advocate John Snyder has been making this claim lately. He's a very smart guy, who makes very convincing Power Point slide shows.

However, the total panel sample is actually smaller than the random diary sample (to save money, of course) and a panel sample isn't exactly random either. Just because something looks 'right' most of the time doesn't mean that scientifically, statistically it is so.

I am hoping that ARB decides to start guaranteeing 100% of their targets for both PPM and diary markets in this year of the 25-34. No less than 100% accuracy is acceptable at a time when radio needs to convince advertisers that we are worth 100% of the money they spend on us, not 80 ot 90%.