Tuesday, March 04, 2008

Why An Index Of 80 Or 90 Isn't Good Enough

While it's good to hear that ARB's Pierre Bouvard is back from his vacation fully-refreshed and enthusiastic for the radio ratings giant's sample improvement goals for the coming year (according to Inside Radio): “2008 is the year of 25-34s.”

After suffering through last Summer’s sagging PPM in-tab rates, IR's Frank Saxe reports he got a phone call from Bouvard yesterday, touring that Arbitron is increasingly confident it has solved the problem of how to hit 18-34 targets. Its performance among adults has been steadily improving, taking its biggest steps forward in recent months due to a focus on the 18-24 demo. It’s an effort that’s included higher sampling rates, richer incentives and more interaction between the company and its panelists.

Arbitron president of sales and marketing Pierre Bouvard says “We’re not done, but we have made a lot of progress in four months.” An upgrade to Arbitron’s software allows its research team to begin honing-in on the next target. Bouvard says “We said we were going to go on a jihad on 18-24s and we did. Now it’s onward an upwards -— 2008 is going to be the year of the 25-34s.”

Since last month they’ve done that by applying many of the same strategies and techniques used on 18-24s. Arbitron’s four-month focus on 18-24s has led to a 13% improvement in that Philadelphia demo — and it’s up 113% over one year ago. Bouvard says “It’s nothing short of spectacular. I’m feeling very good about how the panels are shaping up.”

Here's the problem: as you look at day-by-day, hour by hour and even minute by minute data in PPM, the sample is often four times as big as the daypart, day and even diary page sample. That's because so many diarykeepers 'forget' to write down at least half of their actual listening. The laws of statistical replication state that in order to double the accuracy of a probability sample, you must quadruple the sample size, so in one sense the minute by minute cume a daypart in PPM could potentially be more accurate even than weekly daypart data in the diary, which may be why it looks so reasonable and often passes "the gut test" (does it look believable?). ARB's PPM advocate John Snyder has been making this claim lately. He's a very smart guy, who makes very convincing Power Point slide shows.

However, the total panel sample is actually smaller than the random diary sample (to save money, of course) and a panel sample isn't exactly random either. Just because something looks 'right' most of the time doesn't mean that scientifically, statistically it is so.

I am hoping that ARB decides to start guaranteeing 100% of their targets for both PPM and diary markets in this year of the 25-34. No less than 100% accuracy is acceptable at a time when radio needs to convince advertisers that we are worth 100% of the money they spend on us, not 80 ot 90%.

1 comment:

Steve Casey said...

In the article is this line:

<< That's because so many diarykeepers 'forget' to write down at least half of their actual listening. >>

What is the source of that?

Neither my own work with diary analysis or the studies done by Ball State would suggest that so much listening is lost.

Quarter-hours certainly aren’t higher in PPM, just cume.

And if that is what we’re discussing, the phrase might better be “..so many diarykeepers ‘don’t notice’ many of the audio sources they are exposed to.”

Ya think?

I’m just trying hard to keep all the pros and cons of PPM straight.

Mostly, I wouldn’t care much about the ratings, because the point is to sell advertising.

But I found the diary survey could give us a lot of nice behavioral information to help with programming – particularly newer PDs who haven’t quite learned all the lessons about listening location (like you reiterated in your comments about the loss of at-work listening at Christmas), and how TSL is built mostly by daypart recycling and habitual listening – not by keeping listeners sitting in the parking lot at work for 10 more minutes.

I think a lot of the data being sourced as the basis of the new “insights” gained by Clear Channel and others are misleading, and are leading to decisions that actually damage the programming. Like two commercial breaks per hour. So now I do care. And I’m really concerned. You too?

Very best regards,

Steve