Wednesday, April 15, 2009

"Small Sample Sizes .. But, Also Like Crack Cocaine"



Angus Reid Strategies' Jeff Vidler penned a terrific article (click to read it, pdf) "PPM Info: Too Much Of A Good Thing?" for the April 2009 print edition of Broadcast Dialogue magazine.
"Functionally, small sample sizes limit the accuracy of a lot of the minute-by-minute PPM data. This is particularly true when it comes to individual events."

It puts the issue of sample sizes in a slightly different frame, since PPM panelists have their meters moving for as long as 15 hours a day, seven days a week and the same people, more or less, comprise the sample for the entire survey periods for several years, rather than turning over every week as in diary measurement. Thus, we can drill down to one minute (or perhaps in the future, even shorter!) periods of listening.
"PPM gives you lots of tactical data but very little strategic information."

Yes, we know that the sample is tiny, but it's still fascinating to track. There's a lot of info there, following moment by moment what certainly looks like real radio usage by real folks, by and large.
"Most other products are consumed in the bright white light of day. You can see the clothes people wear and the cars they drive ... but radio is different. There's no price, no trackable transaction. And, being the one-to-one medium that it is, most people listen to radio by themselves."

Now that PPM has changed that. As Vidler writes, "if you've had the chance to see some of this (PPM) data, then you know it's more than a little intoxicating .. one taste isn't enough."

Is it that we only complain about samples when the results aren't to our liking? I don't think so, but these new-found abilities to drill down microscopically so much more than in the past will increase the pressure on ratings providers - radio's referees - to get and keep sample panels as representative and proportionally correct as possible.

Our futures (strategies and tactics) depend on it.

1 comment:

Chris Kennedy said...

Amen to that...while PPM monthly releases on the whole are fine to read, if you want to get to targeted demo brass tacks, it's the trending that matters (up, steady or down) combining several month akin to rolling averages. This is especially true for the new markets (like Montreal) with no historical reference of a year ago.

When you want to look at a special broadcast event minute-by-minute targeted in-demo, samples are light, driven by only a few dozen total devices split amongst the marketplace tuning.

Yes, it does give an indicator in that case...but for tactical and sales info only, not giving solid strategic info...unless it is a repeating and regular "special event". Since this is the currency, strategic decisions will incorrectly be made based on PPM sampling (being "representative" doesn't necessarily equal "accurate")..the best available info for many stations not investing in custom perceptual research.