Making "Bad" 6% Less So

Not to seem ungrateful that we're now just a few more months away from seeing the 6% increase in PPM samples which Nielsen promised for 2014 last November, but - as my post "The Canary Has A Name" suggests - that won't come even close to realizing the dream of true PPM-based cross media measurement with reliable estimates a reality.

As Paragon Research's Larry Johnson recalls "when Nielsen and Arbitron were first collaborating in the early 1990s about rolling out radio PPM jointly, the sample size was to be roughly three times what Arbitron has been offering.  The sample size becomes even more crucial if Nielsen is going to measure multiple platforms with a PPM device.  If you’re going to accurately measure outlets with small, niche audiences, the sample size must increase dramatically.  For example, cable television gets cut up into very small pieces given the huge number of cable channels available.  It’s interesting that Nielsen uses a mixed methodology utilizing both diaries and meters in many of the television markets they rate."

Researcher Mark Ramsey offers a brief statistics course in a two year old article to underscore why it's important that Nielsen make the most of their abilities now that that they own PPM: 

Small samples yield extreme results more than large samples do. Every broadcaster knows this, of course, but the impact is far worse than you think.

Daniel Kahneman discusses this issue in some depth in his book Thinking, Fast and Slow:

Imagine a large urn filled with marbles. Half the marbles are red, half are white. Jack draws 4 marbles on each trial, Jill draws 7. They both record each time they observe a homogeneous sample—all white or all red. If they go on long enough, Jack will observe such extreme outcomes more often than Jill—by a factor of 8.

By a factor of 8!  Extreme (and erroneous) results are 8 times more likely with 4 rather than 7. And how many meters are tuned to your station right now?

Ramsey asks:   "What does it say about the veracity of our message and our messengers when we’re using numbers we know to be flat-out false?"

Another researcher, Kurt Hanson recalls his attempts several decades ago to sell radio on a better sample size in a service called Accu-Ratings, and puts it this way: "introduction of Portable People Meters (PPMs) solved the wrong problem:  Diaries weren’t being filled out precisely to the minute (in fact, lots of respondents filled out their diaries in one sitting at the end of the week), and PPMs “fixed” this problem, but because meters were so much more expensive than paper diaries (hundreds of dollars vs. maybe a dime), Arbitron used far fewer of them per market.  So the sample size issue got *worse,* not better!"

If today's multi-media ratings firms are serious about credibly measuring the increasingly-fragmented pie, they have to see the potential of signing many, many new clients which hopefully will rapidly grow their revenues, while keeping radio and television's rates stable at current levels.

I hope they also understand that to achieve that lofty goal sample sizes must be at least tripled or quadrupled, not merely increased in small increments.

The Canary Has A Name

The canary in radio ratings' coal mine's name is Sean Hannity.

In a cover story interview this week, the syndicated talker tells Radio Ink "one of the problems with radio in general, and this impacts all formats, is that in the last 10 years, our lives, in terms of technology, have changed dramatically. Radio needs to wake up to this fact. It’s the single biggest threat to the radio business today — that is, the measurement system is flawed in a dramatic way."

Hannity believes he has millions of digital listeners he is not able to get credit for.  "For example, in New York, if you’re listening to The Sean Hannity Radio Show on the WOR website, even if you have a Portable People Meter, I can’t get credit for that. We know that WABC, at times, had well over 1.4 million people listening online. If we can’t get credit for that, then all the data that’s being put out there is just inaccurate. Then you add to that those people who listen to me on satellite. That’s unmeasured. If people listen on my website, that’s not measured. I’m a big proponent of iHeart-Radio, but that’s also unmeasured. What we have is a situation where, especially in big cities, it’s almost impossible to get an AM signal, and for the people listening to my show, and they are, there is no way that we can get credit."  Hannity says. he knows for a fact he has millions of online listeners. "I have never seen it below a couple of million people. Ever. This is not a small issue. This is a massive issue."

Sean was an early believer and user of PPM and spoke in 2007 to the annual consultants' fly-in about how he was using the Media Monitors and custom audience flow reports tracking of his show to hone content, so I can testify that he's an extremelty knowledgeable voice on the subject who has been digging deep into PPM data from the beginning.

Also, the responses to his views on Radio Ink's website are generally chirping in agreement:

The current reporting method stations are using for on-line listening is difficult to work with or believe at best. This problem requires a good solution from Nielsen...now.
- Dick Kalt

I say again -- add PPM encoder to your internet streams. This is not shocking revelation.  Parsing streaming server logs is also a trivial matter, albeit you don't get the same demographic data as PPM.
- Joe Cassara

This is an "Elephant in the room" issue for radio. What exactly is being done about it? Nothing yet.
- Iconoclast

For the first time in my life, Hannity and I agree on something. To the earlier comment, server logs are flawed too and subject to manipulation. - Fred Lundgren

I have written about this shortcoming, citing what Canada's BBM is doing for genuine cross-media measurement.  When ARB was radio and Nielsen was TV, using server logs was the only option open to Arbitron.

Now that Nielsen measures both TV and radio with PPM, they can do much better than 6%, as Sean Hannity clearly understands.

Your Audience Is Highest During Commercials? That's A Good Thing?

El Rushbo is famous for spin, of course, but I am scratching my head over this press release, out this morning:

"In a new study released today, Coleman Insights has found that Rush Limbaugh‘s nationally syndicated radio show generates higher listener shares during commercials than during the actual show. This finding shows that the Rush Limbaugh audience has extremely high anticipation for the show and tunes in through commercials—a testament to his extremely strong brand and the loyalty of his audience."

Hey, Rush: when your commercials beat your show, advertisers have got to feel reinforced by this fact, but I'd say you need to work on your content or watch your brand strength deteriorate! (I'd bet Sean Hannity's show does better than his commercials do..)