In All Eight PPM Major Markets: Country Is UP From The Final Diary Trends

(Country format average increase from the final two diary book average compared to the first two PPM month averages)

18-34: +28.4%
18-49: +29.9%
25-54: +29.1%

- Musicrunch's Anthony Acampora and Kevin McCabe at the 2008 ARB Consultants' Fly-In

I thought we had licked the country bias against writing the format down in diaries many years ago, but it seems that in the major markets, at least, that may not be the case.

Why An Index Of 80 Or 90 Isn't Good Enough

While it's good to hear that ARB's Pierre Bouvard is back from his vacation fully-refreshed and enthusiastic for the radio ratings giant's sample improvement goals for the coming year (according to Inside Radio): “2008 is the year of 25-34s.”

After suffering through last Summer’s sagging PPM in-tab rates, IR's Frank Saxe reports he got a phone call from Bouvard yesterday, touring that Arbitron is increasingly confident it has solved the problem of how to hit 18-34 targets. Its performance among adults has been steadily improving, taking its biggest steps forward in recent months due to a focus on the 18-24 demo. It’s an effort that’s included higher sampling rates, richer incentives and more interaction between the company and its panelists.

Arbitron president of sales and marketing Pierre Bouvard says “We’re not done, but we have made a lot of progress in four months.” An upgrade to Arbitron’s software allows its research team to begin honing-in on the next target. Bouvard says “We said we were going to go on a jihad on 18-24s and we did. Now it’s onward an upwards -— 2008 is going to be the year of the 25-34s.”

Since last month they’ve done that by applying many of the same strategies and techniques used on 18-24s. Arbitron’s four-month focus on 18-24s has led to a 13% improvement in that Philadelphia demo — and it’s up 113% over one year ago. Bouvard says “It’s nothing short of spectacular. I’m feeling very good about how the panels are shaping up.”

Here's the problem: as you look at day-by-day, hour by hour and even minute by minute data in PPM, the sample is often four times as big as the daypart, day and even diary page sample. That's because so many diarykeepers 'forget' to write down at least half of their actual listening. The laws of statistical replication state that in order to double the accuracy of a probability sample, you must quadruple the sample size, so in one sense the minute by minute cume a daypart in PPM could potentially be more accurate even than weekly daypart data in the diary, which may be why it looks so reasonable and often passes "the gut test" (does it look believable?). ARB's PPM advocate John Snyder has been making this claim lately. He's a very smart guy, who makes very convincing Power Point slide shows.

However, the total panel sample is actually smaller than the random diary sample (to save money, of course) and a panel sample isn't exactly random either. Just because something looks 'right' most of the time doesn't mean that scientifically, statistically it is so.

I am hoping that ARB decides to start guaranteeing 100% of their targets for both PPM and diary markets in this year of the 25-34. No less than 100% accuracy is acceptable at a time when radio needs to convince advertisers that we are worth 100% of the money they spend on us, not 80 ot 90%.

Kurt Hanson's Blog Is Getting Interesting

If, like me, you drifted away from Hanson's newsletter because it became too much of a cheerleader for internet radio issues and became less relevant to 'our' concerns and problems, it's time to wake up and pay attention again, at least to his blog:

Suppose we used to survey 10 Chicagoans every week to find out what percentage are Chicago Blackhawks fans. That means over the course of a calendar quarter, we’ll talk to 130 people. That’s probably a decent sample size to answer a question like this. (I could do the math on standard deviation and margin of error, but let’s save that for another day.) The analogy for Arbitron’s new approach is to survey maybe 30 Chicagoans — but survey them every week. “Hey, that’s 390 data points every quarter! Better than the old system’s 130 data points!” No, it isn’t. In week #1, 30 people is too small a sample size for the question — we could get a result that’s way off of reality. And surveying the same people every week for 12 more weeks is going to give us, basically, the same wrong answer every week. Yes, the results will be more stable than under the old, 130-person system, but that doesn’t mean it’s a better approach... ...Because Arbitron was too cheap to throw around a few $2,000 bills toward consulting contracts, their stock price took a $200 million hit yesterday — and they’ve put a not-insignificant portion of radio’s annual $20 billion in ad revenues is now at risk, too.

ARB's PPM New York Fiasco Didn't Have To Happen

Jim Carnegie got the quote from Cox Radio CEO Bob Neil no other publication had today and Bob continues to say in public precisely what our friends at Arbitron need to hear:

"I think the next steps are pretty clear. Their customers, broadcasters and agencies, see little or no value in 6-11 year olds in the sample. They were there for TV purposes. Eliminating them allows the sample size to grow 12 plus, which should help at least get bigger numbers in the challenged age cells. The sampling issues must be dealt with, and I think an 85 index against target is a good floor. That's a "B" in High School and we're paying for an "A". If we don't get there, there should be a rebate to broadcasters and agencies that is significant enough to hurt, so it forces Arbitron to keep an unrelenting focus on the sample. Before another market rolls out, Philly and any other proposed PPM markets must get MRC accredited. Because of this snafu, customers are going to demand to know they are getting a quality product, and the MRC gives us confidence that the data is credible. No more waiting on accreditation. I don't think any of this is unreasonable, nor should it cost customers any more than the huge increases we're paying. It's going to be up to Arbitron to set a new direction here by telling their shareholders and Board that this is going to go slower, and frankly, that their profits on PPM won't be what they thought they were going to be. If they shut down their spin machine, and fix these problems, the industry will move into the Electronic Measurement age smoothly and with confidence. If not, Arbitron may be the Hooper and Pulse of the new millennium."

Back in 1999 when industry relations were less contentious between ARB and its customers I remember the company's Research VP Dr. Ed Cohen say that a good guide for proportionality indexes in all target cells is "... like the FM dial, between 88 and 108."

Even 85 isn't 88, but compared to where we have been in the past year, I'd take it.

I See Paragon's Bob Harper And RAISE Him One

Harper's blog post today isn't about birds, though it starts with that old DJ wild track calling for another canary. It's about a very wise broadcaster advocate named Bob, who is anything but a patsy, in spite of its title: "Patsies Pay More."

I am 100% certain if Paragon did a study for a client and, on presentation day, we told him: “I know we said there would be 600 respondents, but we’re pretty sure 507 will be okay. And, some of these age breaks only have one respondent, but we’re pretty comfortable with that, too,” we would never work in broadcasting again. But, yet, the PPM train persistently chugs along. The squeeze, now that PPM has left the depot: 1) Arbitron will want lots more money from Radio to quickly make their shaky product acceptable. 2) Agencies will be demanding accountability, posting, and guarantees. -- Bob Harper

Bob, I see you. In fact. I HEAR you and here's my ante: Is there any other broadcast research company which would have the temerity to actually contract for a project and then after delivering the results ask for either more money or propose changes in the methodology to cut their costs of doing the job right? Especially, after this company had developed and perfected that methodology for more than a decade?

How Could ANYONE Be Jealous Of The Lovable Guys At Paragon Research???

... but anyone with a blog gets green with envy when you see someone else come up with a great reason to pull blog readers back again and again.

Their PPM Rules are a great example, and they just added three more you must read in full, including the responses from other readers:

#11: View the data with skepticism
#12: Promotions should be hyper-focused to the working person and working communities
#13: Don’t be shy about targeting men

How Much Is 65+ Sample Worth To You? Anything?

ARB's timing, given the news that started coming out last Wednesday (Arbitron Blames Software In PPM Houston Snag, Arbitron Says PPM Data Are Reliable) was, depending on your perspective either very "fortunate" (lots of folks were right there to question it) .. or extremely UNfortunate (embarassing, to say the very least, and perhaps more importantly very frightening to the stations which have been sold PPM as their ratings currency for this fall!).

For example, a marketing email from Solutions Broadcast Media Group Friday couldn't resist the temptation to take a swipe at both PPM and diary audience measurement technologies ("...another reason why we (SBMG) only conduct our audits via phone recall methodology: Radio ratings tank with Arbitron PPM. Down 30 percent to 50 percent for top-tier stations Details...").

Not long after, Keymarket Communications VP Frank Bell sent us this PS to his notes on the Fly-in, which he was kind enough to share with A&O:

"One thing to watch for as I review my notes again: ARB keeps promoting "stable listening levels" with PPM regardless of the number of in-tab. They are citing listening for the entire market. But that "fact" is irrelevant to anyone operating a specific radio station. I'm sure the low in-tab can dramatically affect estimates of a single station (or format), which is, of course, what gives us all heartburn."

Join the club.

Speaking of heartburn, Mike O'Malley and I were recently commiserating over the terrible sample wobbles in many of our client markets after the Spring books that have become the rule rather than the exception, complaining how it seems Arbitron can't seen to consistently find country core listeners. He told me about a market he had just reviewed where fewer than 40% of the in-tab sample were non-ethnic 25 to 54 year olds.

That sent me back to Maximiser and my trusty Excel spreadsheets. I happened to have data for 12 of my client stations on my laptop.

After finding that one of my client markets had only 30.1% of its total Spring 2007 market sample from non-ethnic 25-54 year olds, I just HAD to check them all.

Only one of those 12 ARB subscribing stations I worked with this past spring had more than 46% of its entire market diaries from non-ethnic listeners between 25 and 54.

Why complain, since - thankfully - all of the A&O client stations in these cities had pretty good to even great Spring books?

Here's why: when you study the sample trends over the last two years, it becomes clear that two factors are combining to create huge problems from Arbitron....

25-54 Non-Ethnic Diaries - (% of All 12+ Diaries, Spring 2007 Survey)
Amarillo - 34.6%
Anchorage - 56.6%
Charlotte - 35.6%
Cincinnati - 41.2%
Eau Claire - 45.5%
Portland - 44.2%
San Diego - 30.1%
Santa Rosa - 35.4%
Seattle - 43.4%
Spokane - 45.4%
Topeka - 42.8%
Victor Valley - 30.5%

Shocking? Sure.

Surprising? Not really. Those two trends I referred to:

#1: More and more households are moving to "cell-phone only." This is especially true among 18 to 34 year olds, creating levels of weighting and proportionality which are nothing short of totally unacceptable in market after market.

#2: A larger and larger portion of the diarykeepers which ARB gets great response from are 65+, the very demo which is projected to grow in size over the next 15 years as leading edge boomers hit retirement age.

This makes the most reliable non-ethnic data in the Arbitron report, based on consistent sample size, the 55+ and 65+ shares. But, who cares?

It would be nice if we could convince our media buying clients that these people are worth reaching, but don't bet the future on it.

From now on, A&O will be tracking the percentage of all of our U.S. and Canadian markets' sample which is non-ethnic and between the ages of 25 and 54.

Get ready for your hair to stand on end as you see the direction that trend line is rapidly moving.

A (only semi-serious) question: how about proposing that Arbitron simply discard all diaries from 65+ year olds, stop wasting any time processing that unimportant data to the majority of our clients and count only 12 to 64 year olds as they benchmark in tab targets?

Do you have a better idea? I'd love to hear it.

New Arbitron Goof: Rats Ate The PPM Data

If you're going to today's Arbitron Consultants' Fly-In in Columbia, MD., don't get bitten by a rat (and be sure also to wear your BS-detector!)...

Here's a great recap of the current problems (and you sure can't disagree with their final sentence!):

Said one broadcaster (email obtained by ftm): “Which is worse, a 30% drop in sample ... or an error of this scope that no one at Arbitron caught?” All of this is more than vaguely, but materially similar to 2004 episode in the life of RadioControl – another electronic measurement system – in Switzerland. Changing sample provider in the midst of one survey period caused numerous changes in the results, including the ‘disappearance’ of several stations. Neither Radiocontrol nor PublicaData, its marketing service, informed clients of the change in advance and waited until one station owner – also a prominent Swiss political figure – threw what could only be described as a fit. The result, three years later, was a complete reorganization of the radio audience measurement system in Switzerland. Sampling, apparently, is even more volatile with electronic measurement systems. Abitron said yesterday – and last week, last month and last year – that sampling problems with PPM surveys would be corrected. Media measurement services operate like public utilities, mediating – like accounting – a relationship between broadcasters and media buyers. Confidence, on both sides, is built on transparency. Where transparency is weak, quality control weaker and the science reminiscent of Enron derivatives broadcasters and advertisers need to re-think whether or not they are getting what they’re paying for.

Is There Any Way To Measure "Engagement?"

Mark Ramsey's Hear 2.0 blog has an important thread going right now on the pro's and con's of PPM and radio audience measurement (The difference between "Listening" and "Hearing" ):

"...for many listeners radio isn't about high-testing songs and engaging personalities; radio is about comfort. Indeed radio is as much a part of the ambient soundscape as the whistle of the wind, the chirping of the birds, and the songs of the crickets. Something to be heard rather than listened to."

Arbitron Advisory Council Reads And Weeps

(click on the images to enlarge them)

Arbitron, how can this possibly BE? I thought, based on what we had been told by your top level executives that in exchange for that 60% rate increase you'd be tracking compliance on a daily basis and replacing panelists immediately if their use of the PPM started to fall.

PPM markets, get ready for sample buffering just like we currently see in diary samples. It looks to me like they need to start replacing fatigued respondents in week 4, week 8 and weeks 11 and 12 just like they often do in diary markets now.

Given that possibility, you have to ask was this really worth betting the company on? I know it's the first inning of the game and we're all just learning about the process. But, hasn't Arbitron been testing and sampling PPM for a decade or more?

Looking at these charts, I wonder if it would have been wiser to just invest the rate hikes we're paying for PPM in larger diary samples. Weighting, creating unacceptable and unbelievable wobbles in many western markets, for example, this spring has been at unprecedented levels. PPM was supposed, we had been led to believe, to improve compliance and sample stability, reducing the need for sample weighting.

After all these years of development, this info - now that three markets will be dependent on the new technology by the end of 2007 - is more than a little surprising and disappointing to see.

What's ARB Doing To The Philadelphia PPM Panel?

It looks like they are adding African-Americans. And, that doesn't appear to be good for country. Cume increases and TSL drops continue, and ubiquity - as B-100, KYW and WMMR's consistency and growth show (click for the Radio Ink report).

The fate of more niche-driven and geo-demographic lifestyle formats is less certain at this point. A year ago, I loved watching (and believed it was reality since I think country faces a real challnge in diary placement as ethnic and younger demo formats get increased premiums and special treatment) country WXTU's cume growth increase its shares and keep them there month after month.

Now, as the pressure from ethnic broadcasters to add more of their listeners to the panel increases it appears that country shares are dropping (Urban WDAS is now #4 in 6+ cume (from 802,100 last month to 868,900 now) and has been going up in rank in each of the last three months and country WXTU has been falling in rank, while continuing to hold a much larger cume than before). WXTU cume dropped this month to 857,900 from the previous month's 866,300, which is still higher than it ever was in diaries.

Hopefully, the weighting factors that appeared to give country a fairer shake aren't being rejiggered, to our detriment.

TSL in the average diary last year was 20 hours a week. Last month's average TSL for the People Meter samples was twelve and a half hours, a slight uptick from the previous month's twelve and a quarter hours a week.

That makes Persons Using Radio appear to drop, about 28%, which means that cost per point increases by about the same amount. Our sellers are going to face some real challenges as this wave rolls over more markets.

One thing for sure: it isn't going to make much sense if radio has to pay more for PPM measurement which only ends up giving buyers a bigger, more state-of-the-art stick to beat us up with. We need to learn a whole new way to position our medium, now.

Arbitron needs to do a lot more public relations, research studies, education sessions, training to help, since they are the only company, thus far, that - as I see it - stands to gain financially as radio adopts the PPM.

First PPM Results Reveal Ethnic Variations

Erik Sass reports AFRICAN-AMERICANS LISTEN TO RADIO MORE frequently and for longer periods of time than other ethnic groups, according to data released last Friday by Arbitron.

That finding is part of the first "currency" radio ratings in the Philadelphia area, based on results from its Portable People Meter, a passive electronic measurement device. The ratings, covering March 8-April 4, revealed that 9.4% of African-Americans over the age of 6 were listening to the radio during an average quarter-hour measurement period, as compared with 9.1% of the overall radio market.

African-Americans also listened to the radio for a longer period of time, on average, than the population at large: They averaged 12 hours of listening a week, versus 11.5 hours for the total population. Interestingly, the Arbitron data shows that Hispanics listened to radio less than the population at large, on average, with 8.6% of Hispanics surveyed listening during an average quarter hour, and a total average listening time of 11 hours.

Could this hurt country? Not so far, it seems, since WXTU is also doing VERY well in the Philly PPM data (their cume almost doubles and XTU ranks #6 in 12+ share, trending 3.7-5.5)! Radio as a whole has greater reach (96% of persons 6+), but much lower Time Spent Listening -- including among African-Americans and Hispanics, according to the ARB PR), so let's suspend judgment for a bit until we've seen these 'facts' replicated consistently over the next two years, since a change in the panel sample could radically change the results and the average panelist lasts about six months.

I Am Not Surprised

Inside Radio's bulletin: A second bombshell at the RAB - The Media Audit/Ipsos get millions from broadcasters for a full-market test in Houston.

I think this is all about $$. Arbitron had to know this was coming, even as they got MRC Accreditation for PPM last week in at least the market PPM is online right now, given their insistence on a 65% rate increase for the Portable People Meter. I still don't understand why they've haven't been pricing for market share while they had the exclusive.

Now, it gets very interesting. It would be great to see, as good as ARB and its people are, some real competition in the radio ratings business! Unfortunately, it means at least one more year of uncertainty, but (fortunately) we've all become very good (of necessity!!) at adaptation, flexibility and change in the business climate of this last decade...