The Right Way To Sample

It's very good to see Arbitron begin cell-phone only household sampling in an initial 151 diary markets for the Spring 2009 radio survey which starts today.

However, in an Arbitrends market whose numbers came out just yesterday, where 7% of the population is African American, 15% of the in-tab in this month's proportionality report sent to stations by ARB is African American. Meanwhile, almost all of the market's non-ethnic stations which target 25-54 trended down.

How long do we need to complain about sampling before real dramatic change happens?

Read this, from longtime researcher Ted Bolton and take a guess when it was written:

"There is a very disturbing Arbitron bias that needs to be changed. The bias is a straight forward problem with Arbitron sampling methodologies that can be changed, and must be changed.

THE ARBITRON EQUAL OPPORTUNITIES VIOLATION

Without getting too technical, in order to understand the Arbitron problem, you first need to understand just the basics of sampling theory. Sampling theory is based upon the concept of equal opportunity and the equal probability of being selected for the sample. If the universe consisted of 1,000 people, then all 1,000 people would have EXACTLY the same chance of getting selected.

The problem begins with how Arbitron samples. Arbitron selects an individual from a random list and then sends diaries to all of the individuals within the family unit. That means if the respondent is married with four people in the household over the age of 12, that respondent would receive six diaries!

The respondent who is single and lives alone (or even lives with somebody else on a non-married basis) receives a total of one diary.

In this scenario, Arbitron has completely violated the concept of equal opportunity.

Family members are eating up the age quotas much to the dismay of the single/live alone respondents.

What if you were doing callout research and instead of sampling individual listeners, you decided to sample each member of the household you contacted. Would this make any sense whatsoever?

The answer is no, yet we live by an Arbitron system where this is the norm.

THE RIGHT WAY TO SAMPLE

Arbitron sampling should occur on an individual level, not on a family level. The only reason Arbitron samples on a family level is to reduce the costs of completing the survey.

Take it from a researcher who knows filling quotas on a family level is cheaper than filling quotas on an individual level. We all want our listeners to have as much of a chance of being selected or an Arbitron survey as does anyone else."

Believe it or not, the above paragraphs are from "Radio Trends," a newsletter published by Bolton in 1995.

It could have been, should have been, written yesterday. Which is why I reprint it here, and now.