It’s been a while since I tried to explain a part of the Nielsen Audio system that you may not know much about. So let’s look the reporting of one of the basic parts of a survey: the response rate.
Why is the response rate important? Consider the concept of “non-response bias.”
Very simply, do the people that respond have similar behaviors — in this case, radio listening habits — to those that don’t respond? If so, wonderful! If not, well, that can be a problem.
For Nielsen Audio, because we slice and dice the estimates so finely, non-response bias can take on greater significance. Let’s say you’re working for an urban-formatted station. Like any other commercial radio station, you want the largest audience possible, but your focus will be on some specific age cell within the black population of your market. If your metro is 20 percent black, you’ve probably chosen to ignore the 80% of the market that isn’t black. The remaining 20 percent will be sliced up because you might be after 18-34 or 25-54 or more interested in what men or women within the target cell are doing.
The sample for your target demo will be small. Even with Nielsen’s best efforts, let’s say the company has recruited households that listen to less radio than the average would be if we could measure everyone in the market at the same time.
You’ve made sure your station sounds great, the music is right, the talent is connected, the clocks are perfect, etc., but the results will likely be less than you need to stay competitive. On the other hand, if Nielsen happened to recruit a black sample that spends a lot of time with radio, well, don’t spend your bonus too quickly.
In the diary service, Nielsen may end up recruiting households that use a lot of radio or households that use very little, if any, because they don’t ask up front (and shouldn’t). Unlike in the past when interviewers called random households to ask for participation, the service has evolved to use what’s known as an “address-based sample”. The old system was based on landline telephone numbers which worked very well last century when nearly everyone had a landline.
As we moved to a large majority of homes having only cell phones, the frame evolved to include residential addresses. The reality is that this is more inclusive with essentially everyone in the frame except those living in group quarters (dorms, prisons, military bases, etc.) and the homeless.
It used to be that response rate was a key element of how the services were evaluated, but response rates have dropped to such a low level that the only reason to harp on them today is to make sure that Nielsen doesn’t focus on cutting costs at the expense of quality in the surveys. With private equity owning the company now, you can bet the top people are looking for any way they can to bring down costs while maintaining or increasing revenue.
Response rates in just about all private surveys are poor. It’s just a fact of life. My recent column about the Pew Center’s public broadcasting support questions noted a net response rate of three percent. And Pew is rightfully considered one of the best survey operations out there.
Nielsen uses a somewhat convoluted response rate, but not because the company is trying to be sneaky. Let’s examine the diary service. The sample is a two-step process. First, questionnaires are sent to randomly selected residential addresses.
If Nielsen’s vendor can add other information (names, household size, ages, race/ethnic) to the address, they will. Nielsen mails the questionnaires using a set procedure and if you check your E-book, you’ll find a twelve month questionnaire response rate (click on “Market Info”, then “Population Estimates and Diary Placement” and then “Diary Placement/Return” and go to the bottom of the page). The average response rate for the questionnaire will probably range from the teens to the low 30s).
From the returned usable questionnaires, Nielsen pulls a sample to receive diaries as the company now knows the composition of the household based on the questionnaire responses. They can mail the proper number of diaries with the correct monetary incentives.
If you look at the same page in the E-book that I pointed you to in the previous paragraph, you’ll see something called a “Metro Unified Response Rate” which will range from the single digits to probably as high as 20 percent. That’s a response rate based on how many usable diaries came back (intab) divided by how many were sent out.
The real diary response rate is the response to the questionnaire times the “Metro Unified Response Rate” which will be abysmally low. You can complain about it to Nielsen but gaining tenths of a point will not change matters. It’s just how surveys work today.
How about PPM and the Sample Performance Indicator (SPI)? That’s another column entirely and another set of explanations.
Let’s meet again next week.
Barrett Media produces daily content on the music, news, and sports media industries. To stay updated, sign up for our newsletters and get the latest information delivered straight to your inbox.
