When you get upset about something, you might say “There ought to be a law against it.” Sometimes that makes sense, but on occasion, the law can’t foresee what will happen in the future. We have plenty of ridiculous laws still on the books. Here in Kentucky, it is required by law that you take a shower at least once a year. Also, you cannot throw flowers at a public speaker. While I’ve not registered my actions with the state, I’m in compliance with both laws.
This column will cover one part of the Nielsen diary service response rates and to do so, you have to know about a Federal law, specifically, the Telephone Consumer Protection Act of 1991 (47 USC 227 for those of you who like to look these things up).
The most important aspect of the law was to prohibit the use of “automatic telephone dialing system or an artificial or prerecorded voice to any telephone number assigned to…cellular telephone service”. It affects how Nielsen Audio and for that matter, most any survey research company that uses the phone or used to use it, operates today.
Let’s go back to 1991. If you were around at the time, did you have a cell phone? If so, it was likely either a bag phone or if you were really cool, you had a Motorola DynaTAC, better known as a brick phone.
Calls were expensive and if you were roaming, exorbitant. While it’s tough to find data from the time, in 1996, Southwestern Bell Mobile announced that customers in the Dallas-Fort Worth area could now roam throughout Texas and pay only 50 cents per minute with a statewide “reduced” roaming rate. Prior to this change, a five-minute roaming call would cost “almost $8”. $8 in 1996 would be $15.54 today. If you lived in Dallas, would you expect to pay over $15 to make a five-minute call when visiting Houston today? I don’t think so.
At that time, many telemarketing operations (and a few survey ones as well) were not that sophisticated. Some started with one phone number and simply added one before making the next call. This was parodied on The Simpsons in late 1996 (“Lisa’s Date with Destiny”) which had a subplot of Homer acquiring an autodialer. Like most of Homer’s ideas, it didn’t work out well.
Combine an autodialer with expensive cell phone rates and guess what happened? Congress got into the act. The TCPA was passed and unfortunately, legitimate survey research was not exempted. In the early ‘90s, this wasn’t a big deal as just about every American household had a landline. As of 2022, only 27% of American households still had a landline. Under two percent had a landline with no cell phone service.
This preamble explains how Nielsen has to build the diary sample. The change started in 2009 when the cell phone sample was first included. The sampling process evolved to what is known as the “address-based sample” or ABS. The ABS system involves sending a questionnaire to randomly selected survey households in advance to get information.
While I can’t tell you what the questionnaire looks like today, when it started, there were a couple of questions about radio and TV usage along with household composition questions. If you offered up your cell number, the fine print said you agreed to let Arbitron and later Nielsen call you on your cell. Eventually, Nielsen figured out that if you know how many people 12 and older live in the household as well as their ages, genders, race/ethnicity, and location, why bother making expensive calls? Just send the right number of diaries with the proper incentives as well as other mailings. While the address-based system is more expensive than the old random calling method, eliminating calls cuts costs.
This all plays into the diary service response rates which have very complex calculations. Once again, you’ll have to go to your E-Book, which, if you’ve been reading my columns, you’ve discovered that there is a lot of worthwhile information in there. If you look go to the “Diary Placement/Return” section of the E-Book, you’ll see how your market’s sample was derived. There is “listed sample”, “unlisted mailable sample”, “unlisted unmailable sample”, and “address-based sample”. Since there are so few landline households, a large majority of the sample is now address-based.
You can calculate the response rates by sample type if you wish. Take the number of usable diaries and divide that into the “estimated persons in usable households”. Nielsen does report the overall response rate which is typically very low. Also, you’ll find one more interesting statistic: the percentage of diaries returned from the “ABS Landline Supplement”.
Check the footnote and you’ll see that these diaries are “preweighted” because these households have a better chance of being selected to be in the sample as they are in both the ABS sample and the landline sample. If one of those diaries spends a lot of time with one of your stations, you won’t be happy. That diary is worth less than the others.
Response rates matter, but low response rates are not an indictment of Nielsen. Even the US Census Bureau has trouble with response rates. People and companies are interested in the results of surveys, but getting people to do them is very tough. The fact that Nielsen Audio can even put out diary surveys in so many markets on a timely basis in 2024 is probably something of a miracle.
Still, the rates matter because if the non-respondents are very different than the respondents, you have a bias problem. Weighting makes up for some of this, but not if listening habits of the non-responders vary. However, on the plus side, who is more likely to be a non-respondent? If you don’t listen to radio and you’re asked to do a radio survey, even when Nielsen is sending you cold, hard cash on a regular basis, would you be less likely to participate? I can’t prove it empirically, but it makes sense.
In other words, maybe we shouldn’t complain about low response rates.
Let’s meet again next week.