I hope you had a great Memorial Day weekend! Last week, I gave a little history of the seven-day Nielsen radio paper diary which, if you believe the trades (and the trades are always right including this one!), could be replaced with an online version by late 2025.
This week, I’ll discuss the issues surrounding what would be a major methodological change in a system that not everyone loves, but for the most part, everyone involved knows well.
Will the results differ when Nielsen changes methodologies? The answer is almost always a resounding “yes”.
Even the smallest changes can affect the results and when results differ from expectations, someone will be upset or as I’ve told people for years, “We’ve never figured a way to create more than 100 share points in radio”.
On a share basis, if someone gains, someone else has to lose. And for now, let’s not think about what might happen to usage levels.
Let’s consider the potential reasons for differences in results between the paper diary and an online version. The list is not exhaustive, but you’ll get the general idea of the complexity:
- Will the sampling be the same as currently used? The Nielsen Audio sampling system has evolved over the years from using only landline telephone numbers to adding cell phones in 2009 (probably the first major survey of its kind to do that in the US) to using mail-out pre-surveys (the address-based sampling frame) ensuring the most everyone has a chance to be part of the sample.
- Will individuals that fill out paper diaries be the same ones who fill out the online diary?
- What will be the effects on response rate and proportionality?
- Will the listening levels (PUR meaning Persons Using Radio) be statistically the same, in other words, the results could be different, but not due to the method?
- What about the number of entries and the quality of those entries in the online diary compared to the paper diary? Will the online diary be easier or harder to edit when there are unclear entries?
- The paper diary is completely unaided, in other words, the “diarykeeper” fills in whatever they think they’re listening to with no help from Nielsen. An online diary could include prompts. Will such a version be tested?
- Will diarykeepers be more likely to fill out the diary in “real-time” versus taking care of the chore at the end of the week? We know that a majority fill out the current diary after the fact.
- For that matter, the “diary week” has been Thursday to Wednesday for nearly 60 years. Could that change and if so, what would be the effect of that change?
- How will incentives be delivered? Currently, money is included with the package that includes the diaries, but I don’t think Nielsen wants to try using Venmo or PayPal or some other method to send out incentives. If there is one thing that’s been learned over time about survey incentives, cash is still king. That means plenty of mailings just as is done currently.
- How many markets will be part of the test?
- What kind of pretesting will be done? Will various designs be tested with focus groups or “one on ones” ahead of a major test?
- What will be considered an “acceptable” online diary? It’s easy to leave a question blank in the paper diary, for example, race and ethnicity, but the online diary could force an answer. Perhaps the online diary will offer “none of the above” or “I don’t want to answer this question”, but would options like that lead to more ascription (meaning filling in the data based on other information)?
You’ve just read twelve research and operational issues and almost certainly, I’ve missed something. The researchers inside of Nielsen have likely put together a longer list and also considered the costs and benefits.
These and other questions will be asked by the various industry groups and committees such as the Media Rating Council, NAB’s COLRAM (Committee on Local Radio Audience Measurement), the NRRC (Network Radio Research Council), and the big ownership groups.
My perspective is that this change should have taken place years ago. As noted last week, the original E-Diary had one easily fixable design flaw which should have been changed and then refined over time so that the industry could adjust.
Instead, the early termination in 2007 and focus on the 48 PPM markets put the industry in the situation we have today. Nielsen Audio has painted itself into a corner. The online diary has to work because failure means the paper diary will be with us through most of this decade.
The financial side of the house has likely figured the cost savings from the change which means “damn the torpedoes, full speed ahead” regardless of the test results unless the test is a complete disaster, which is unlikely.
Further, any change means IT fixes which are often slow. Speeding things up requires the assumption that the test version can be implemented as designed, in other words, the programming changes will be done in parallel. If the test works well, you’ll see far faster implementation, which is great. If the results are controversial, you’ll also see a fast implementation, whether you like it or not.
Considering that the video side of Nielsen is facing an unprecedented amount of competition as well as criticism, the company would no doubt like to see even more profit from the near-monopoly radio service. The industry may complain, but the end result will be similar to the change to monthly rolling averages a few years back.
If you don’t like it, find another vendor. Of course, competition in this field is rather limited. My advice to the industry is to work together, ask a lot of questions, and fully understand what the outcomes mean for your companies, your stations, and your talent.
Let’s meet again next week.
