If you’ve been around ratings for a while, the recent complaints from the NFL regarding Nielsen estimates sound familiar. I have yet to hear anyone in the media business argue publicly that their ratings were too high. Or that Nielsen overshot and the reality is lower. Occasionally, someone at the station level may privately admit that. But complaints about low numbers are as consistent as New York Jets losses.
The NFL is just the latest. I’ve been on both ends of this complaint in my career. The more you know, the more you understand the system gives out odd results from time to time. More decibels rarely result in higher estimates.
But I’m citing the NFL issue for a different reason. Sports Media Watch quoted Nielsen CEO Karthik Rao’s response to the NFL that the company has spent “months working hand in hand” with its clients and industry figures “to make Big Data + Panel even more accurate.”
When I started at Arbitron in 1999, one of the company’s attorneys, Pat O’Donnell, stopped by my office with a memo that she handed out to newbies who would be talking to outsiders. The memo stated that Arbitron was under a Federal Trade Commission consent decree. In the vein of George Carlin’s “7 Words,” there were certain words we could not use in outside conversations. Not insults, but words like “truth” and “accurate.” If memory serves, there were twenty words, give or take.
I didn’t take the memo with me when I left the company in 2015. I was always curious though about the consent decree’s specifics. When you want to look up something like a consent decree online, it should be easy. This was a bear. Nevertheless, your intrepid columnist found it, and not only that, there were two more!
The 1962 Consent Decrees
Let’s ride the time machine to the end of 1962. An entity called CEIR Inc. (Committee for Economic and Industrial Research) owned Arbitron at that time. A few years later, CEIR Inc. was sold to Control Data, which became Ceridian. They spun off Arbitron in 2001 before Nielsen purchased Arbitron in 2013. Keep in mind that in 1962, Arbitron, then known as the American Research Bureau, measured only television.
The document is “In the Matter of C-E-I-R Inc.,” dated December 28, 1962, docket C-289, which is a consent order “in regard to the alleged violation of the Federal Trade Commission Act.” The summary states:
“Consent order requiring a Washington, DC company engaged in the measurement of television audiences and the publication and sale to broadcasters and advertisers of data and reports based thereon, to cease representing falsely that the television station and program ‘ratings’ and audience ‘totals’ compiled by it — which did not disclose the number that failed to cooperate and included hearsay reports and estimates — were accurate measurements arrived at through the use of techniques and procedures free from error other than sampling error.” (61 F.T.C. pages 1468-69)
The consent decree goes on for a few more pages with specifics, ending on page 1473.
Turning to page 1474, the same summary appears, but surprise, Docket C-290 is “In the Matter of A. C. Nielsen Company.” Then we reach Docket C-291 with a similar summary, “In the Matter of The Pulse Inc.” Those of you old enough to remember Pulse radio ratings know that the company shut down in 1978.
Now you understand why Nielsen reports have all that boilerplate that you’ve never read. The requirement is most likely older than you are.
The Harris Committee and a Lasting Rule
The next year, 1963, saw the companies investigated by a House subcommittee, known as the Harris Committee for the chair, Congressman Oren Harris of Arkansas. One upshot of the hearings was that the industry should self-regulate, which begat what we know today as the Media Rating Council (MRC). While the MRC was not created by an act of Congress, Congress clearly told the industry to get their house in order. If it didn’t, Congress would do it for them. Can you imagine a Federal Ratings Commission?
My point is that the consent decrees still exist, and while C-E-I-R and the A.C. Nielsen Company have taken on different forms over the decades, ratings suppliers can’t refer to “accuracy,” “truth,” or similar words when talking about ratings. Sampling error is a legitimate measure, but non-sampling error (everything else involved in surveys) probably accounts for far more variation. And Nielsen’s PR and legal teams need to be aware of the quotes they give their CEO or what he says publicly.
This doesn’t mean that Nielsen is under-representing audiences to NFL games, nor does it mean Nielsen does a poor job of measuring radio audiences (thankfully, online columns don’t come with laugh tracks). But long before any of us got into this crazy business, there were concerns about ratings. I’ll leave you with a quote:
“I frankly worry about broadcasters becoming locked up in jails they build for themselves. Creativity, for example, is now being curtailed by slavish addiction in some quarters to audience measurement or ratings of questionable validity and administered outside any qualitative control of broadcasters.”
Does that sound like something that has crossed your mind? The speaker was LeRoy Collins, NAB President, at a panel at Northwestern University. It’s from 1961.
Let’s meet again next week.
Barrett Media produces daily content on the music, news, and sports media industries. Sign up for our newsletters to stay updated and get the latest information right in your inbox.
