I spent more than half my career in research, eventually launching and serving as President/CEO of Pinnacle Media Worldwide for fifteen years. Prior to that I was with Broadcast Architecture for ten years, founded by the late and incredibly talented Frank Cody. It was Frank who taught me that research is not just a data dump of numbers but rather a combination of art and science.
Back in the day, our offices were across the court from the Gallup Organization in Princeton, NJ, and we had an opportunity to meet with one of the grandfathers of research, George Gallup, Jr. His dad mastered modern-day polls. We explained how, at that time, most broadcast research companies used scantron sheets to score 8 or 9-second hooks of songs. He seemed intrigued and extremely engaged as we explained the process of playing the slated hooks (i.e. “song #43”) with the respondents filling in each corresponding bubble to score the song from 1-5, indicating with another bubble whether they were familiar with the song and if they were tired of the song (in some cases to what degree.) Some would ask on which station they would expect to hear that song? Today, the responses are usually gathered online via computer, phone, or tablet, but the basic scoring method is the same.
It was at this point in our conversation with Gallup when he stopped us and inquired, “Wait – so you play the segment of the song again before each of these desired responses, right?” We answered, “No, they have to answer them all before the next song starts.” He began to chuckle and tell us how ridiculous he thought that was. “You expect them to think about all of this in 8 seconds? How in the world do they even keep up?” Frankly, we already knew from watching enough auditorium testing that the respondents actually could not keep up and often fell behind by several songs.
He went on to explain the best research asks only one question at a time. If you want answers to each of those questions, you need to play the hook again, but his real question was, “Why not just ask how much they want to hear the song today? Isn’t that all you really need to determine?” That’s when it hit us and it’s where we took a methodological U-turn and began using dial testing to measure how someone “feels” about the music or product we were testing. We simply asked how much or how little they wanted to hear it if they heard it on the radio today.
Gallup also told us that he could predict the country with forty carefully well-selected respondents. His record was pretty darn good at the time. The conversation was eye-opening back in the early 90s. It’s when I began to realize how each aspect of an important research project could make or break the results. Every variable matters, and the more variables you have, the more impact each could have on the result. This includes the respondent sample and origin of the database, the location of the project, the structure and verbiage of the questions asked, the length of time it takes to go through a screener, and much more.
I always suggest caution when creating questionnaires, avoiding unintended bias and emotionally charged verbiage. Most importantly, always give respondents an out so you don’t end up forcing them into a box in which they don’t wish to be. For example, the following simple question fails to offer a response for everyone. “How often do you listen to WXXX? Often, sometimes, or rarely? Obviously, “never” is missing. I suggest choices that are less nebulous, such as “more than 3 hours a day, 2-3 hours a day, 1-2 Hours a day, less than an hour a day”. These responses are more quantifiable.
A common section in strategic research is to delve into various styles which are played for the respondents. The question asked is often, “If a radio station played that kind of music, would you listen often, sometimes, rarely, or never?” The wording of this question is extremely important. Does the question above suggest a steady, 100% diet of this music style, a couple of songs an hour, or perhaps even less? It matters since I may like rhythmic music, but maybe I don’t want a station that plays only rhythmic songs. Perhaps an efficient way to ask the question would be, “If a radio station played a mix that included some of that style of music, would you listen often, sometimes, rarely, or never?” They can be very small differences, but it is always best to be as accurate in your questions as possible.
I always suggested some basics when planning a research project:
- Remain goal-focused. What are the most important issues you need to determine? A project designed to focus on tactics is very different from a strategic study.
- Take nothing for granted. When you believe you already know how your listeners feel and what they think, you will likely be shocked and disappointed in the end.
- Don’t be greedy in questionnaire design. The temptation is to try and ask everything you ever wanted to know. You have to trim the project screener and questionnaire down to something reasonable, or respondents will drop off before completing.
- Take George Gallup’s advice and ask one question at a time. Don’t put too many variables within one question.
- The best research projects are a cooperative collaboration between management, programming, marketing and your research firm. There are several great firms from which to choose. In my office at Pinnacle Media I had a framed poster featuring various listeners that read, “In the final analysis, what we think doesn’t matter!”
Bob Lawrence serves as a Market Manager for Seven Mountains Media, overseeing the company’s Parkersburg, WV/Marietta, OH cluster. He has held virtually every position in the business over his 40+ year career, from being on-air in Philadelphia, San Diego, and San Francisco as well as programming legendary stations including KHTR St. Louis, KITS Hot Hits and KIOI (K101) San Francisco.
Bob also honed his research skills over ten years as Senior VP of Operations at Broadcast Architecture, eventually launching his own research company and serving as President/CEO of Pinnacle Media Worldwide for 15 years. Bob spent five years as VP of Programming for Saga Communications before joining New South Radio in Jackson, Mississippi as GM/Market Manager. Prior to joining Seven Mountains Media, Bob served as General Manager for the Radio Advertising Bureau, overseeing its “National Radio Talent System”.
