It’s been over a year since the Nielsen-Arbitron deal happened, and in this week’s Programming To Win, Paragon’s Larry Johnson takes a look at the state of radio ratings. Johnson gives a little recent history of the PPM, as well as the overall past of Arbitron and radio ratings.Johnson critiques the current radio ratings methodology and takes a look into the near future as well.
Larry Johnson
By Larry Johnson
Here we are well into the 21st century and the yardstick we use to measure our programming success seems badly warped. Radio ratings should allow us to know how our programming efforts are being received by the public. Audience data also is the grist for media planners and buyers advertising on radio. And, reliable data on all medium’s audiences is needed as all the new media platforms are integrated into our everyday lives.
Real time radio measurement as provided in the Portable People Meter (PPM) has given us a truth about how people listen to radio. People listen to nearly twice the number of stations using the PPM technology than do people who self-report their behavior in the diary-based ratings. However, people spend roughly only half the time with the stations in PPM as they did in the diary. People appear to ‘vote’ for their stations in the diary ratings methodology.
However, for those of us trying to measure the success of our craft, PPM has been a maddening development. The Program Director, General Manager, and anyone else held accountable are subject to wild ratings wobbles from month to month. Getting a measure on how our programming efforts are doing is made difficult if not impossible given these significant statistical variances.
We see it in audience analysis: It’s nearly impossible to establish a target audience when your core audience varies from month to month. One month your core audience is Women 35-44; the next it’s Men 45-64. Argh!
Nielsen certainly knows how to do diaries because they inherited Arbitron’s rating expertise. Despite being a time-warped recall system based on memory (at best), the diary gives us reliable data, especially when rolled into a two or four book average. We’ve learned how to spot a spike that most likely will return to the norm next rating period.
So we’ve got a real-time PPM methodology that gives the ad buying community purportedly more ammunition in defending their buy, but results in programmers going insane and occasional job losses. The diary isn’t looking so bad through this lens. Nielsen ratings analysis software is excellent. One can analyze unreliable PPM data as well as diary-generated ratings in a number of constructive and creative ways. The problem with PPM is that we’re slicing and dicing unreliable data. (A reliable sample is one that would yield the same results within a 5% margin of error if two surveys using the same sampling and methodology were conducted at the same time.)
Research is a business: the business model must be viable. Nielsen has to report back to their shareholders. Research is expensive to conduct. Somebody’s got to be paid to place those People Meters, someone has to make the diary call reminders, and someone has to physically go to a household not reachable by landline or cell phone. Nonetheless, there are statistically defensible, minimum sample sizes.
Yet, radio group heads keep buying PPM with the current unreliable sample sizes. There is no desire to spend more for the ratings, even if it means making the PPM sample reliable. The overall picture seems close enough for broadcast firms subscribing to the ratings, and PPM satisfies ad buyers’ demand for real time measurement.
How Did We Get Here?
In the early days of radio research, respondents were called by phone and asked to recall their listening for the past two days (Pulse Radio Ratings.) Arbitron came in with the diary. The diary measured a respondent for seven days, thus a more reliable sample because of more observations about each respondent. In those early days the number of phone sample and diary sample were fairly commensurate. Arbitron warned us about reading too much meaning into monthly trends (and definitely not weekly trends). But alas with the People Meter, the not-so-great compromise was struck: Arbitron answered the media planners’ drumbeat for real time measurement, but cut the sample size 66 to 75% in order to meet their own business plan. Arbitron contended that these remaining respondents would report a whole month’s worth of listening on the Portable People Meters thereby making up for the sampling shortfall. Well, if you ask basically the same people in a small sample every month about their preferences, you’re likely to get about the same listening patterns from each respondent. There’s about a 7 to 10% drop off in panelists each month. That’s why when one, two, or three new panelists change, you can see profound changes in listening patterns and audience composition given the initial, small PPM sample size.
When Nielsen and Arbitron were going to partner up on the People Meter in the 1990s, Arbitron was conducting television and radio ratings. However, Nielsen pulled out of the deal, and Arbitron gave up rating television. So Arbitron was stuck with a commitment to the ad buying community to enter the 21st century with real-time radio-only ratings.
So here we are in the first part of the 2000s with a flawed PPM sampling system, and a diary methodology that works, albeit as a measure of people voting for their favorite stations. Looming large is the onslaught of digital and new media. Getting accurate audience measurement of how people consume ALL audio media is more and more essential with streaming, podcasts, mobile phone as a platform, and time-shifting. Nielsen purports to measure it all.
Yet, how can a measurement firm like Nielsen purport to measure all sorts of different audio mediums in real and shifted time when they can’t even provide reliable radio ratings? Something seems amiss with a business model that can’t deliver reliable radio ratings product, yet purports to geometrically expand to rate content over all sorts of platforms.
It would appear that the only pragmatic way to measure all audio media would be to insert an embedded code in all audio content. That way the Portable People Meter could measure all audio content. One can only hope that Nielsen’s business model will allow it to reliably measure all audio listening on all media platforms. That means dramatically larger sample sizes and the dramatically increased costs (hopefully spread out among many more subscribers). It’s difficult to envision the media industry embracing the huge sample size and resulting costs that would be necessary for a reliable PPMs sample of all mediums.
The most promising development in reliable research comes from a canvas of what people are accessing over the Internet. Google Analytics and Triton Digital offer such services now. This is great for streaming measurement. And, as people listen over their cell phones, that listening could be measured via the device. Canvas measurement of all audio could remain elusive until the time that all platforms would also automatically report media usage. Each device would be an ipso facto people meter. That would be ongoing census research with little or no error. It will be interesting to see how the technological progress matches up with our ability to measure listening to audio. Let’s hope the reliability and integrity of the measurement are there so people can truly know how their audio products are being received by the public, and the ad buying community can trust that they’re making decisions on reliable data.
Larry Johnson is President/North American Radio for Paragon Media Strategies. Reach him at 831-655-5036 or via e-mail at ljohnson@paragonmediastrategies.com.
