Kit and Caboodle: A Software Development Kit to Measure All Digital TV Impressions

Kit and Caboodle: A Software Development Kit to Measure All Digital TV Impressions

Video streaming technologies are changing how we watch television today. Think about how, not that long ago, we were still arranging our lives around TV schedules—not just for sports and news, but for other programs as well. We would fix dinner, get the laundry out of the way, and sit down for our weekly dose of Thursday night comedy.

Today, at a minimum, we have the option to catch up on the shows we’ve missed, but our liberation from the TV guide goes well beyond DVRs: we can now pay just for the channels and the shows we want to watch and watch them when we want; we can subscribe to massive libraries of licensed and original content and watch entire series all at once; even the commercials we see are starting to be catered to our personal tastes; and, of course, while we can tunnel all of that exciting content to our connected television sets, we can also watch it on the go on our laptops, tablets and smartphones.

These advances are based on digital technologies, and while they’re empowering for the consumer, they can be a challenge for media researchers. There are many more devices to measure and, therefore, more engineering solutions that need to be developed. But most fundamentally, it’s the proliferation of channels and on-demand content that’s creating the highest hurdle, because it’s stretching the capabilities of the panel-based measurement solutions that have been the backbone of media research all these years. Simply put, even with a panel of 40,000 households and some 100,000 people (which is the current size of the national television panel that Nielsen operates in the U.S.), there are television shows today whose audience is too small to show up reliably in the ratings data.

One part of the solution is to increase the size of the panels we use in the industry for television measurement. Nielsen has major engineering developments underway that will make that possible*. But there’s no question that the solution also needs to involve engineering developments that can capture and make sense of TV usage outside those panels—something we refer to as ‘census data collection.’

How do we accomplish this? Nielsen engineers have created a software library—called the software development kit, or SDK—to install on all content owner apps, aggregator apps and browser pages that render television content. It’s not a physical meter, but rather a collection of software plug-ins that can be embedded in existing applications and turned on to capture impressions. If the consumer is watching content on a laptop, then the SDK is embedded in the browser. If the consumer is watching on an iPhone, then the SDK is embedded as an iOS app. If the consumer is watching on a Roku box, then the SDK is embedded in the Roku media player. In order to minimize the number of independent SDK implementations, we have also partnered with Adobe to provide an implementation option of their SDK that has our measurement built in. This is typically offered to mutual clients of Nielsen and Adobe.

The SDK needs to be flexible enough to recognize a variety of unique content identifiers. In the case where the content originates from linear TV, it already contains a Nielsen watermark—an inaudible code that’s inserted in the audio signal by one of thousands of encoders deployed at content distribution facilities around the country. That watermark is the foundation of the current TV ratings system, but it can be used as well if that same content is distributed and consumed as digital content. It just needs to be transcoded first because audio is difficult to get to on digital platforms, for security reasons. To address this issue, Nielsen created software that is embedded in most leading transcoders (those appliances that create the streaming content) to extract the watermark from the audio and insert it as metadata in the digital stream. This metadata tag, called ID3, is supported on most leading streaming formats and is much easier to extract from the streaming content.

In the case where the content is natively digital and doesn’t have a Nielsen watermark to begin with, we rely on metadata from the providers’ content management systems (CMS) to identify what’s being played: name of program, episode, genre, etc. We may also apply our own tags if a particular provider’s CMS is difficult to access. Tagging digital content with CMS tags is an important function not just for programs, but also for ads. Dynamic ad exchanges and programmatic technologies are revolutionizing the media industry. The ads a consumer sees with their television content might be the same as those associated with that content in the linear TV experience, but they might also change after a period of time or be an entirely different set of ads from the outset. Without a separate tag to identify those ads, we wouldn’t know what ads people were exposed to.

Once the identification mark is captured for content and ads, whether in the form of an ID3 tag or a CMS tag, it’s aggregated with other marks and transmitted to back office facilities for final processing and analysis. But there’s one piece that’s still missing before digital content and ad ratings may be computed: demographics.

Only a small portion of the census impressions collected by the SDK correspond to Nielsen panel members. We have detailed personal data for panel members (demographics, lifestyle data, technology ownership, etc.), but not for the millions of consumers sending us data from their laptops, smartphones and other connected devices. Through data partnerships with carefully selected third-party providers and using procedures that safeguard consumers’ privacy, we’re able to receive aggregated demographic data that is calibrated against our panel data. This allows us to put a face on all the faceless impressions, and finally combine those digital ratings with linear ratings to paint a complete picture of a show’s performance—its ‘total audience.’

We don’t live in a world anymore where one technology, or one delivery mechanism, completely dominates the entertainment industry. The options have multiplied, the viewer is in charge now, and that means that media researchers need to be quick to adapt to emerging technologies. Audience measurement has never been more complicated, but with the right architecture in place and a culture of engineering innovation, the industry is up to the challenge.

* See “Smaller Cheaper Stronger: the Nano Meter” in this issue.