Covering Disruptive Technology Powering Business in The Digital Age

Over the past month and especially the past week, there has been a considerable amount of discussion and media coverage regarding Facebook’s acknowledgement that one of its key video display metrics had generated confusion. In short, the Average Duration of Video Viewed was defined in its documentation as “total time spent watching a video divided by the total number of people who have played the video” while in reality, the metric actually recorded “the total time spent watching a video divided by *only* the number of people who have viewed a video for three or more seconds.” As the advertising community digested this news and Facebook replaced this metric with a new one, it generated a firestorm of criticism about the opaqueness of the metrics that define the online world.

Yet, from the standpoint of someone with a data sciences background, it is difficult to understand what the perceived controversy is about and why many in the advertising community feel misled by Facebook’s documentation. Indeed, a quick perusal of the leading headlines from the last 48 hours are filled with ones describing an almost sinister attempt to mislead and deceive the advertising community. But, if one simply looks back to Facebook’s original announcement of the video metrics dashboard, one will find that they prominently and clearly announced this definition from the very first release.

On May 5, 2014, Facebook announced its forthcoming new video metrics dashboard giving content authors better insights into how their videos were performing on the platform. Right up at the very top of the page, Facebook states “a ‘video view’ is defined as a view of three seconds or more and will appear for all videos, including those that come to life as people scroll through News Feed.” While at the time this redefinition of a view from YouTube’s 30 seconds to Facebook’s 3 seconds proved controversial, Facebook clearly and unequivocally offered a precise definition that all of its platform video metrics would define a “view” as a duration of at least three seconds.

Facebook did not respond to a request for comment, but as someone whose work focuses heavily on how we quantize our world into precise measurable metrics, it has been fascinating to see the strong reaction to Facebook’s clarification of its video viewership metric given that it had already precisely defined what constituted a video “view.” The visceral reaction to Facebook’s announcement appears to be a combination of legions of technology journalists who are simply repackaging viral stories without digging deeper and an advertising community that made wrongful assumptions about what their metrics were telling them instead of reading the documentation. At worst, Facebook is guilty merely of having its documentation spread across two different pages rather than copy-pasting it as a reminder for the harried analytic professional who was in too much of a rush to do a quick Google search on what Facebook considered a “view.” While others might argue Facebook’s previous definition deviated from other accepted industry norms of simply counting every access of a video, again when it comes to the analytic world, one always has to look at documentation, not make assumptions from one’s personal experience.

This is not a story limited to Facebook. I encounter such definition issues every day when it comes to everything from social media to sensor data. As I’ve written here again and again, the rise of easy access to data and computing tools means people are increasingly using data without taking the time to understand what it is they have in their hands. There certainly is truth to the fact that many data vendors do not provide all of the information necessary to fully document their datasets (for example, how platforms define what constitutes an “active user” is notoriously fuzzy and fluid) or have errors or mismatches between what’s in their data and what the documentation says should be in there. But, at the end of the day the majority of the troubles people run into when it comes to understanding data comes from a failure to carefully examine it before jumping right in and using it in a real world application. Perhaps in the future the advertising community will spend a little more time reading the definitions of the metrics they use and perhaps the broader analytics community will use this as a teachable moment in considering how they communicate their own metrics to their users to ensure they don’t run into similar errant assumptions. Always remember that data itself is meaningless without the definition that says what it measures and when publishing data always make sure to make those definitions precise and well documented for your users.

This article was originally published on www.forbes.com and can be viewed in full here

(0)(0)