Last Friday Facebook announced that it had discovered a bug in Facebook Page Insights. The bugs were likely introduced around the same time that Facebook made changes to its EdgeRank algorithm, however the bugs were unrelated to the EdgeRank changes, originating with changes made to Facebook's Android and iOS apps.
Facebook didn't specify exactly when they became aware that Page Insights were not being properly reported, only that they began working on a solution immediately once the errors were found. I don't have any new information for you here, however I have taken information from three sources, Facebook's original announcement on FacebookStudio.com, TechCrunch's article a few hours later, and the notification that all Facebook page admins will see at least twice.
I've stripped out all of the fluff to give you this easily digestible guide that answers your most pertinent questions and provides you with the next steps you should take to understand and minimize the fallout in the reporting structure for your own Facebook Pages.
Which Facebook Metrics Were Affected?
Reach- Paid Reach, Organic Reach, and Viral Reach
Impressions- Organic, Paid, and Viral Impressions
(Note: Check out the link if you're not familiar with the difference between Reach and Impressions.)
Engagement- Metrics that include Reach or Impressions in their calculation, such as individual post virality, will also be incorrect.
How Are Those Metrics Incorrect?
Reach and Impressions were not recorded for mobile users. Essentially, as far as Page Insights were concerned, fans who saw your posts on mobile devices didn't exist, and were not counted.
Impressions for ads that appear in the newsfeed were counted twice. Once as organic and once as paid. This bug only affected PC/Mac/Linux users, not mobile users.
- Facebook says that all ads were delivered properly, however, and thus no one was over or under-charged for ads. Facebook also reported that their ads platform has extensive monitoring and verification measures already in place to prevent something like this from happening to their Ad Insights.
Engagement metrics, Virality being one example, that are calculated by comparing one metric to Reach or Impressions will also be affected. When Reach increases and engagement does not, Virality drops. So, if your Reach was under-reported, then your post's Virality has likely been reported as higher than it actually was.
What Changes In Metrics Should I Expect Going Forward?
Facebook reports that changes will vary from page to page, and some may not be affected at all, or at least not very significantly. You may however see an increase in Reach and Impressions, and if so, you'll likely also see a decrease in post virality, because it's a comparison of people who saw your post vs. people who interacted with your post.
How Long Has This Been Going On?
Facebook did not specify exactly how long their Page Insights have been incorrectly reported, but it's safe to assume that data reported from late September up until today is not accurate. You should expect accurate data to be reported once again starting tomorrow (Monday, February 24th, 2013).
Facebook also announced that metrics affected during this period can't be corrected for historical data, so the currently incorrect data from this time period will remain in place.
What Is Facebook Doing About it?
Facebook claims to have already placed new quality and verification measures in with their metrics data to both verify the data and actively monitor for bugs in the future.
So, What Should I do?
- Don't trust your metrics data from August of 2012 through February 24th, 2013. Reach and Engagement are essential Facebook metrics for most brands, so these bugs make the meat and potatoes of your metrics data useless for this period.
- Starting February 25th, 2013 you should monitor the metrics mentioned in this post for about a month. Looking at data from February 25th through around March 27th will give you a better idea of what your actual numbers are for the affected metrics.
- Comparing the correct data starting on February 25th to the measurements for the same metrics from August 2012 - February 24th, 2013 will give you some idea of just how off the reported metrics were for your page while the bugs were active. So, you can at least have an accurate ballpark figure as to what your true measurements were while these bugs were active, and know if the changes caused these metrics to be over-reported, or under-reported for your page.
Some people are speculating that these bugs coinciding with the changes in how Facebook calculates EdgeRank are too much of a coincidence. I doubt this is the case, after all, having to admit that key metrics have been miss-reported for months is far more embarrassing than having to admit that EdgeRank was altered to encourage additional ad spend from brands. In fact it could lead to less confidence in Facebook ads and Facebook as a business marketing platform. It certainly does lead to some interesting questions about the data Facebook has shared with us though, particularly their claims that the changes to EdgeRank at the end of September did not hurt Reach for most brands, when the bugs would have certainly caused a noticeable decrease in Reach, which Facebook should have seen and attributed to something. Considering that Facebook now admits Reach most likely did decrease for many brands, just not for the reasons we all thought, their previous denial of any decrease stands at odds with their own data.
For the optimist in all of us, at least once you compare the buggy metrics to your month's worth of correct metrics, you'll have some great data for proving to management exactly how important mobile is for your brand's efforts on Facebook (since the bugs essentially caused insights to pretend like mobile users didn't exist), and what you stand to miss out on if you don't optimize content for those users!
