Don’t diss me!

posted on Nov 26 2012

In our last two posts we concentrated on proving that increasing PTAT has a direct effect on the organic reach of your Facebook page. In this and the next update, we want to analyze the second value that now has an effect on the Edgerank algorithm since the change in September 2012: negative feedback. In this post we will show that this value is relatively unlinked to organic reach. Next time we will show how you can improve your frequency by careful monitoring of negative feedback. In a third and last post on this, we will give our opinion on whether the usage of this value drives a better user experience.

by Jon Handschin
Co-Founder moviepilot.com

There has been a heated debate on how negative feedback affects Facebook pages performance since the algorithm update. A central piece was the article “Killing rumours with facts” by Techcrunch writer Josh Constine. Summarizing his position, he argues that the reduction in organic reach experienced by many pages in the last week of September is due to the roll out of the new negative feedback feature on the news stream rather than based on Facebook’s intent to push the promotion feature in an attempt to make page owners pay for additional reach. While Techcrunch’s article concentrates on the reason why the organic reach is decreasing, we instead want to understand which values are driving this change in order to gain insight on how page managers can improve their page’s performance.

First of all, let’s define Facebook’s term ‘negative feedback.’ Until now this value has not included the ubiquitous trolls that go ranting and raving in the comments of your posts; especially prevalent in the absence of a “dislike” option on Facebook. (Note that we strongly believe that a semantic analysis of comments will follow in the near future.)

Negative feedback instead summarizes the four negative actions a user can take when displayed with a page post. The user can: 1. Hide a story, indicating he or she is not interested in this particular post, 2. Hide all stories, indicating that he or she is not interested in any posts from your page anymore, 3. Report the story as offensive or as spam, thus alerting the Facebook security system, 4. Completely unlike your page,

These options have been made more accessible in the Facebook UX design. Here is a current screenshot:

Constantine argues that by making these negative options more accessible, more users express their dislike. With the help of Page Lever he composed the following chart. It aggregates the complaint ratio for more than 700 Facebook Pages across a period of several months. This data shows that the roll out of the more accessible negative feedback feature led to a large increase in negative feedback throughout September. This was subsequently matched by a decrease in organic reach in order to improve the user experience.

Unfortunately, there isn’t a clear indication for how the complaint ratio for this has been derived since Facebook offers you a variety of values. They provide unique complaints vs total number of complaints, as well as views, meaning impressions or reach. The TC article falls short, in our view, of determining the underlying ratio accurately.

Still, we believed that if their findings were true for so many other pages, we should be able to reproduce similar findings in our data, especially for those pages that were experiencing a decrease in organic reach throughout September. In order to analyze this, we aggregated the weekly organic reach from five of our comparable pages. All of them had between 100K-600K fans during September and October 2012. The chart represents the aggregated weekly organic reach across all of these five pages from September 1st until the middle of November.

It provides an accurate example of the drop we experienced when the update took place. It also shows that the change in our posting strategy, from posts aimed at driving traffic to posts aimed at engagement, was successful in regaining our organic reach. By increasing the PTAT values across pages from October 1st onwards, we were able to exceed the organic reach we had before the update.

We then calculated a daily complaint rate for all of these pages. We chose to define the complaint rate as the total amount of a page’s negative feedback divided by its daily organic impressions. The rationale behind this is that we are fairly certain that negative feedback is used as an indicator to Facebook if a page is posting too much, and thus influences the frequency of the distribution of the page’s posts. If the Techcrunch article was correct, we expected to find a similar increase in the daily complaint rate in September before the decrease in organic reach.

The below chart uses a double axis in order to represent the aggregated organic reach vs the aggregated daily complaint rate across our pages. But the curves do not resemble the Page Lever / Techcrunch chart. The complaint rate doesn’t peak extraordinarily high after the roll out of the new negative feedback feature in September. Rather, the value goes down simultaneously with the decrease in organic reach. Note that it also slowly rises again with the increasing organic reach. To us, it appears to be more of a dependent variable in this model rather than a driver.

We are not arguing that the Page Lever data is incorrect, we just haven’t been able to reproduce the same direct effect on our pages’ organic reach. Instead we have observed that organic reach is best driven by improving a page’s PTAT. Should you therefore forget about the complaints of your fans? Definitely not. They influence a value even more important than your organic reach, one that Facebook has recently hidden a little deeper in your Insights section — the weekly organic impressions of your page posts.

In our next post we will discuss how successfully monitoring the complaint rate increases your page’s frequency rather than its organic reach. So stay tuned and please leave (positive) feedback if you found this helpful.
You can also follow me on Twitter here.

  1. moviepilotlabs posted this
comments powered by Disqus