Helping a Feature Find Its Fit: The Story of Buffer’s Content Suggestions

Important update: We’re retiring the Suggestions feature – beginning August 1, we’ll start turning off Suggestions in phases. Read more…

One of the coolest things about working at a true startup like Buffer is how many jobs each of us gets to do in the course of a day or a project.

I joined Buffer thinking that most of my time each day was going to be spent writing, and then I discovered that I could also be working in search optimization, paid ads, conversion optimization and lots more on any given day. Pretty awesome!

One of my favorite projects I’ve had the privilege of taking on so far has been Buffer’s content suggestions. I’d love to walk you through the evolution of this experiment and share with you where this feature could be going next.

Starting out: A small experiment with suggestions

The idea of curating great articles, photos and quotes for Buffer users to add to their queues originally arose to help new Buffer users understand the app more quickly and easily.

We started out in early 2014 with 5 suggestions on weekdays—generally a mix of marketing, inspirational and lifehacking content with a great quote thrown in. Curating content was one of the first things I took on when I joined Buffer in February.

The process was beyond lean—just a spreadsheet we used to store our daily links, which I input into a CMS.

As we curated more content and expanded into weekends as well, we quickly discovered that the feature was helping all sorts of customers, not just those new to Buffer. That felt great!

Growing from experiment to feature

If five suggestions a day was good, could more suggestions be even better?

In May 2014, we expanded the suggestions feature to 25 new posts per day across five different content categories (decided from a quick qualitative survey): marketing, lifehacking, inspiration, business and startups, and design.

As shares of these suggested posts grew to up to 20,000 shares a day, the privilege of hand-curating this daily bounty was mostly mine (with lots of backup from my amazing teammates).

I had a blast reading widely from a huge variety of online sources and informative newsletters, all of which we have shared with you in various posts on the Buffer Social blog.

As our small experiment continued its journey into a feature, we began to hear from some of the content creators we featured in our suggestions and see blog posts pop up about their experiences.

Their enthusiasm was so great that a number of content producers began to ask how their pieces could be considered for inclusion, so we created a quick form (now disabled) to allow anyone to alert us to awesome content. Over the time that the form was active, more than 1,000 pieces were submitted. (And we looked at all of them!)

Content suggestions go mobile with Daily

By the time June rolled around, we focused on making content suggestions a mobile experience as well. We added content suggestions into the latest version of our mobile app, and then introduced a fun, Tinder-for-content concept with our app, Daily.

Daily ramped up even further the number of shares that content suggestions received, as it made sharing content as easy as a simple swipe or click.

Listening to customers’ feedback

Our content suggestions experiment was working well for some people, but the feature certainly was not perfect. We heard from many of our awesome customers and community members that suggestions would be more useful if they were more tailored to the user’s profiles and interests.

A Qualaroo survey we followed up with revealed the same ideas:

Another challenge was beginning to present itself, too: With Buffer’s large customer base receiving the same 25 suggestions each day, the number of shares each post was getting sometimes became overwhelming to the content creators we intended to honor. Here’s a look at the Twitter stream of the author of one content suggestion on the day it was promoted:

We were hearing a diverse number of experiences with content suggestions, and it was becoming quite interesting to think on where to take them next.

Finding product fit

Luckily, around the same time Buffer adopted a new advice-based process for making decisions, meaning that I (as the primary decision-maker) got to ask all my teammates (and even some folks outside the company) for their great advice on how to move forward.

I acted first on advice from Leo to check out Sean Ellis’ simple survey method for determining product fit. Here’s how Sean describes it on his blog:

“I ask existing users of a product how they would feel if they could no longer use the product. In my experience, achieving product/market fit requires at least 40% of users saying they would be “very disappointed” without your product. Admittedly this threshold is a bit arbitrary, but I defined it after comparing results across nearly 100 startups. Those that struggle for traction are always under 40%, while most that gain strong traction exceed 40%. Of course progressing beyond “early traction” requires that these users represent a large enough target market to build an interesting business.”

When we created a survey like Ellis suggests using Qualaroo and ran it on the site for a few hours, it was almost immediately apparent that we had not achieved product-market fit with content suggestions.

Based on this clear message, and knowing the amount of time that was being devoted to content suggestions every day, I made the decision to close down content suggestions totally.

I created a phaseout plan and we even went so far as to turn suggestions off for many users before I changed my mind and gave them a reprieve. (Turns out I could use a bit of improvement on my decision making!)

Preparing for the next evolution

So. What to do with a time-consuming, lovingly hand-curated feature that most users either don’t know about or don’t feel very strongly about?

It seems as if the ideal solution would be to reimagine content suggestions altogether. And to do that, we needed a bit of extra time.

So the latest evolution of content suggestions is designed to set the stage for a larger scale revamp in the future while countering some of the existing challenges we’ve encountered.

Fantastic Buffer developer Niel worked with me to randomize our existing pool of suggestions so every user sees a different selection each day.

Here’s a look at some of his technical specs:

Today we have a pool of 5,000-ish content suggestions, and each Buffer users sees a slightly different set of 25 each day. This has improved our feature in a few big ways:

• It evenly distributes social media posts for each suggestion (no more Twitter floods!)
• It allows posts that users might have missed to be seen again (previously, if you missed a day’s suggestions they were gone forever)
• It drastically cuts down on time spent curating content suggestions

I still love curating great, exemplary content, and doing it without an arbitrary quota each day makes sure the quality of content suggestions stays as high as possible.

Seeking out customer conversations

While Niel was working on randomizing suggestions, Happiness Hero Patrik has been talking with a wide variety of customers—some of whom love suggestions, some of whom see lots of room for improvement.

We appreciate all their viewpoints so much, and they’ve led us to a much deeper understanding of how we can make this feature soar.

Here are the validated ideas we’ve uncovered so far, from our working Hackpad:

To the next phase of content suggestions

With great insights from our users, I’m really excited to see where content suggestions will go in the future. Some ideas we have so far include:

• Personalized suggestions based on who you follow on social media and/or the feeds you’ve chosen in your Buffer account
• Personalized suggestions based on content categories you choose
• Curated suggestions by experts in tons of different fields
• Working with an existing content curation tool to add similar functionality into Buffer
• Creating our own content curation algorithm

It’s been a very exciting journey so far with this Buffer feature, and I know the best is yet to come.

If you’ve ever worked on a challenge like this one or a similar feature, how did you approach it and iterate on it? I’d love to hear your insights. And if you’re a Buffer user, it would be great to hear any thoughts you have on content suggestions!