Engineering Innovation Ahead: Instagram, Redux, Snackchats
Engineering Report July 2016
Welcome to our first innovation & engineering report!
We’ll be sharing all the highlights from engineering and technical teams at Buffer. These includes snippets from our product engineering, data, systems and architecture teams.
This past month has seen a major product launch, Redux building, diving deeper into service-oriented architecture and also internal webinars called “Snackchats.”
Here’s a bit more about what happened in July!
Metrics:
Key Buffer server environment stats
- API number of requests: 954 million
- API average response time: 91.8 ms. Reduced from 98.8 ms a 7.08% perf improvement due to server environment upgrades
Buffer.com
- Buffer.com page views: 7.81 million
- Buffer.com average response time: 230ms. Reduced from 325ms. A 29% perf improvement from the previous month
Bugs and Quality
- 9 S1 (severity 1) bug reports – 6 closed 66%
- 38 S2 (severity 2) reports – 23 Closed 61%
Team updates:
We’re super pumped to welcome a new Android Developer to the team soon, as well as another new team member for the the architecture team.
Instagram launch lessons: Code freezes and early deploys
This month was a big launch. We’ve been building Instagram for Buffer for most part of the year with Mike, Andy and Marcus being the key engineers behind it. This was the first time where the mobile and web teams worked very closely together on project. It went really well and as a result the mobile team is coming in much closer to the web side of things, so we can create a cohesive experience for Buffer, regardless of the platform.
The launch went off without any major hiccups, thanks to a few weeks of code freeze before the launch. We found a nice bunch of bugs and improvements in that time to keep pushing for the extra bit of polish. There was, however, a tiny speed bump: the deploy going out right at the time of the launch, which was supposed to open up the feature for everyone, unexpectedly took longer than usual. This caused our scheduled Tweets to go out without the pages being ready and we had to quickly delete those Tweets. Luckily after 3 minutes the deploy was successful and things continued smoothly. Definitely a learning for us to open up the gates an hour or two before the marketing launch so things are ready for our users to enjoy.
Overall, Instagram from a technical perspective has been a big success. There have been very few bugs and things have scaled well. Big ups to Mike, Andy & Marcus!
Redux adoption is growing
There’s a been a lot of excitement for Redux building in the team. We’ve been using React for a couple years and have used Flux and many of our own hybrids of it in our projects. A few engineers have started doing much more research into it and are also building some side-projects with it. We’ve got some awesome projects in the pipeline where you might see this in action pretty soon!
The cool thing is that this interest happened independently by coincidence among teammates and before you know it, a Slack channel was born and engineers are already sharing insights and learnings from it. (Do you happen to have any cool resources or learnings to share with us? We’d love to read more!)
Pushing into service-oriented architecture
The architecture/systems team has accomplished a lot this month to further the service-oriented architecture objective. For non-engineers, we’re now employing the use of containers, which is a technology that allows us to build components of an application in a modular and scalable way.
It allows us to decouple the complexity of a large system and a larger engineering team. This way, newcomers will be able to move fast without needing to understand the entire system, and when things break, it’s only parts of the application that don’t work instead of the entire system go down.
The systems team has learned a ton about Kubernetes/k8s (the technology that will power our container cluster) and have set up various processes/services to make development easier. Our page metadata scraper was the first production service that was 100% rolled out on this new architecture in July.
It’s seen fast adoption so far as Mike is rebuilding our image resizer, Sunil is migrating all widgets/links, and Tigran has moved his analytics cronjob to this new cluster as well. Dan, Steven, Adnan and Eric continue to move at a great pace, getting logging, monitoring and dev processes in place so that we can have a fully reliable platform to develop on.
Respond tam building a prototype client
The Respond team has been moving fast to build a prototype Respond client with new technologies (React+Redux and Express+Socket.io).
They’ve found Meteor (the framework the Respond.ly product was using) a bit too constraining and have decided to test out how fast they can build a prototype we a completely new architecture. They’re also exploring running this on a Kubernetes cluster on the Google Container Engine cloud hosting platform.
This would be the first time we’re not using Amazon Web Services (which I’m excited about, so that we’re not locked to a specific cloud vendor.)
Introducing Snackchats!
Throughout the history of Buffer, especially being a remote distributed team, we’ve always struggled to get our engineers together to share and do quick talks to spread the knowledge. This time around I think we’ve hit on a concept that could last: Snackchats!
The idea is for anyone on the team to suggest (or request) 30-minute talks for a specific slot on Thursdays. The chat is recorded so other engineers who missed it, could also catch up. We’ve already had 3 awesome Snackchats talks this month from Jordan sharing about open source, Dan about containers 101 and Tigran on how our new Gnip collector and analytics pipeline works.
Exploring machine learning
Michael and Super have started experimenting with potential product opportunities that machine learning can have with our large data sets. We’ve already found neat findings with hashtags and possible suggestions for users. There’s also a new `#data-machine-learning` slack channel.
New challenge: Time-to-social-network is increasing
Colin has done an incredible investigation recently. He’s noticed that the lag between the scheduled time of an update and when the update is actually posted has been increasing slightly by 1–2 minutes.
We made some major improvements to our delays a few years back and it seems we’ve finally hit some small boundaries of failure here, especially around updates that post on the hour, as there’s usually a higher load there. The Systems team is working on this challenge to see if we can improve this delay and make sure or users’ updates always go out to the exact minute.
Giving back: Open source at Buffer
We’ve just begun exploring how to give back to the huge world of open source code. Much of Buffer would not be possible without the vast amount of open source technologies we use. We feel it’s important to get better in this area, as open source code is an intrinsic part of our values of being transparent and doing the right thing. Jordan has been leading the way on the team and has started open sourcing various components we’ve built for our own iOS app.
Today, we have three open source projects. The BFRImageViewer is a solution we built to provide an easy and configurable way to display images in iOS apps. This is a common pain point among developers, and we’ve enjoyed a great reception from the community regarding it. The second project, BFRGifRefresh, is a custom “pull to refresh” component that shows the Buffer logo while data is being fetched. Lastly, Humber wrote BufferSwiftKit, an entire wrapper over our Buffer API in Swift—which is huge since the Swift community is growing exponentially.
Over the next year, we plan to continue open-sourcing components we feel others would benefit from. We’re so excited to share that giving back to the global community of developers is now a key priority for the Buffer engineering team.
It’s been amazing to see our code help other developers out there. As of today, our open source projects have been used in 90 different apps, downloaded 3,210 times and have 253 stars in Github! We’re super thankful to be able to contribute to the open source world, and look forward to continue work in this area!
Over to you!
I hope you enjoyed July’s report. Would love to hear if there anything you’d like to hear from us and any metrics you’d like to see as well. We’re keen to live to our value of transparency here!
Check out more of our July 2016 monthly reports:
Try Buffer for free
140,000+ small businesses like yours use Buffer to build their brand on social media every month
Get started nowRelated Articles
Note: This is the quarterly update sent to Buffer shareholders, with a bit of added information for context. We share these updates transparently as a part of our ‘default to transparency ’ value. See all of our revenue on our public revenue dashboard and see all of our reports and updates here . It's been quite the y
Editor’s Note: Thanks for checking out this post! We’ve released our updated 2021 pay analysis here. You can’t improve something if you don’t know that it needs to be improved. That was very true for us four years ago when we first started looking into equal pay at Buffer. We have long used a salary formula to determine all of our salaries – the same role in the same part of the world receives the same salary. That m
Ever since the world got turned upside down by COVID-19, it’s been “business as unusual” for everyone – Buffer included. I sent this update out to Buffer’s investors one week ago. I hesitated on whether to share it more widely, as I know a lot of companies have been impacted more severely in these times. That said, I believe it makes sense to lean into our company value of transparency, since there may be some companies this could help, and it shows Buffer customers that we will be around beyon
