Slack meet Looker. An experiment in bringing data to the team.

A while back I read a pretty inspiring blog post, Serving Analytics the Right Way

It’s a great read by Kevin Teh, a Data Scientist at OpenDoor. Like at Buffer, they also use Looker as an analytics platform.

One part of the post I really liked was the idea of ‘Bringing data to the Team’.

“all of the data is updated in real time without requiring an analyst to do any repetitive manual work. However, there is still no guarantee that people will actually go look at the data. People get busy, and they can go weeks at a time without thinking about data, and that’s just not good. Sometimes a better approach is to have the data come to the team, rather than require the team to seek out the data for themselves.”

“Since Opendoor primarily uses Slack to communicate, the solution that works best for us is to automatically and periodically send out reports to various Slack channels”

I love this idea. It feels like it can keep key metrics front of mind, and since data is being shared in Slack where everyone can look at it, it’ll hopefully make discussions around data even smoother, more so than sending Slack messages with links to Looker.

Being inspired by this idea at Buffer we tried to see if we can have a similar setup, but ran into issues that I think the OpenDoor team also did.

Looker has the cool feature that lets you schedule saved ‘Looks’ (predefined views on to you modelled data, which can be visualized easily) and send them out as email.

We could send these emails to Slack using the built in email integration, but they weren’t formatted very nicely and would end up not being very useful.

OpenDoor ended up writing a custom app to send data to Looker, but it doesn’t seem like they open sourced it. We couldn’t find the time to look into building such an app for us. But with a cool feature added recently in Looker, we were able to get it working.

With the 3.36 Early Access release I noticed that a new feature, still currently in Labs, lets you send scheduled emails with an Inline Image attachment.

We could use this new feature to get rudimentary Slack integration with our Looker data and was able to post graphs to a specified channel in Slack. The messages look something like this.

Using Looker’s powerful data visualization capabilities, we can send out messages with all kinds of graphs, with an attached link that makes it easy to click through and explore the data in Looker.

And the best part of all? We did it by writing (almost) no code ?

How does it work?

Most of the work here is done by Looker’s email scheduling feature under the hood, but with some fun Zapier magic we’re able to send these emails to a Zap that will craft a Slack message and post it to a specific room.

To start out with, we created a Zap in Zapier that will receive a scheduled email from Looker, do some processing and formatting and then send it to a Slack room.

Here is how the Zap looks in Zapier:

This is the email address we can use to send scheduled emails from Looker to (blurred out here)

The next action will run some Javascript, using the email body to extract the ‘Explore this data in Looker’ link, which channel to send the message too, as well as a CC list of people to @mention, if needed.

Here is the full code snippet, which is pretty hacky and could definitely be improved.

// Get url
var reGlobal = /<a[^>]* href="([^"]*)"/g;  
var reGroup = /<a[^>]* href="([^"]*)"/;  
var url = '';  
if (input.bodyHTML) {  
    var matches = input.bodyHTML.match(reGlobal);

    matches.forEach(function (item) {

      var match = reGroup.exec(item);
    if(match && item.indexOf('open-in-looker') > 0)  {
        url = match[1];
    }
    });
}
if (url.indexOf('looker.buffer.com') == 0) {  
  url = 'https://looker.buffer.com' + url;
}

var cleanSubject = input.subject;  
var ccString = '';

//check for !here

if (cleanSubject.match(/\@here/g)) {  
  cleanSubject = cleanSubject.replace(item, '@here');
  ccString = ccString + '  <!here>';
}

//Get @handles
var reAt = /@\w+/g;  
if (input.subject) {  
    var matches = input.subject.match(reAt);

      if(matches) {
          ccString = ccString + ' cc';
        matches.forEach(function (item) {
            cleanSubject = cleanSubject.replace(item, '');
              ccString = ccString + ' ' + item;
          });
    }
}


//Get #room

var reHash =/#(\w|-)+/i;  
var room = 'data-alerts';

if (input.subject) {  
    var match = input.subject.match(reHash);

      if(match) {
          cleanSubject = cleanSubject.replace(match, '');
          room = match[0].replace('#', '');
    }
}

output = [{  
  'url' : url, 
  'cleanSubject': cleanSubject, 
  'ccString': ccString, 'toRoom' : room
}];

The final step will send the message to Slack. In the message template, we can use a combination of the values that the our Zapier email and Javascript actions have collected to send a nicely formatted message:

In Looker

With the Zap up and running, it’s pretty easy to set up scheduled messages in Looker, by just saving and scheduling a Look and using the Zapier email address. Just don’t forget to send it using an ‘Inline Visualization’.

Looker gives you fine grained control on how often to send the messages, as well as conditions like if there are results, if there are no results or if the results have changed since the last run.

And that’s it! Pretty easy. This can be done on any Look, but you might want to experiment with the visualization a bit and see how it ends up looking in Slack.

Looker comes with a bunch of built in visualizations, but also supports custom visualizations, which a can also be rendered and sent to Slack.

Here is an example of a Heat map I built for us to see daily movements of MRR across our different plan types.

Controlling the channel and @mention’ing people.

Since Looker doesn’t really let you send any other data in the scheduled email beyond the attached visualization, we came up with a hack to have the Zap to extract that data from title that the Look gets saved with, which gets attached to the message.

By default, any scheduled email will go to a room called #data-alerts.

If you want to change that, just add the name of the channel to Look name. You can also optionally add a list of @mentions or @here in the Look name, which will be included in the message and ping those people on Slack (obviously this should be used with care!)

The Zap javascript will extract that data from the Look title, use it as appropriate and clean up the title in the message that gets sent to Slack.

Summary

Our hope with using Looker with Slack is that we could use this as a way to surface key metrics and also alert us to important or interesting events.

Ultimately the idea is to ‘Bring data to the Team’, instead of needing people to have to go and find what they look for.

Using Slack also makes data more visible and transparent to everyone and makes discussion and collaboration much smoother.

I’m hoping that ultimately Looker will make this kind of thing easier, by perhaps providing a public API for scheduled looks or built in Slack integration.

Having data be more visible is only one side of the problem though. The hard part is finding the right time to present data that is important, insightful and actionable. Plus it should be easy to understand and visualized in such a manner that it could easily digested at a glance. This is as always an ongoing work in progress, but this feels like a great step in bringing data to the team.