We recently started a small project to clean up how parts of our systems communicate behind the scenes at Buffer.

Some quick context: we use something called SQS (Amazon Simple Queue Service. These queues act like waiting rooms for tasks. One part of our system drops off a message, and another picks it up later. Think of it like leaving a note for a coworker: "Hey, when you get a chance, process this data." The system that sends the note doesn't have to wait around for a response.

Our project was to perform routine maintenance: update the tools we use to test queues locally and clean up their configuration.

But while we were mapping out what queues we actually use, we found something we didn't expect: seven different background processes (or cron jobs, which are scheduled tasks that run automatically) and workers that had been running silently for up to five years. All of them doing absolutely nothing useful.

Here's why that matters, how we found them, and what we did about it.

Why this matters more than you'd think

Yes, running unnecessary infrastructure costs money. I did a quick calculation and for one of those workers, we would have paid ~$360-600 over 5 years. This is a modest amount in the grand scheme of our finances, but definitely pure waste for a process that does nothing.

However, after going through this cleanup, I'd argue the financial cost is actually the smallest part of the problem.

Every time a new engineer joins the team and explores our systems, they encounter these mysterious processes. "What does this worker do?" becomes a question that eats up onboarding time and creates uncertainty. We've all been there — staring at a piece of code, afraid to touch it because maybe it's doing something important.

Even "forgotten" infrastructure occasionally needs attention. Security updates, dependency bumps, compatibility fixes when something else changes. This led to our team spending maintenance cycles on code paths that served no purpose.

And over time, the institutional knowledge fades. Was this critical? Was it a temporary fix that became permanent? The person who created it left the company years ago, and the context left with them.

How does this even happen?

It's easy to point fingers, but the truth is this happens naturally in any long-lived system.

A feature gets deprecated, but the background job that supported it keeps running. Someone spins up a worker "temporarily" to handle a migration, and it never gets torn down. A scheduled task becomes redundant after an architectural change, but nobody thinks to check.

We used to send birthday celebration emails at Buffer. To do this, we ran a scheduled task that checked the entire database for birthdays matching the current date and sent customers a personalized email. During a refactor in 2020, we switched our transactional email tool but forgot to remove this worker—it kept running for five more years.

None of these are failures of individuals — they're failures of process. Without intentional cleanup built into how we work, entropy wins.

How our architecture helped us find it

Like many companies, Buffer embraced the microservices movement (a popular approach where companies split their code into many small, independent services) years ago.

We split our monolith into separate services, each with its own repository, deployment pipeline, and infrastructure. At the time, it made sense: each service could be deployed on its own, with clear boundaries between teams.

But over the years, we found the overhead of managing dozens of repositories outweighed the benefits for a team our size. So we consolidated into a multi-service single repository. The services still exist as logical boundaries, but they live together in one place.

This turned out to be what made discovery possible.

In the microservices world, each repository is its own island. A forgotten worker in one repo might never be noticed by engineers working in another. There's no single place to search for queue names, no unified view of what's running where.

With everything in one repository, we could finally see the full picture. We could trace every queue to its consumers and producers. We could spot queues with producers but no consumers. We could find workers referencing queues that no longer existed.

The consolidation wasn't designed to help us find zombie infrastructure — but it made that discovery almost inevitable.

What we actually did

Once we identified the orphaned processes, we had to decide what to do with them. Here's how we approached it.

First, we traced each one to its origin. We dug through git history and old documentation to understand why each worker was created in the first place. In most cases, the original purpose was clear: a one-time data migration, a feature that got sunset, a temporary workaround that outlived its usefulness.

Then we confirmed they were truly unused. Before removing anything, we added logging to verify these processes weren't quietly doing something important we'd missed. We monitored for a few days to make sure they were not called at all, and we removed them incrementally. We didn't delete everything at once. We removed processes one by one, watching for any unexpected side effects. (Luckily, there weren't any.)

Finally, we documented what we learned. We added notes to our internal docs about what each process had originally done and why it was removed, so future engineers wouldn't wonder if something important went missing.

What changed after clean up

We're still early in measuring the full impact, but here's what we've seen so far.

Our infrastructure inventory is now accurate. When someone asks, "What workers do we run?" we can actually answer that question with confidence.

Onboarding conversations have gotten simpler, too. New engineers aren't stumbling across mysterious processes and wondering if they're missing context. The codebase reflects what we actually do, not what we did five years ago.

Treat refactors as archaeology and prevention

My biggest takeaway from this project: every significant refactor is an opportunity for archaeology.

When you're deep in a system, really understanding how the pieces connect, you're in the perfect position to question what's still needed. That queue from some old project? The worker someone created for a one-time data migration? The scheduled task that references a feature you've never heard of? They might still be running.

Here's what we're building into our process going forward:

• During any refactor, ask: what else touches this system that we haven't looked at in a while?
• When deprecating a feature, trace it all the way to its background processes, not just the user-facing code.
• When someone leaves the team, document what they were in charge of, especially the stuff that runs in the background.

We still have older parts of our codebase that haven't been migrated to the single repository yet. As we continue consolidating, we're confident we'll find more of these hidden relics. But now we're set up to catch them and prevent new ones from forming.

When all your code lives in one place, orphaned infrastructure has nowhere to hide.

Delivering consistent mobile experiences is hard.

Between iOS and Android's distinct design languages, different versions of native components, and Buffer's own design language, mobile apps can sometimes feel fragmented. Designers and developers end up speaking different languages, duplicating work, and shipping experiences that feel inconsistent across platforms.

At Buffer, we really felt this friction. Our mobile design workflow wasn't as efficient as it could have been. We spent too much time reinventing the wheel, manually patching together screenshots, and playing catch-up with our web app counterpart. We knew we needed a better way.

So we built one.

Meet 🍿 Popcorn To Go

Buffer's new mobile design system for iOS and Android. It's our answer to the chaos, and it just passed its first major test: helping us ship our iOS app with Apple's new Liquid Glass design language the moment iOS 26 launched back in September 2025.

Let's dig in. 🍿

Why we built it

Before Popcorn To Go, our mobile development process had some painful friction points:

• Miscommunication between design and engineering. Without a shared design language, handoffs were slow and error-prone. Our iOS app ended up with 300+ colors, most of which were slightly different shades of the same color. No source of truth existed.
• Design decisions made on the fly. With no source of truth, engineers were left to improvise and take on-the-fly design decisions to make things work.
• Inconsistent and inaccessible UI. Minor differences crept in between platforms, and even between different screens on the same platform. Our apps didn't feel as polished as they could be, and we weren't fully using the accessibility features built into native components.
• Dated look and feel. With all these things piling up, it became harder to adopt the latest native components or implement changes to Buffer's general look and feel.

These problems started to hold us back. Our vision for Popcorn To Go was simple: create a system that delivers efficiency, consistency, accessibility, and future-proofing, without sacrificing the unique character and advantages that native components bring to a small mobile team like ours.

The goals of Popcorn To Go

We set out with four clear goals:

1. Efficiency for engineering and design teams through standardized components and smart use of native platform components.
2. Unified design language that reduces miscommunication and speeds up iteration.
3. Accessibility baked in by inheriting best practices from iOS and Android's native components.
4. Readiness for platform evolution, like iOS 26's Liquid Glass, so we can move fast when the platforms do.

How it works

At its core, Popcorn To Go is built on two key concepts: tokens and component kits.

Tokens are the design decisions that define your visual language — things like colors, spacing, typography, and border radii. Think of them as the ingredients in a recipe. Instead of hardcoding "use brand green #8FC67D," we define a token like fill-brand that automatically adapts across light mode, dark mode, and different platforms. This means less chance of the wrong color being applied at any point.

Component kits are pre-built UI building blocks (buttons, cards, navigation bars) that use those tokens. They live in Figma for designers and are implemented in code for engineers, creating a shared source of truth.

The tricky part? Balancing platform specificity with cross-platform consistency.

iOS and Android have their own design languages: Apple's Human Interface Guidelines and Google's Material Design. We didn't want to flatten everything into a generic "lowest common denominator" experience. Instead, Popcorn To Go respects each platform's native patterns while maintaining a cohesive Buffer feel.

This approach comes with a bonus: we get to use ready-made components that are stress-tested by the native platforms for accessibility and cross-device compatibility — a huge asset for a two-person mobile engineering team.

Here's how we structured it in Figma:

Token relationships between Figma files across the Web and Mobile design systems

• Mobile/Styles: Our foundation layer with primitive colors and platform-specific tokens. We used Material 3 naming for Android and custom naming for Apple. The primitive colours mirror those in our web app.
• Mobile/Android M3: Components built with Google's Material 3 Expressive language, fully linked to our Android tokens.
• Mobile/iOS & iPadOS 26: Native iOS 26 components using Apple's Liquid Glass design language linked to our Apple tokens.
• Mobile/iOS & iPadOS 18: A lighter-touch kit for the previous iOS version (since we support one version back).
• Mobile/Custom Components: Buffer-specific components that don't exist natively on either platform.

Design operations challenges we solved

Getting this system working smoothly meant tackling some gnarly design operations challenges:

• Figma linking: The biggest challenge we faced was linking primitives. In an ideal world, the primitive colors would come directly from our main design system, Popcorn, and Popcorn To Go would simply map these to Android or Apple-specific tokens. However, Figma's current feature set doesn't support this. We had to create a new primitives file for Popcorn To Go that manually mirrors the web's primitives.

• Token naming: Creating a naming system across web, iOS, and Android that is somewhat streamlined whilst respecting platform-specific conventions.

• Kit styling: Applying our tokens to platform-specific kits while maintaining flexibility for future updates. This required using several handy plugins like Figma Tokens and Variables Importer.

Honestly, it's not the perfect, smoothly connected & humming system every designer dreams of when setting up a design system.

Apple's component kits, in particular, are complex and sometimes inconsistent, whilst Android's token naming is very specific and tricky in its own way. But we landed on pragmatic solutions that work for everyday use and achieve the goals we set out to achieve.

Strategic timing: The iOS 26 test

We launched Popcorn To Go with intentional timing. iOS 26 was on the horizon, bringing Apple's new Liquid Glass design language: a fresh, modern aesthetic with frosted glass effects, refined animations, and elevated visual polish.

By building Popcorn To Go before iOS 26 launched, we positioned ourselves to:

• Be ready from day one when iOS 26 dropped
• Leverage the latest platform capabilities immediately
• Ship our app's visual refresh alongside Apple's new design language for maximum impact.

And it worked. When iOS 26 launched in September, we were ready. Our updated iOS app shipped with both Liquid Glass and Buffer's refreshed brand aesthetic, delivering a polished, modern experience that feels native to the platform while staying distinctly Buffer.

What's next

Popcorn To Go is live and working, but we're just getting started. Here's what's on the roadmap:

Short-term:

• Applying to Android and refining based on feedback on both platforms.
• Expanding token coverage beyond colors (spacing scales, border radii, typography scales).
• Improving discoverability with better documentation.

Medium-term:

• Building out our custom component library with Buffer-specific patterns.
• Creating comprehensive usage guidelines for the system.
• Evolving with platform updates as iOS and Android continue to iterate.

Long-term:

• Keeping pace with platform evolution (iOS 27 and beyond, Material Design updates, etc.).
• Exploring opportunities to bring learnings back to our web design system, Popcorn.

Why it matters

For our designers and engineers, Popcorn To Go means smoother collaboration, faster prototyping, and less time spent on repetitive work. Instead of getting stuck on which colour to use where, teams can focus on solving more complex problems and crafting better experiences.

For Buffer users, it means more polished, consistent, and accessible apps. When design systems work well, users might not consciously notice — but they feel it. Interactions are smoother, the UI is more predictable, and everything just works better.

Raising the bar

Building Popcorn To Go wasn't just about solving today's problems but about setting ourselves up for the future.

Mobile platforms are constantly evolving. Design trends shift. User expectations rise. By investing in a solid foundation now, we're making it easier to keep pace, ship faster, and maintain quality as we grow.

This project was a true team effort: designers, iOS engineers, Android engineers, and product leaders all collaborating to make it happen. It's the kind of work that doesn't always get the spotlight, but it's what enables everything else we build.

We're proud of what we've created, and we're excited to keep building on it. If you want to see Popcorn To Go in action, download our iOS app and check out the new Liquid Glass experience.

Not on an Apple device? Keep an eye out, Popcorn To Go is coming to Android soon!

Here's to smoother collaboration, better apps, and a little more consistency in the chaos. 🍿

At Buffer, security has always been a balance: keeping our customers’ accounts safe while making login as seamless as possible for our global user base.

A few months ago, we made a decision that might sound surprising — we removed SMS-based two-factor authentication (2FA) and moved fully to email-based verification.

It wasn’t a change we took lightly. SMS has long been seen as the standard for 2FA. But over time, the drawbacks began to outweigh the benefits.

Here’s the story of how we got there, what the transition looked like, and what we’ve seen since.

Why we moved away from SMS

SMS-based 2FA has long been considered a security standard, but our team discovered several critical issues that made us reconsider:

Security vulnerabilities were more common than expected

SIM swapping attacks have become increasingly sophisticated, allowing attackers to hijack phone numbers and bypass SMS-based security.

Additionally, SMS messages travel unencrypted through multiple carriers, creating potential interception points.

Costs were scaling unsustainably

Every authentication SMS costs money, and with our growing user base, these seemingly small fees were adding up to hundreds of dollars monthly. International SMS rates made this even more challenging because our global user base.

International regulations and Sender ID requirements

SMS regulations vary dramatically by country, making compliance a constant challenge. Each country has different requirements for Sender IDs (the name that appears as the sender of an SMS), with some requiring pre-registration that can take weeks or months to complete.

For example, Singapore requires business verification documents, India demands a template pre-approval process, and the UAE has strict content restrictions.

Managing these requirements across 100+ countries created an enormous administrative burden that grew with each new regulation.

Additionally, failing to comply with any local regulation could result in messages being blocked, and ultimately customers being unable to log into Buffer.

Third-party dependencies created failure points

We relied on SMS gateway providers that occasionally experienced outages, delivery delays, or rate-limiting issues.

When these services go down, our users can not access their accounts—a critical problem for a tool that powers social media strategies worldwide.

Why email made more sense

When we looked for alternatives, we realized we already had a stronger option: email.

So instead of just removing SMS and calling it a day, we reimagined our authentication flow by incorporating email as another venue.

We implemented time-limited, single-use verification codes sent via email with enhanced security headers and encryption. Our email infrastructure, which we already maintained for notifications and updates, proved more reliable than third-party SMS gateways.

We also added rate limiting and anomaly detection to prevent abuse.

The unexpected benefits of switching to email

The transition delivered improvements beyond our initial expectations:

• Security actually improved. Email accounts typically have more robust security options than phone numbers, including their own 2FA, recovery options, and activity monitoring. Users maintain better control over their email accounts than their phone numbers, which can be transferred without their knowledge.
• Support tickets decreased. We saw a drop in authentication-related support requests. Users no longer struggled with international SMS delivery issues, changed phone numbers, or carrier-specific problems.
• Development velocity increased. Our engineering team no longer needs to maintain integrations with the SMS provider, debug delivery issues across different carriers, or handle country-specific SMS regulations.

How we rolled out the switch

Making this transition required careful planning.

We communicated the change to users well in advance, explaining the security benefits and addressing concerns. We provided detailed migration guides and temporarily supported both methods during the transition period.

For users who strongly preferred SMS, we helped them understand that modern email security, especially with providers like Gmail or Outlook that offer robust protection, provides equal or better security than SMS.

We also enhanced our email delivery infrastructure to ensure reliability, implementing redundant email service providers and monitoring delivery rates closely.

The right choice for Buffer

This decision won't be right for every company. Services that don't have users' email addresses or that serve demographics with limited email access might need different solutions. However, for Buffer — where every user already has an email account associated with their profile — this change aligned perfectly with our needs.

Three months after the transition, the results speak for themselves: a reduction in authentication-related support tickets, and significant monthly savings that we've reinvested in product improvements.

Looking ahead

Removing SMS authentication initially felt like swimming against the current, but it forced us to think critically about security theater versus actual security. Sometimes the "standard" solution isn't the best solution for your specific context.

We're continuing to explore additional authentication options, including support for hardware security keys. But our email-first approach has proven that simpler can indeed be more secure.

We share these kinds of stories because we know other teams face similar tradeoffs. Have you reconsidered a “standard” security practice recently? We’d love to hear from you on our social media! Find us @buffer everywhere and follow Carlos on LinkedIn here.

We’ve all experienced it at some point — a change is deployed to an API and suddenly, clients stop working.

User experience deteriorates, negative reviews start coming in, customer advocacy starts dealing with requests, multiple engineers start digging into the issue, and it quickly becomes all hands on deck.

Not only does this lose the trust of our users and interrupt their workflows, but it also costs an organization a lot of time and money to resolve these issues.

One way to prevent all of this from ever happening is to detect these breaking changes before they are merged into our repository at a Pull Request stage.

This way, we can prevent such changes from ever being merged, avoiding a breaking experience for our clients and reducing downtime for our users. 

As part of our commitment to transparency and building in public, I’m going to share how we’re doing this for our own GraphQL API via the use of GitHub Actions.

When it comes to detecting breaking changes, we can detect these by taking the schema representation on the branch of the pull request and comparing it with the schema representation on the main branch. We can then use the result of this diff to determine whether breaking changes exist in our schema changes.

When it comes to this workflow, we can break this down into several steps:

• Generate schema for the current branch: This will give us a schema that represents the changes we have made in our pull request
• Generate schema for the main branch: This will give us a schema that represents our current production API
• Perform verification of the current branch schema against the main branch schema: This will tell us what changes exist in our schema comparison and if any of them will break clients
• Post the result to the Pull Request: This will allow us to ‘fail’ the pull request to prevent it from being merged, along with alerting the author of the breaking changes

With this in mind, we’re going to build out an automated workflow that will run these operations for any Pull Request in our repository. For these pull request checks, we’re using GitHub Actions, but most of the following code will work for whatever CI setup you are using.

Note: We won’t be diving too much into the concepts of GitHub Actions here. If you are not familiar with Actions, I suggest following the quickstart tutorial.

Setting up the workflow

We’re going to start by setting up a new GitHub Action, we’ll create a new file named breaking_change_check.yml and start by giving our Action a name.

name: Schema Change Verification

Next, we’ll want to specify when this action is going to be run — this will essentially allow us to define when we want to perform the checks in the PR. 

We’ll not only want to do this on opened events (when the PR is initially opened), but also if it is reopened or synchronized — which will allow the checks the re-run if there are additional commits pushed to the pull request. 

This ensures that we are always checking for breaking changes on the latest commits pushed to the branch.

name: Schema Change Verification:  
on:
  pull_request:    
    types:      
      - opened      
      - reopened      
      - synchronize

We’ll also only want to run these checks when .graphql schema files are changed, so we’ll specify this rule using the paths property.

name: Schema Change Verification
on:
  pull_request:
    types:
      - opened
      - reopened
      - synchronize
    paths:
      - 'graphql/*/**.graphql'

graphql/*/**.graphql is the path where our GraphQl schema files are located; you will need to adjust this according to your project.

Generating the current branch schema

Now that we have the foundations of our action configured, we can move on to defining the jobs that will be responsible for generating and verifying our schema.

We’ll start here by defining a new job generateChangedSchema and defining the use of the ubuntu-latest runner.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    runs-on: [ubuntu-latest]

Next, we’ll need to perform a couple of setup operations for our job. We’ll want to start by checking out the repository at the branch of our PR, for which we’ll use the checkout action.

Our action is also going to utilize node, so we’ll need to install this using the setup-node action.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    runs-on: [ubuntu-latest]
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v3
        with:
          node-version: '18'

At this point, we’re now ready to move on to the generation of our schema. For this, we’re going to need to load our schema files and then merge them into a single schema file. This makes the verification process much simpler as we only need to work with a single file instead of multiple.

To make this process easier, we’re going to utilize a couple of dependencies from graphql-tools. We can see that these are named according to what we need them for, we just need to add them to our package.json file as a dev-dependency.

"@graphql-tools/load-files": "7.0.0",
"@graphql-tools/merge": "9.0.1"

With these dependencies in place, we’re now going to write a small script that will load and merge all of the .graphql files in a given directory.

import * as fs from 'fs'
import { loadFilesSync } from '@graphql-tools/load-files'
import { mergeTypeDefs } from '@graphql-tools/merge'
import { print } from 'graphql/language/printer'

const mergeFiles = async (): Promise<void> => {
  try {
    // using the provided path, load the types from the schema files
    const typesArray = loadFilesSync(`../../graphql/src`, {
      extensions: ['graphql'],
    })
 
    // merge all of the types from the recieved schemas, compressing all of the types     // into a single place
    const result = mergeTypeDefs(typesArray)
    // write the schema to a single file to be used for diffing
    await fs.promises.writeFile('generated/schema.graphql', print(result))
  } catch (e) {
    console.error("We've thrown! Whoops!", e)
  }
}

;(async (): Promise<void> => {
  try {
    // the merged schema file will be created in the generated directory, so create
    // the directory if it does not yet exist
    if (!fs.existsSync('generated')) {
      fs.mkdirSync('generated')
    }
    await mergeFiles()
  } catch (e) {
    console.error("We've thrown an error! Whoops!", e)
  }
})()

We’ll then add a new [task] to our package.json file so that we can easily execute this with the required arguments. This script takes a single argument when being executed, which is the path for the merged schema to be saved. 

If you need to provide paths for the location of schema files, this can be done through additional arguments. Our schemas are located in a single directory, so we hardcode this inside of the script itself.

"graph:generateSchema": "ts-node scripts/generateSchema.ts"

With this in place, we can now execute this command from our generateChangedSchema job. For this, we’ll use a bash script step where we’ll need to navigate to the directory where the generateSchema command exists and then use the node command to execute it.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    runs-on: [ubuntu-latest]
    steps:
      ...
      - name: Generate Schema
        run: |
          cd services/api-gateway
          node graph:generateSchema

At this point, we will have a merged schema file that contains the contents of all our schema files. To wrap up this job we’re going to attach this schema file to our workflow run, this is so that we can download that file for use within the next job in our workflow. 

While this could all be done in a single job, your actions can be kept far more organized if work is broken down into smaller chunks. Here, we’ll use the upload-artifact action and attach the schema file from the path that we saved it to, assigning it a name of branch-schema for referencing it later.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    runs-on: [ubuntu-latest]
    steps:
      ...
      - name: Attach schema
        uses: actions/upload-artifact@v1
        with:
          name: branch-schema
          path: services/api-gateway/generated/schema.graphql

At this point, we have created a merged schema file that represents all of the schemas in our project and attached this single file to our workflow, meaning that the step for generating the schema representation for the current branch is now complete.

Verify schema changes

Now that we have the schema for our branch generated, we’re going to want to verify this against the schema representation on the main branch. 

We’ll start here by setting up the second job in our workflow, performSchemaVerification. We’ll use the needs property to declare that this job will wait for the generateChangedSchema to complete successfully before running.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    runs-on: [ubuntu-latest]
    needs: generateChangedSchema

Similar to the previous job, we’ll configure some foundations for this job by checking out the repository and configuring node. The only difference here is that when triggering the checkout action we’ll pass a ref property with the value of main. 

This is because we need to generate the schema for the main branch for verification, so we need the main branch to be the current branch when checking out the repository.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    runs-on: [ubuntu-latest]
    needs: generateChangedSchema
    steps:
      - uses: actions/checkout@v2
        with:
          ref: main
      - uses: actions/setup-node@v3
        with:
          node-version: '18'

Before we can verify schemas, we’ll need to go ahead and download the schema that we generated in the last job. 

For this we can use the download-artifact action, providing the name reference for the file that we want to download (which we previously defined as branch-schema), followed by the path that the schema should be downloaded to.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    runs-on: [ubuntu-latest]
    needs: generateChangedSchema
    steps:
      ...
      - name: Download branch schema
        uses: actions/download-artifact@v1
        with:
          name: branch-schema
          path: services/api-gateway/branch-schema

And then so that we have a schema representation for our main branch to compare this to, we’ll go ahead and execute our generateSchema command. This will do the same as before, except this time, we will have a single schema file that represents our main branch instead of the branch for our pull request.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    runs-on: [ubuntu-latest]
    needs: generateChangedSchema
    steps:
      ...
      - name: Generate main schema
        run: |
          cd services/api-gateway
          node  graph:generateSchema

At this point, we have the two schema files that we need for the verification step. When it comes to the verification step, we’re going to utilize graphql-inspector. 

This tool contains a verification process that allows you to diff two schemas and returns you a result of the changes in that diff. We’ll start here by adding these as dev-dependencies to our project.

"@graphql-inspector/ci": "^4.0.2",
"@graphql-inspector/diff-command": "^4.0.2"

Next, we’ll add another command to our package.json file that we will use to execute this verification process. For this command we’ll use graphql-inspector and its diff command, for which we’ll need to provide two arguments. 

The first is the schema path for the main branch, which we just generated in this step. The second is the schema path for the PR branch, which we previously downloaded and saved to the branch-schema path. 

It’s important here that your main schema is passed as the first argument, as this represents your base schema, while the second represents the changes your branch is introducing.

"graph:verifySchema": "graphql-inspector diff generated/schema.graphql branch-schema/schema.graphql",

With this command in place, we’re now going to execute it in our step. We’ll need the result of this command so that we can depict the breaking change state, so we’ll save its output to a variable reference.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    runs-on: [ubuntu-latest]
    needs: generateChangedSchema
    steps:
      ...
      - name: Generate main schema
        run: |
          cd services/api-gateway
          node graph:generateSchema
          OUTPUT=$(node graph:verifySchema || true)

When it comes to the result of the diff operation, there is a lot of content that is output to the console. In the context of a PR comment, the author is only going to care about the information in the context of their changes. For this reason, we’re going to extract this information from the output so that we can use it for the PR comment.

The content that we wish to extract starts from “Detected N breaking changes,” so we’ll use this to extract the text from the diff output.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    runs-on: [ubuntu-latest]
    needs: generateChangedSchema
    steps:
      ...
      - name: Generate main schema
        run: |
          cd services/api-gateway
          node graph:generateSchema
          OUTPUT=$(node graph:verifySchema || true)
          // grab everything after the Detected string
          CONTENT=${OUTPUT#*Detected}
          // reapply the Detected string at the start of the contnet
          FORMATTED="Detected"$CONTENT
          // write the extracted content to an environment variable
          echo "PR_COMMENT<<EOF" >> $GITHUB_ENV
          echo "$FORMATTED" >> $GITHUB_ENV
          echo "EOF" >> $GITHUB_ENV

At this point we now have the message for the comment stored in an environment variable, this will look something like the following, depending on the result of the diff.

Detected the following changes (1) between schemas:
[log] ✖ Input field aNewType of type String! was added to input object type OrganizationIdInput
[error] Detected 1 breaking change

Now that we have this content, we’re going to want to publish it as a comment on the Pull Request. For this we’ll use the create-or-update-comment action, posting the content of the previously created environment variable.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    runs-on: [ubuntu-latest]
    needs: generateChangedSchema
    steps:
      ...
      - name: Create comment
        uses: peter-evans/create-or-update-comment@v1
        with:
          issue-number: ${{ github.event.pull_request.number }}
          body: ${{ env.PR_COMMENT }}
        if: ${{ !contains(env.PR_COMMENT, 'No changes detected') }}

Now that we have the status of our breaking change, we’re going to want to set the status of our PR check based on this. Here we’re going to add a new step to our job, but only run this step if the generated comment does not contain the ‘success’ label. 

This is because by default, the Action will have the success status, it will only be otherwise if we manually mark it as failed (or it fails for some other reason). When this is the case, we’ll utilise the github-script action to set the failed status for our check, along with a failure reason. 

This way, the check will fail within the Pull Request and the author will be unable to merge the Pull Request until the issue is resolved.

name: Schema Change Verification
...

jobs:
  generateChangedSchema:
    ...
  performSchemaVerification:
    steps:
      ...
      - name: Set Breaking Change status
        if: ${{ !contains(env.PR_COMMENT, 'success') }}
        uses: actions/github-script@v3
        with:
          script: |
            core.setFailed('Schema Breaking Changes detected')

Wrapping up

With all of the above in place, we will now be able to see the breaking change states published as comments on our Pull Request. For success states, engineers will be made aware of their schema changes, highlighting that no breaking changes were detected.

On the other hand, any breaking changes will be highlighted in the published comment.

When there is a breaking change, the check will be marked as a failure and the pull request will be unable to be merged.

Now that we have breaking change checks in place, engineers will be unable to merge changes that will break the client experience. This allows to reduce any downtime for our users, increasing the trust in our product and reducing business cost from incident management.

If you’re not already using breaking change checks in your CI, now is the time to get started! I’d love to hear about any learnings you have along the way to making this a part of your development workflow. Comment below or find me on LinkedIn. 

We recently launched a new feature at Buffer, called Ideas. With Ideas, you can store all your best ideas, tweak them until they’re ready, and drop them straight into your Buffer queue. Now that Ideas has launched in our web and mobile apps, we have some time to share some learnings from the development of this feature. In this blog post, we’ll dive into how we added support for URL highlighting to the Ideas Composer on Android, using Jetpack Compose.

We started adopting Jetpack Compose into our app in 2021 - using it as standard to build all our new features, while gradually adopting it into existing parts of our application. We built the whole of the Ideas feature using Jetpack Compose - so alongside faster feature development and greater predictability within the state of our UI, we had plenty of opportunities to further explore Compose and learn more about how to achieve certain requirements in our app.

Within the Ideas composer, we support dynamic link highlighting. This means that if you type a URL into the text area, then the link will be highlighted - tapping on this link will then show an “Open link” pop-up, which will launch the link in the browser when clicked.

In this blog post, we’re going to focus on the link highlighting implementation and how this can be achieved in Jetpack Compose using the TextField composable.

For the Ideas composer, we’re utilising the TextField composable to support text entry. This composable contains an argument, visualTransformation, which is used to apply visual changes to the entered text.

TextField(
    ...
    visualTransformation = ...
)

This argument requires a VisualTransformation implementation which is used to apply the visual transformation to the entered text. If we look at the source code for this interface, we’ll notice a filter function which takes the content of the TextField and returns a TransformedText reference that contains the modified text.

@Immutable
fun interface VisualTransformation {
    fun filter(text: AnnotatedString): TransformedText
}

When it comes to this modified text, we are required to provide the implementation that creates a new AnnotatedString reference with our applied changes. This changed content then gets bundled in the TransformedText type and returned back to the TextField for composition.

So that we can define and apply transformations to the content of our TextField, we need to start by creating a new implementation of the VisualTransformation interface for which we’ll create a new class, UrlTransformation. This class will implement the VisualTransformation argument, along with taking a single argument in the form of a Color. We define this argument so that we can pass a theme color reference to be applied within our logic, as we are going to be outside of composable scope and won’t have access to our composable theme.

class UrlTransformation(
    val color: Color
) : VisualTransformation {

}

With this class defined, we now need to implement the filter function from the VisualTransformation interface. Within this function we’re going to return an instance of the TransformedText class - we can jump into the source code for this class and see that there are two properties required when instantiating this class.

/**
 * The transformed text with offset offset mapping
 */
class TransformedText(
    /**
     * The transformed text
     */
    val text: AnnotatedString,

    /**
     * The map used for bidirectional offset mapping from original to transformed text.
     */
    val offsetMapping: OffsetMapping
)

Both of these arguments are required, so we’re going to need to provide a value for each when instantiating the TransformedText class.

• text - this will be the modified version of the text that is provided to the filter function
• offsetMapping - as per the documentation, this is the map used for bidirectional offset mapping from original to transformed text

class UrlTransformation(
    val color: Color
) : VisualTransformation {
    override fun filter(text: AnnotatedString): TransformedText {
        return TransformedText(
            ...,
            OffsetMapping.Identity
        )
    }
}

For the offsetMapping argument, we simply pass the OffsetMapping.Identity value - this is the predefined default value used for the OffsetMapping interface, used for when that can be used for the text transformation that does not change the character count. When it comes to the text argument we’ll need to write some logic that will take the current content, apply the highlighting and return it as a new AnnotatedString reference to be passed into our TransformedText reference. For this logic, we’re going to create a new function, buildAnnotatedStringWithUrlHighlighting. This is going to take two arguments - the text that is to be highlighted, along with the color to be used for the highlighting.

fun buildAnnotatedStringWithUrlHighlighting(
    text: String, 
    color: Color
): AnnotatedString {
    
}

From this function, we need to return an AnnotatedString reference, which we’ll create using buildAnnotatedString. Within this function, we’ll start by using the append operation to set the textual content of the AnnotatedString.

fun buildAnnotatedStringWithUrlHighlighting(
    text: String, 
    color: Color
): AnnotatedString {
    return buildAnnotatedString {
        append(text)
    }
}

Next, we’ll need to take the contents of our string and apply highlighting to any URLs that are present. Before we can do this, we need to identify the URLs in the string. URL detection might vary depending on the use case, so to keep things simple let’s write some example code that will find the URLs in a given piece of text. This code will take the given string and filter the URLs, providing a list of URL strings as the result.

text?.split("\\s+".toRegex())?.filter { word ->
    Patterns.WEB_URL.matcher(word).matches()
}

Now that we know what URLs are in the string, we’re going to need to apply highlighting to them. This is going to be in the form of an annotated string style, which is applied using the addStyle operation.

fun addStyle(style: SpanStyle, start: Int, end: Int)

When calling this function, we need to pass the SpanStyle that we wish to apply, along with the start and end index that this styling should be applied to. We’re going to start by calculating this start and end index  - to keep things simple, we’re going to assume there are only unique URLs in our string.

text?.split("\\s+".toRegex())?.filter { word ->
    Patterns.WEB_URL.matcher(word).matches()
}.forEach {
    val startIndex = text.indexOf(it)
    val endIndex = startIndex + it.length
}

Here we locate the start index by using the indexOf function, which will give us the starting index of the given URL. We’ll then use this start index and the length of the URL to calculate the end index. We can then pass these values to the corresponding arguments for the addStyle function.

text?.split("\\s+".toRegex())?.filter { word ->
    Patterns.WEB_URL.matcher(word).matches()
}.forEach {
    val startIndex = text.indexOf(it)
    val endIndex = startIndex + it.length
    addStyle(
        start = startIndex, 
        end = endIndex
    )
}

Next, we need to provide the SpanStyle that we want to be applied to the given index range. Here we want to simply highlight the text using the provided color, so we’ll pass the color value from our function arguments as the color argument for the SpanStyle function.

text?.split("\\s+".toRegex())?.filter { word ->
    Patterns.WEB_URL.matcher(word).matches()
}.forEach {
    val startIndex = text.indexOf(it)
    val endIndex = startIndex + it.length
    addStyle(
        style = SpanStyle(
            color = color
        ),
        start = startIndex, 
        end = endIndex
    )
}

With this in place, we now have a complete function that will take the provided text and highlight any URLs using the provided Color reference.

fun buildAnnotatedStringWithUrlHighlighting(
    text: String, 
    color: Color
): AnnotatedString {
    return buildAnnotatedString {
        append(text)
        text?.split("\\s+".toRegex())?.filter { word ->
            Patterns.WEB_URL.matcher(word).matches()
        }.forEach {
            val startIndex = text.indexOf(it)
            val endIndex = startIndex + it.length
            addStyle(
                style = SpanStyle(
                    color = color,
                    textDecoration = TextDecoration.None
                ),
                start = startIndex, end = endIndex
            )
        }
    }
}

We’ll then need to hop back into our UrlTransformation class and pass the result of the buildAnnotatedStringWithUrlHighlighting function call for the TransformedText argument.

class UrlTransformation(
    val color: Color
) : VisualTransformation {
    override fun filter(text: AnnotatedString): TransformedText {
        return TransformedText(
            buildAnnotatedStringWithUrlHighlighting(text, color),
            OffsetMapping.Identity
        )
    }
}

Now that our UrlTransformation implementation is complete, we can instantiate this and pass the reference for the visualTransformation  argument of the TextField composable. Here we are using the desired color from our MaterialTheme reference, which will be used when highlighting the URLs in our TextField content.

TextField(
    ...
    visualTransformation = UrlTransformation(
        MaterialTheme.colors.secondary)
)

With the above in place, we now have dynamic URL highlighting support within our TextField composable. This means that now whenever the user inserts a URL into the composer for an Idea, we identify this as a URL by highlighting it using a the secondary color from our theme.

In this post, we’ve learnt how we can apply dynamic URL highlighting to the contents of a TextField composable. In the next post, we’ll explore how we added the “Open link” pop-up when a URL is tapped within the composer input area.

At Buffer, we’ve been working on a better admin dashboard for our customer advocacy team. This admin dashboard included a much more powerful search functionality. Nearing the end of the project’s timeline, we’ve been prompted with the replacement of managed Elasticsearch on AWS with managed Opensearch. Our project has been built on top of newer versions of the elasticsearch client which suddenly didn’t support Opensearch.

To add more fuel to the fire, OpenSearch clients for the languages we use, did not yet support transparent AWS Sigv4 signatures. AWS Sigv4 signing is a requirement to authenticate to the OpenSearch cluster using AWS credentials.

This meant that the path forward was riddled with one of these options

• Leave our search cluster open to the world without authentication, then it would work with the OpenSearch client. Needless to say, this is a huge NO GO for obvious reasons.
• Refactor our code to send raw HTTP requests and implement the AWS Sigv4 mechanism ourselves on these requests. This is infeasible, and we wouldn’t want to reinvent a client library ourselves!
• Build a plugin/middleware for the client that implements AWS Sigv4 signing. This would work at first, but Buffer is not a big team and with constant service upgrades, this is not something we can reliably maintain.
• Switch our infrastructure to use an elasticsearch cluster hosted on Elastic’s cloud. This entailed a huge amount of effort as we examined Elastic’s Terms of Service, pricing, requirements for a secure networking setup and other time-expensive measures.

It seemed like this project was stuck in it for the long haul! Or was it?

Looking at the situation, here are the constants we can’t feasibly change.

• We can’t use the elasticsearch client anymore.
• Switching to the OpenSearch client would work if the cluster was open and required no authentication.
• We can’t leave the OpenSearch cluster open to the world for obvious reasons.

Wouldn’t it be nice if the OpenSearch cluster was open ONLY to the applications that need it?

If this can be accomplished, then those applications would be able to connect to the cluster without authentication allowing them to use the existing OpenSearch client, but for everything else, the cluster would be unreachable.
With that end goal in mind, we architected the following solution.

Piggybacking off our recent migration from self-managed Kubernetes to Amazon EKS

We recently migrated our computational infrastructure from a self-managed Kubernetes cluster to another cluster that’s managed by Amazon EKS.
With this migration, we exchanged our container networking interface (CNI) from flannel to VPC CNI. This entails that we eliminated the overlay/underlay networks split and that all our pods were now getting VPC routable IP addresses.
This will become more relevant going forward.

Block cluster access from the outside world

We created an OpenSearch cluster in a private VPC (no internet-facing IP addresses). This means the cluster’s IP addresses would not be reachable over the internet but only to internal VPC routable IP addresses.
We added three security groups to the cluster to control which VPC IP addresses are allowed to reach the cluster.

Build automations to control what is allowed to access the cluster

We built two automations running as AWS lambdas.

• Security Group Manager: This automation can execute two processes on-demand.
• -> Add an IP address to one of those three security groups (the one with the least number of rules at the time of addition).
• -> Remove an IP address everywhere it appears in those three security groups.
• Pod Lifecycle Auditor: This automation runs on schedule and we’ll get to what it does in a moment.

How it all connects together

We added an InitContainer to all pods needing access to the OpenSearch cluster that, on-start, will execute the Security Group Manager automation and ask it to add the pod’s IP address to one of the security groups. This allows it to reach the OpenSearch cluster.
In real life, things happen and pods get killed and they get new IP addresses.Therefore, on schedule, the Pod Lifecycle Auditor runs and checks all the whitelisted IP addresses in the three security groups that enable access to cluster. It then checks which IP addresses should not be there and reconciles the security groups by asking the Security Group Manager to remove those IP addresses.
Here is a diagram of how it all connects together

Extra Gotchas

Why did we create three security groups to manage access to the OpenSearch cluster?

Because security groups have a maximum limit of 50 ingress/egress rules. We anticipate that we won’t have more than 70-90 pods at any given time needing access to the cluster. Having three security groups sets the limit at 150 rules which feels like a safe spot for us to start with.

Do I need to host the Opensearch cluster in the same VPC as the EKS cluster?

It depends on your networking setup! If your VPC has private subnets with NAT gateways, then you can host it in any VPC you like. If you don’t have private subnets, you need to host both clusters in the same VPC because VPC CNI by default NATs VPC-external pod traffic to the hosting node’s IP address which invalidates this solution. If you turn off the NAT configuration, then your pods can’t reach the internet which is a bigger problem.

If a pod gets stuck in CrashLoopBackoff state, won’t the huge volume of restarts exhaust the 150 rules limit?

No, because container crashes within a pod get restarted with the same IP address within the same pod. The IP Address isn’t changed.

Aren’t those automations a single-point-of-failure?

Yes they are, which is why it’s important to approach them with an SRE mindset. Adequate monitoring of these automations mixed with rolling deployments is crucial to having reliability here. Ever since these automations were instated, they’ve been very stable and we didn’t get any incidents. However, I sleep easy at night knowing that if one of them breaks for any reason I’ll get notified way before it becomes a noticeable problem.

Conclusion

I acknowledge that this solution isn’t perfect but it was the quickest and easiest solution to implement without requiring continuous maintenance and without delving into the process of on-boarding a new cloud provider.

Over to you

What do you think of the approach we adopted here? Have you encountered similar situations in your organization? Send us a tweet!

At Buffer, we’re constantly experimenting with ways we can improve our products and try out new ideas. We recently launched Start Page, a beautiful, flexible, mobile-friendly landing page that you can build in minutes and update in seconds. As a Software Engineer on Buffer’s team I’ve tackled a long list of fun projects, including Start Page. One thing I love about this project, is that as we foray deeper and deeper into user-generated content and customization, we’re discovering new engineering challenges that we haven’t had in our frontends before. In this case, we wanted to introduce 13 new font options (for a total of 16 fonts) and we wanted to make sure that they loaded nice and quickly. As I worked on this, I learned so much I didn’t know about fonts so in this post I want to share more about how we went about this for anyone facing similar challenges.

Fonts are render-blocking

Let’s start with the ‘why’. Fonts are generally pretty light resources, which are usually cached in browser so why is it important to ensure a quick loading strategy? Because fonts are high-priority, synchronous requests which means they’re render-blocking. If we can load fonts quickly and/or asynchronously, we can improve site speed.

FOUT and FOIT

Ok, so you don’t want to block your rendering, there are generally two strategies to chose from to handle text loaded before it’s custom font:

FOUT - Flash Of Unstyled Text
Renders the text but with a fallback font. Google Fonts can now return with display=swap which instructs the browser to use the fallback font to display the text until the custom font has fully downloaded. If you want to be meticulous, you can find a better fallback font using this app: Font Style Matcher

FOIT - Flash Of Invisible Text
Here, the text is rendered with an invisible font until the custom font has fully downloaded. This one makes more sense to use for something like a logo where the brand would be affected if rendered with a fallback font (although for a logo I’d use an SVG but examples!)

THE trick for fast fonts

The general advice nowadays is to preconnect to the font server:

<link rel="preconnect" href="https://fonts.gstatic.com/" crossorigin />
<link rel="preconnect" href="https://fonts.googleapis.com" />

then preload the fonts:

  <link
      rel="preload"
      as="style"
      href="https://fonts.googleapis.com/css2?family={your font families here}&display=swap"
    />

Finally as a fallback, request the fonts async by setting media to “print” for browsers which don’t support rel="preload" (about 12% of browsers in this the year 2021)

<link
      rel="stylesheet"
      href="https://fonts.googleapis.com/css2?family={your font families here}&display=swap"
      media="print"
      onload="this.media='all'"
    />

This works because a regular stylesheet is render-blocking but a print stylesheet is assigned idle priority. After it’s loaded, the link’s media is applied to all.

Hosting your own fonts is the fastest but Google Fonts does a lot for you:

• Returns multiple alphabets
• Returns a css file customized to the user agent that requested it
• When you have multiple fonts, it’s best to make 1 request so it's quicker
• You can tailor your requests to target specific font-weights and formats (bold, italic, thin)

Font Loading API

There’s a new-ish CSS Font Loading API that can request fonts on demand but I found that this doesn’t play nice with Google Fonts because you need the source URL for the fonts and the Google Fonts URL that you get isn’t the source, it’s the request. Google, along with Typekit, does have a library called Web Font Loader, that works like the Font Loading API but plays better with Google Fonts.

So what did we do in Start Page?

We implemented the popular strategy for the builder (the app itself) and while we do have some FOUT on first load ever (remember browser caching!) it’s very minimal, if seen at all. For generated pages, we get the fonts used in the theme before generating the HTML so we can inject only the fonts we need. This makes our generated pages much faster and lighter.We’re excited to see how this experiment will play out and if folks are keen to get more font options. If that’s the case, we might very well look into a more dynamic strategy (like loading only the currently used fonts on load and then sending another request if a user clicks on Appearance to change their fonts). Another option we could look into is implementing a way for requesting multiple fonts if we hosted them ourselves.

That’s it for now! Thanks for making it this far, I hope this was interesting for you! Know anything neat about fonts that I didn’t mention here? Share it with us on Twitter.

Resources:
The Fastest Google Fonts
Loading Google Fonts and any other web fonts as fast as possible in early 2021
FOIT vs FOUT: a comparison on web font loading
CSS Tricks - font-display

Header Photo by Pearse O’Halloran on Unsplash

For our Android clients we have a small component library which is used to shared common visual elements across the different Android applications that we work on. We recently updated our applications to the Material Components library, meaning that our component library itself needed to go through the same transition.

Within this component library we have a custom button component – the button is styled to suit the design system at buffer and having this as a custom component allows us to easily toggle between the different custom attributes that it provides. The button comes in three different states:

In order to achieve these states there was actually quite a bit of code – each button has it’s own selector state for both the text color and background, meaning that we ended up with something like this in our resources:

When it comes to these buttons and their corresponding files, each text selector defines the text color states for each button type:

We then have a similar selector but for the background of each button type. Finally, each of these button states then has a shape drawable to create the rounded corner background for the button.

Whilst this works… a) there’s a lot of code here for something that in appearance doesn’t look so complex and b) this isn’t quite going to work when we migrate to use the MaterialButton component.

When it comes to the MaterialButton component, the way we style the button can become a little different. Where we previously had all of the different background selectors that defined not only the background color but the shape, we can now achieve this in a centralised style.

To begin with, this means that we can begin by changing our background selector so that some simple color references are used. If we want to do this in XML, then we can do this the following way:

This selector works the same way as the previous, however now we’re not dealing with the different shapes within our xml file – meaning that we end up with a lot less files to represent how our button looks:

For each of these color / background selectors we can then set them within our custom MaterialButton component. We set these depending on the custom attribute passed:

setTextColor(ContextCompat.getColorStateList(context,
        R.color.selector_light_button_text))
backgroundTintList = ContextCompat.getColorStateList(context,
        R.color.selector_light_button_background)

So that’s the text color and background color sorted, but what about the styling of the button? We previously had all of those different background selectors so that we could modify the shape, however, the styling for the MaterialButton allows us to achieve this directly through the Widget.MaterialComponents.Button style.

To make use of this we’re going to create our own style, RoundedButtonStyle, and define some properties for the shapeAppearance attribute:

You’ll notice here that for each corner we are defining a cornerFamily attribute, this defines how the corner of the button is rendered – currently this can be either cut or rounded. We won’t get too much into this here, but rounded gives us the same rounded corner effect that we previously had in place – just with much less code. We then use the cornerSize attribute to define the radius used for the rounded corner.

Once we set the style for our button, our button has the intended look and feel, now with the consistency of the rest of our application with the use of the material component library.

In this post we’ve taken a quick look at how we can take our existing Button styling and convert it to a Material Button component. Now with this in place, we have greater consistency throughout our app when it comes to buttons ? Have you worked with something similar recently or looking to make the jump to material components also? Feel free to reach out in the comments below if so ?

Header Photo by Icons8 Team on Unsplash

Modularizing your Android projects can bring a number of different advantages to your team. Some of these include reduced build times, a greater separation of concerns and the ability to reuse components throughout our applications. As we started to get more and more modules in our projects, I started to think more about how these were being run on our CI server. For example, we open a pull request and for that changed code all of our tests and checks are run for the entire project. When you have a couple of modules you probably won’t see a concern here. But what if we have 30 modules, each with plenty of code / tests, and we open a pull request that only makes changes to one of those modules? In this article I want to share how we’ve made some additions to our CI to help here!

We’ve been trying to reduce our CI times recently, so with our modularisation this seemed like a good place to start looking. We have unit tests in every feature module in our application and we have around 20 modules currently. Whilst unit tests don’t take too long to run, being able to shave some time off of each build that occurs will add up over the days, weeks and months that our CI is building our tasks. With restricted concurrent builds on our current CI plan, that saved time helps to free up our CI server quicker, keeping us more productive in our work.

Unfortunately, there’s no magic way to detect what modules have changes to them and only run the tests for those modules. In Android we can either run a gradle test task from the root of our project, or individually for each of the modules in our project. Even on our CI server (bitrise) the test step takes a single test command, which by default uses the test task from the root of the project. When it comes to running unit tests via gradle, we can however provide a list of test commands to run during our test task, for example:

./gradlew :moduleA:testDebugUnitTest :moduleB:testDebugUnitTest

That would solve all of our problems when it comes to running our unit tests, but how do we get there? There are a couple of things that we need to do in order to build our test commands dynamically.

We need to begin by detecting the modules in our code that have changed files in them. Again, there’s no straightforward way to detect this from within the CI server – so we’re going to need to perform some git diffing and calculate the changed modules using those diffs. This is going to look something like so:

dest=origin/_branch_merging_into_
branch=origin/_branch_for_pull_request_

changed_modules=""

git diff --name-only $dest..$branch | { while read line
    do
      module_name=${line%%/*}

      if [[ ${module_name} != "buildSrc" && 
            ${changed_modules} != *"$module_name"* ]]; then 
              changed_modules="${test_modules} ${module_name}"
      fi
    done
}

We need to start by retrieving the destination that our branch is being merged into, along with the actual branch for the pull request that has been opened. You can’t hardcode these as everytime you open a pull request this code is going to be run. On bitrise you can access environment variables to get these values:

dest=origin/$BITRISEIO_GIT_BRANCH_DEST
branch=origin/$BITRISE_GIT_BRANCH

Next we’re going to perform the git diff operation against these two branches. From the code above, the section below takes our two branches and loops through each line that is presented in the diff. However, we don’t care for the actual diff content, we only want the names of the files that have changes. Using the –name-only command when performing the diff means that we be presented only with the file names, instead of the file diff content.

git diff --name-only $dest..$branch | { while read line
    do

    done
}

Now that we have our changed files, we need to pull out the module name from each of them. In the line below, we pull out the string content up until the first forward slash character.

module_name=${line%%/*}

To note, this isn’t a sure fire way of getting the module name as it can yield unexpected values. For example, if we change a gradle or text file in the root of our project which isn’t within a module, then this module_name variable could be assigned with something that doesn’t represent a module. The same goes for modules that we have deleted – whilst these would appear in the diff, we wouldn’t want to run the tests for them as the module no longer exists. The next script that we write will handle this, for now we just want to get a list of everything that has changed. This way, we can also re-use this script if we decide to selectively run other things on our CI server.

The last piece of code in our script will be used to build our list of changed module names. So for each module_name variable we’re going to add it to our changed_modules variable, in the end this will result in a single string representing separated module names.

You may notice that this is all wrapped in an if statement – this checks as to whether  the changed_modules already contains the current module_name and if so, we don’t want to re-add it to our changed_modules variable (otherwise we will end up with duplicates!).

if [[ ${changed_modules} != *"$module_name"* ]]; then 
      
      changed_modules="${changed_modules} ${module_name}"

fi

At this point we have a list of module names in a single string. Depending on your CI service, you may need to pass this to another script step. In the case of bitrise, you can write this value to an environment variable to re-use in other script steps for build module specific commands:

envman add --key CHANGED_MODULES --value "${changed_modules}"

From the above operations we’re going to now have a string that represents all of the different module names in our application. This might look something like:

moduleA moduleB moduleC

Now that we have a collection of these module names, we need to go ahead and build the test task using those names so that we can run the unit tests for that module. For this we’re going to need to take each one of the module names from our string and create the test command for each one. For running our unit tests we’re going to want to end up with a string that looks like:

:moduleA:testDebugUnitTest :moduleB:testDebugUnitTest ...

However, as previously mentioned, it might be the case that some module names that we’ve acquired aren’t actually modules within our application. For example, if you’ve edited the build.gradle / gradle.properties files, your buildSrc module or even deleted moduleA from your application, then these will all be contained within your module name string. For this reason, before we build our test command string we need to filter out anything that doesn’t support us running unit tests for the module within our application. The code to achieve this looks like so:

AVAILABLE_TASKS=$(./gradlew tasks --all)
modules=$CHANGED_MODULES

test_commands=""

for module in $commands
do 
    if [[ $AVAILABLE_TASKS =~ $module":" ]]; then 
        test_commands=
            ${test_commands}" :"${module}":testDebugUnitTest"
    fi
done

if [[ $test_commands == "" ]]; then
    test_commands="test"
fi

envman add --key UNIT_TEST_COMMANDS --value "${test_commands}"

We begin by retrieving all of the available gradle tasks within our application:

AVAILABLE_TASKS=$(./gradlew tasks --all)

Whilst this doesn’t provide us with the actual commands for running tests, it does tell us the modules that can have commands run against them. For example:

• If we make changes to the buildSrc module, this has no gradle tasks to run against it that will come back from our tasks command

• A deleted module would not show any gradle commands that can be run for it

• If root files have been edited (root build.gradle, gradle.properties) that are not in a module, these names will not match any modules and their commands

With the tasks –all command we can retrieve a collection of modules and check our module names against them. With this collection of commands we can now take our generated module names from the last script and check these agains the available commands. We’ll begin by retrieving this module names from the environment variable that we saved them to:

modules=$CHANGED_MODULES

Next we need to check whether our available tasks contains a reference to the modules within our module names string. For this we loop through each of the module names in our string and verify that the module name is supported for our needs:

for module in $modules
do 
    if [[ $AVAILABLE_TASKS =~ $module":" ]]; then 
        test_commands=
            ${test_commands}" :"${module}":testDebugUnitTest"
    fi
done

If you run the tasks –all command then you’ll see something like the following:

moduleA:someTask moduleA: anotherTask moduleB:someTask ...

And in our script above we have the following line:

if [[ $AVAILABLE_TASKS =~ $module":" ]];

Here we are taking our module name, appending it with a colon and asserting whether our variable containing the tasks has a reference to this string value. If so, we can presume that the module supports the unit test command that we want to run. If so, then we append our test_commands variable with the command to run the unit tests for our module.

test_commands=
            ${test_commands}" :"${module}":testDebugUnitTest"

If for some reason out test_commands variable is empty, either something has gone wrong or no modules have been changed – maybe only the build.gradle file has only been changed in the current PR. Here you can either not run any tests, or have a safeguard in place that will just run all of the tests for the project.  This can be done by assigning the “test” string value to our test_commands variable.

if [[ $test_commands == "" ]]; then
    test_commands="test"
fi

With the above done we should now have a string variable that looks something like so:

:moduleA:testDebugUnitTest :moduleB:testDebugUnitTest

This is great! Now we have  a collection of the commands that need to be run for our unit test task. The only thing left to do is to save this to an environment variable so that our CI can use it within the unit test task.

envman add --key UNIT_TEST_COMMANDS --value "${test_commands}"

This next part will really depend on the service you are using for your CI. For us we are using Bitrise, Bitrise provides a Gradle Unit Test step which is used to run the unit tests in a project. This step tasks a Test task input variable – this is where we are now gong to pass a reference to our UNIT_TEST_COMMANDS variable

Now when our unit tests are run, only the unit tests for the changed modules are run. This will help us to shave some times off our builds, allowing us to be more productive and efficient when building our products!

With all of the above you are able to put something in place that allows you to selectively run unit tests for your modularised android project. Even if the above isn’t exactly what you’re looking to put in place, you may be able to use some of the module specific scripting for something else within your CI server.

Are you already using scripting for these kind of things, or looking to put something in place? Feel free to reach out and I’ll be happy to chat over any of these things!

All right folks, I intend to keep this one short and that’s what I will do. I mean, it’s supposed to be easy but the official documentation(1, 2) makes it unnecessarily confusing. So I think maybe I can help to fill in the gap.

I will be using one of our business requirements at Buffer in this project, as an example for this blog post.

Quick recap

So, we need a few nodes that are dedicated to running cronjobs, and nothing else. At the same time, we want to make sure the cornjobs are scheduled to these nodes, and nowhere else. This means we need 2 things

• Tainted nodes that don’t take other workloads
• The workload that only goes to the destination nodes

Now, let’s start from nodes, then the workload

Nodes

Since the requirement is broken down to 2 aspects (see above), there are 2 things we will need to specify for nodes. As always, kops is my weapon of choice.

In kops, you can do this kops edit ig <INSTANCE GROUP IN INTEREST>

apiVersion: kops.k8s.io/v1alpha2
kind: InstanceGroup
metadata:
  labels:
    kops.k8s.io/cluster: steven.k8s.com
  name: frequent-cronjob-nodes
spec:
  image: kope.io/k8s-1.13-debian-stretch
  machineType: m4.xlarge
  maxSize: 2
  minSize: 2
  nodeLabels:
    kops.k8s.io/instancegroup: frequent-cronjob-nodes
  role: Node
  subnets:
  - us-east-1b
  - us-east-1c
  taints:
  - dedicated=frequent-cronjob-nodes:NoSchedule

Tainting nodes

This prevents other workloads from being scheduled to them. It’s achieved by these 2 lines

taints: 
- dedicated=frequent-cronjob-nodes:NoSchedule

Labeling nodes

This helps a specialized workload to locate the nodes. It’s achieved by these 2 lines

nodeLabels:   
kops.k8s.io/instancegroup: frequent-cronjob-nodes

I know there are people who don’t use kops out there. If you are one of them, here are 2 commands to help

kubectl taint nodes <NODE IN INTEREST> dedicated=frequent-cronjob-nodes:NoSchedule

kubectl label nodes <NODE IN INTEREST> kops.k8s.io/instancegroup=frequent-cronjob-node

Workload

Similar to nodes, we will need to do 2 things to the deployment/cronjob yaml file. I’m including a complete yaml to save our eyes from this (yeah, you know what I’m talking about).

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  namespace: dev
  name: steven-cron
  labels:
    app: steven-cron
spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          nodeSelector:
            kops.k8s.io/instancegroup: frequent-cronjob-nodes
          tolerations:
          - key: dedicated
            value: frequent-cronjob-nodes
            operator: "Equal"
            effect: NoSchedule
          containers:
          - name: steven-cron
            image: buffer/steven-cron
            command: ["php", "./src/Crons/index.php"]
          imagePullSecrets:
            - name: buffer

Tolerating taints

This makes sure the workload can be scheduled to the tainted nodes. It’s achieved by these lines

tolerations: 
- key: dedicated   
  value: frequent-cronjob-nodes   
  operator: "Equal"   
  effect: NoSchedule

Specifying destination nodes

This makes sure the workload is only to be scheduled to the specified nodes. It’s achieved by these 2 lines

nodeSelector:   
  kops.k8s.io/instancegroup: frequent-cronjob-nodes

Profit

This is it. We can now rest assure the right workload will be going to the right nodes. In this way, we can start building some specialized node groups for specialized workloads, say GPU nodes for machine learning or memory-intensive nodes for local caching.

I hope this helps in any way. Until next time, please feel free to hit me up on Twitter should you have any questions. ?

So, it all started on September 1st, right after our cluster upgrade from 1.11 to 1.12. Almost on the next day, we began to see alerts on kubelet reported by Datadog. On some days we would get a few (3 – 5) of them, other days we would get more than 10 in a single day. The alert monitor is based on a Datadog check – kubernetes.kubelet.check, and it’s triggered whenever the kubelet process is down in a node.

We know kubelet plays an important role in Kubernetes scheduling. Not having it running properly in a node would directly remove that node from a functional cluster. Having more nodes with problematic kubelet then we get a cluster degradation. Now, Imagine waking up to 16 alerts in the morning. It was absolutely terrifying.

What really puzzled us was all the services running on the problematic nodes seemed to be innocuous. In some cases, there were only a handful of running services, and some high CPU usage right before. It was extremely hard to point the finger on anything when the potential offender might have left the scene, thus leaving no trace for us to diagnose further. Funny enough, there weren’t any obvious performance impact such as request latency across our services. This little fact added even more mystery to the whole thing.

This phenomenon continued to start around the same time every day (5:30AM PT), and usually stopped before noon, except for the weekends. To a point, I felt I could use these Datadog alerts for my alarm clock. Not super fun, and I certainly got some grey hair with this challenge.

Our investigation

From the start, we knew this was going to be a tough investigation that would require a systematic approach. For brevity, I’m going to just list out some key experiments we attempted and spare you from the details. As much as they are good investigative steps, I don’t believe they are important for this post. Here are what we tried

• We upgraded the cluster from 1.12 to 1.13
• We created some tainted nodes and moved all our cronjobs to them
• We created more tainted nodes and moved most CPU consuming workers to them
• We scaled up the cluster by almost 20%, from 42 nodes to 50 nodes
• We scaled down the cluster again because we didn’t see any improvements
• We recycled (delete and recreate) all the nodes that previously reported kubelet issue, only to see new nodes followed suit on the next day
• Just between you and me, I even theorized the Datadog alert might be broken because there wasn’t any obvious service performance impact. But I couldn’t bring myself to close the case knowing the culprit might still be at large.

With a stroke of luck and a lot of witch-hunting, this piqued my attention

We saw 10 buffer-publish pods were scheduled to a single node for around 10 minutes, only to be terminated shortly. At the same time the CPU usage spiked, kubelet cried out, and the pods disappeared from the node in the next few minutes after termination.

No wonder we could never find anything after alerts. But what were so special about these pods, I thought? The only fact we had was the high CPU usage. Now, let’s take a look at the resource requests/limits

resources:
  limits:
    cpu: 1000m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 50Mi

CPU/Memory requests parameter tells Kubenetes how much resource should be allocated initially

CPU/Memory limits parameter tells Kubenetes the max resource should be given under all circumstances

Here is a post that does a much better job in explaining this concept. I highly recommend reading it in full. Kudos to the team at kubecost!

Now, back to where we are. The CPU requests/limits ratio is 10, and it should be fine, right? We allocate 0.1 CPU to a pod in the beginning and limit the max usage to 1 CPU. In this way, we have a conservative start while still having some kind of, although arbitrary upper boundary. It almost feels like we are following the best practice!

Then I thought, this doesn’t make any sense at all. When 10 pods are scheduled in a single node the total CPU this parameter would allow for is 10 CPUs, but there aren’t 10 CPUs in a m4.xlarge node. What would happen during our peak-hours, say 5:30AM PT when America wakes up? Now I can almost visualize a grim picture of these node killing pods taking all CPU, to a point that even kubelet starts to die off, then the whole node just crash and burn.

So now, what we can do about it?

The remedy

Obviously the easiest way is to lower the CPU limits so these pods will kill themselves before they kill a node. But this doesn’t feel quite right to me. What if they really need that much CPU for normal operations, so throttling ( more on this) doesn’t lead to low performance.

Okay, how about increasing the CPU requests so these pods are more spread out and don’t get scheduled into a single node. That sounds like a better plan, and that was the plan we implemented. Here are the details:

Figure out how much you typically need

I used the Datadog metric kubernetes.cpu.usage.total over the past week on the max reported value to give me some point of reference

You could see in general it stays below 200m (0.2 CPU). This tells me it’s hard to go wrong with this value for CPU requests.

Put a limit on it

Now, this was the tricky part, and like most tricky things in life, there isn’t a simple solution. In my experiences, a good start would be 2x of the requests. In this case, it would be 400m (0.4 CPU). After the change, I spent some time eyeballing the service performance metrics to make sure the performance wasn’t impacted by CPU throttling. Chances are if it were, I would need to up it to a more reasonable number. This is more of an iterative process until you get right.

Pay attention to the ratio

It’s key not to have low requests tricking Kubernetes into scheduling all pods into one node, only to exhaust all CPU with incredibly high limits. Ideally, the requests/limits should not be too far away from each other, say within 2x to 5x range. Otherwise, an application is considered to be too spiky, or even has some kind of leaks. If this is the case, it’s prudent to get to the bottom of the application footprints.

Review regularly

Applications will undergo changes as long as they are active, so will their footprints. Make sure you have some kind of review process that takes you back to Step 1 (Figure out how much it typically needs). This is the only way to keep things in tip-top shape.

Profit

So, did it work? You bet! There were quite a few services in our cluster with disproportional requests/limits. After I adjusted these heavy-duty services, the cluster runs with more stability Here is how it looks now ?

Wait! How about efficiency promised in the title? Please note the band has gotten more constricted after the changes. This shows the CPU resource across the cluster is being utilized more uniformly. This subsequently makes scaling up to have a linear effect, which is a lot more effective.

Closing words

In contrast with deploying each service on a set of dedicated computing instances, service-oriented architecture allows many services to share a single Kubernetes cluster. Precisely because of this, each service now bears the responsibility of specifying its own resource requirements. And this step is not to be taken lightly. An unstable cluster affects all the residing services, and troubleshooting is often challenging. Admittedly, not all of us are experienced with this kind of new configurations. In the good ol’ days all we needed was to deploy our one thing on some servers, and scale up/down to our liking. I think this might be why I don’t see a lot of discussions around the resource parameters in Kubernetes. Through this post, it’s my hope to help a few people out there who are struggling with this new concept (I know I did). More importantly, perhaps learn from someone who has some other techniques. If you have any thoughts on this, please feel free to hit me up on Twitter.

Originally published at http://github.com.

Today we’re happy to announce our new engineering podcast, The Buffer Overflow Podcast. And, our first episode is now available for streaming!

How to Listen

The Buffer Overflow Podcast is now available on all major platforms today, so if you open your podcast player of choice and search for us, we should be there! Here are some direct links:

• Google Podcasts
• Spotify
• Breaker
• Overcast
• Pocketcasts
• RadioPublic
• RSS feed

So, Why Start a Podcast?

It’s built into our D.N.A. here at Buffer to try and be transparent about the things we know, are learning or even have struggled with as engineers. Those are things we want to talk about more openly, and that’s one of the goals of our new podcast.

We plan to talk about topics common to engineering (how we do plan application architecture), to operational things (how do we plan the same features across many platforms) or even remote work topics (how to structure your day).

To kick things off, Joe and myself  will be hosting the beginning episodes as we get things off the ground. Expect to hear from more members across our engineering team soon, though!

We hope you find some value from our new podcast, and thanks for listening!

Android 10 is officially here! On 3rd September it began rolling out for pixel devices, so we wanted to be sure that our app was ready to serve our users who would have Android 10 installed on their device. When these OS updates come around, as developers we sometimes don’t know how many changes we’re going to need to make to our applications. When Android Marshmallow was released, permissions changes were quite a big thing for many applications. As we move our way through versions up to Android Pie we’ve also seen restrictions in various forms for permissions as a whole, along with other changes to restrictions on the size of data passed through intents and various media focused changes. When it comes to Android 10, there are a collection of changes (as outlined on the developer site) that could potentially affect your app – these changes also include some enhancements that are also available in this latest release.

Whilst we we haven’t had to make too many changes for our app, in this post we’ll share the things we did to prepare our app for the Android 10 release and how we adapted some of the new features into our application.

Updating versions

Other than updating the targetSdkVersion of your application to 29, you’re going to want to update a couple of android related dependencies that will improve the release of your application for Android 10.

To begin with, you may have seen that Android 10 features the ability to enable gesture navigation – this removes the software buttons at the bottom of the device, allowing the user to rely on gestural navigation to move through applications. Whilst this is something that is not enabled by default, we wanted to make sure that our app behaved as expected when this gestural mode was enabled. In order to get the best experience with gestural navigation you’ll want to update to the latest version of the androidx drawlayout dependency. Yes, this is an alpha version but without this there may be some UX difficulties when it comes to certain interactions. For example, on 1.1.0-alpha02 we experienced some oddities when trying to open the navigation drawer within our app. For this reason you should be sure to use at least 1.1.0-alpha03 if you are targeting sdk 29.

If you’re using the androidx appcompat dependency, then updating this to 1.1.0-Rc01 might be required if you are planning on making use of the sharing improvements, with backwards compatibility, for Android 10. When it comes to the changes we made to sharing improvements, this update was required to make use of the ShortcutInfoCompat and ShortcutManagerCompat classes.

Gestural Navigation

This was a big change coming for Android 10 – whilst an optional setting that can be toggled from the system settings, gestural navigation allows the user to navigate through the system, apps and backstack via the user of swipe gestures. With the aim to replace the software buttons located at the bottom of the device, this gives us more screen estate to play with as well as give the option to provide a more streamlined navigation experience.

After enabling gestural navigation through Settings > Gestures > System Navigation you’ll be able to try gestural navigation for yourself. Whilst at first this may just seem like a new way to navigate through your device, it can actually have a big impact on the user experience of applications. Because we can now swipe horizontally from either side of the screen, this can interfere with components within your application.

Now, this does really depend on the application in question, as some application may not be affected by this change – it really depends on the components that make up your project. Before updating to alpha03 of the navigation drawer dependency, within alpha02 we experienced an issue where the navigation drawer would not open correctly. As it was, this would not be releasable as that’s where our users changes their currently selected account, which is a core part of our application. If you’re unaware of gestural navigation or missed out on testing this, it’s worth double checking that you’re on the latest version of this library and that the navigation drawer in your app behaves correctly.

Alongside the navigation drawer, anything component that may usually be intercepted within the bounds of this back-swipe gesture should be tested to ensure that there is no interference here. For example, if we had an edge-to-edge swipe-able view component, this could potentially be affected by gestural navigation. Whilst we didn’t have anything else that was affected by the gestural navigation changes, this might not be the same for your application. Check out this blog post for more information on how to overcome these kind of issues.

Scoped Storage

As the versions of Android have progressed, we’ve often seen changes being made to permissions and/or how we access files on the device. This is an important topic and we can understand these changes being made as our users content needs to be both contained and available by other applications when requested. Scoped Storage has been an interesting topic, right back to when the original beta releases and documentation came out for Android 10. There are more information on these changes here, but to summarise how media access now looks across the system:

Compared to the original specification for scoped storage, what’s in place now for Android 10 aimed to provide a way that was both accessible for user and developers. For Publish, we needed to make a couple of changes as we were seeing some errors when trying to handle media from certain locations on the device. In some places we were previously making use of the file path to access files on the device from other apps, and on Android 10 this can cause issues as we may not have permission to access the file. If we are looking to retrieve a file then we should now be accessing it using the content resolver – on Android 10 there is now a function available called openFile() to do this.

val fileDescriptor = context.contentResolver.openFile(mediaUri, "r", null)

Once we have the ParcelFileDescriptor from this call we can use that to access the file, without running into any permission related issues (provided that we have been granted read access to the users storage where required). This method for access files will also need to be taken into account when trying to read Exif data for files – so when trying to create a new instance of an ExifInterface this will need to take the file retrieve from the media store using the above approach. The same rules will apply for any approaches involved when trying to deal with media – your best bet is to read up on the storage related changes for Android 10 and see how they might affect your app.

Settings Panels

This feature allows users to access key settings for the device (related to connectivity and audio) so that they can easily change common settings without leaving the context of your application. This functionality is only available on Android 10, so is not backward compatible with older versions of the Android OS. A lot of applications make use of networking features, so the connectivity side of things is something that these apps will be able to (and should really) make use of.

Let’s take a look at one example of where we make use of this. Within the Buffer Publish app we load content from our API into the social queue of an account. If this request fails, we show an error view that allows the user to retry the request. At this point, the retry button can be pressed and we’ll attempt to reload the content. However, if there is a connectivity issue (maybe the user has airplane mode turned on, or is not connected to WiFi with their data connection disabled) then this retry button will send the user into an endless loop of failed retries. We decided to make use of settings panels here so that if there is no connection available then instead the button will instead launch a connectivity settings panel to allow the user to change their connectivity settings from within our application:

activity.startActivityForResult(
    Intent(Settings.Panel.ACTION_INTERNET_CONNECTIVITY), 
        REQUEST_CODE_SETTINGS_PANEL)

We use startActivityForResult here so that we can detect when the user has returned from this event using REQUEST_CODE_SETTINGS_PANEL. Ideally here you would register a connectivity listener and change the state of the error view based on events coming from there. However, to keep lean and manage our priorities right now we decided to just presume that the user had changed their settings and switch the button to present “Retry” so that they can reload the content. If that then fails again, we go back to step one based on whether or not there is a connection available. This approach works well and allowed us to get this enhancement in place – there’s definitely room for improvement there in future if we see that the settings panel is commonly interacted with.

Sharing Improvements

Android 10 sees some improvements being made to the share sheet functionality, as outlined in the release notes. This is a huge improvement for users, as previously sharing routes would take some time to load – causing a lag effect when trying to share something using the system share sheet. With this new approach, share targets are shared in advance so that the system has them available for display instantly – the cool thing is that this approach uses the existing app shortcuts API to provide this functionality.

We’ve made use of sharing shortcuts so that the share sheet will show individual social accounts from your Buffer account – that way, our users can share content directly to the composer with that social account selected:

ShortcutInfoCompat.Builder(context, profile.id)
    .setShortLabel(profile.formattedUsername)
    .setIcon(IconCompat.createWithBitmap(bitmap))
    .setCategories(shareCategories)
    .setIntents(arrayOf(intent
        .setAction(ACTION_MAIN)
        .setFlags(Intent.FLAG_ACTIVITY_CLEAR_TASK or Intent.FLAG_ACTIVITY_NEW_TASK)))
    .setPerson(Person.Builder()
        .setKey(profile.id)
        .setName(profile.formattedUsername)
        .setIcon(IconCompat.createWithBitmap(bitmap))
        .build())
    .build()

You may notice the setCategories(shareCategories) call in the builder. This is setting the share category for the shortcut, as per the documentation, we the have this shortcut defined within our shortcuts.xml file

<?xml version="1.0" encoding="utf-8"?>
<shortcuts xmlns:android="http://schemas.android.com/apk/res/android">
    <share-target android:targetClass="org.buffer.android.composer.ComposerActivity">
        <data android:mimeType="text/plain" />
        <data android:mimeType="image/*" />
        <data android:mimeType="video/*" />
        <category android:name="org.buffer.android.category.COMPOSER_SHARE_TARGET" />
    </share-target>
</shortcuts>

Because we have the intent set for the shortcut, this will launch the home screen for that account if the app shortcut is interacted with. On the other hand, the category declaration will handle the case where the shortcut is triggered from the share sheet.

With this approach above we can now provide a more streamlined (and frictionless) sharing experience into our application throughout the system.

Note: If you’re looking to use the Compat classes for sharing shortcuts, then you’ll need to be on at least version 1.1.0-Rc01 of the androidx compat dependency

Biometric prompt improvements

We never previously had biometric login from our application, mainly because it wasn’t a priority for us to implement. With Android 10 containing a few more improvements to biometric prompts, we decided to add some functionality to our application to take advantage of what the system has to offer here.

From within the settings of our app, the user can now enable a setting to require fingerprint authentication when the app is opened (provided that they are logged in). Then when the app is launched and this setting is enabled, we make use of the BiometricPrompt class (available from API level 28) to display this biometric prompt to the user.

BiometricPrompt.Builder(context)
    .setTitle(title)
    .setSubtitle(subtitle)
    .setDescription(description)
    .setNegativeButton(negativeButtonText, context.mainExecutor, onClickListener)
    .build()
    .authenticate(cancellationSignal, context.mainExecutor, callback)

The main change here is that behind the scenes the BiometricManager from API level 29 is used. This class provides us with a more convenient way to check the biometric capabilities of the device, as well as the ability to provide a fallback route (PIN, pattern etc) incase the user cannot use biometric authentication.

As you can see from this article, we didn’t have to make too many changes in-order to get our app ready for our users who are running Android 10. We found that whilst these changes were small, there was a lot of manual / automated testing that needed to take place to ensure that our app functioned as intended for this OS upgrade. As time goes on, we may look at adding some of the other features / enhancements that Android 10 has introduced – be it for current or future feature implementations.

Is your app ready for Android 10? We’d love to hear about the things you’ve put in place for this release, along with any questions that you may have during that process!

Header Photo by Alexander Andrews on Unsplash

When it comes to building android applications, there’s no doubt that we’ll need to include some form of navigation to move between the different parts of our app. With modularisation becoming more and more popular when it comes to android development, navigation becomes a big part of this process.

At Buffer we’ve begun creating a lot of shared code between our applications – some of these are utilities, widgets and even features (note, these are not yet dynamic feature modules, instead they are library modules). When it comes to features, often these will need to navigate to another part of the app – however, because these library modules are not aware of the base android app module they are unable to satisfy the navigational requirements in our app.

In this post we are going to explore a solution that not only solves this problem for us, but helps us to separate the navigational concern from the rest of our application, providing us with numerous benefits in the process.

In the example we can see that we can’t easily navigate to some of the screens in our application, purely because we do not have a reference to them. Whilst we are forced to solve this initial problem here, this gets us thinking about navigation in general within our app. With single module applications we would normally handle the navigation by calling startActivity() , followed by passing in an Intent that has a direct reference to the Activity class which we were navigating to. However, this approach brings the question of whether or not our activities should have the knowledge of where it is they are navigating to? If anything, this leaks another concern & responsibility into the activity. As well as this, the class that is launching that activity now has a direct reference to it, which adds to the concepts that our launching class is aware of and being directly tied to that destination – which is something that a library (in most cases) cannot do. Finally, when it comes to the testing of our class, navigation also becomes a concern of these tests too – which reinforces the argument of there being another responsibility that is a part of our activity.

As it is becoming more common now to introduce modularisation to the mix, this is likely to become a common problem faced amongst applications. Because of this and the above issues, it may make sense in some cases to split out the navigation of our app to be handled by some classes outside of the ones which may be currently performing the navigation. Not only does this make navigation possible from these internal library modules, but it helps to separate the concerns of our navigation and it makes it far easier to test the implementations of navigation within our apps.

With all of this in mind, how can we achieve the above when it comes to navigation within our android apps? Let’s begin by taking a look at an approach that meets all of the above requirements.

Because we’re focusing in this post on library modules, let us begin by taking a look at a simple library module we have within our applications. Both of our apps (Publish and Reply) share the same on-boarding screens and because of this, we make use of a shared library that we import as a gradle dependency. This dependency shows a couple of on-boarding steps that the user can swipe through, but the important part here are the two buttons that are displayed to the user. These buttons allow the user to either Navigate to the Sign-Up or Sign-In screens – these are not part of the on-boarding library as currently these following screens are verify different for each of the apps. As previously mentioned in this post, the on-boarding library does not have (and cannot have) a reference to the base app module due to it being a library module. So as it is, it cannot navigate to the activities inside of the base app module – and because it is being reused for multiple apps, the paths to the desired activities will be completely different so this is not something that should be hard coded.

With this in mind, the Onboarding library is going to need to provide an interface which states the navigational requirements that are going to need to be satisfied. This allows the onboarding module to define and enforce these requirements without having any knowledge around the actual details of these actions.

From here, the application module using that onboarding library can implement the interface and satisfy the navigational requirements. So for example, the activity launching the screens of the onboarding library may implement that interface and when the methods are triggered, handle the navigation around sign-up and sign-in.

Whilst in the process of implementing this however, it got us thinking about the responsibility of navigation. Our activities and fragments are already handling other things, we could make an improvement here by removing this responsibility from these components and handling the navigation elsewhere.

For this reason, we decided to introduce a Navigation module. The purpose of this module is to encapsulate all of the navigational logic of the application – allowing us to remove this knowledge from our activities / fragments and make it far easier to test the navigational aspects of our app.

With this approach, the Navigation module needs to have a reference to the modules that contain the interfaces defining the required navigation. Whilst this module may end up having a reference to multiple modules, this is fine as its not intended to be reusable and is also fulfilling its purpose – it also removes the need to have these dependency references from within our app module itself (in most cases).

Whilst the Navigation module implements the navigation, the Publish module here still needs a reference to navigation for two reasons. First of all, we don’t have Navigation handling its own dependency injection – navigation becomes a part of our Dagger Component and Module that we have configured within our Publish module. This allows us to provide the required injections inside of the onboarding library – whilst this isn’t ideal having this injection requirement here, seeing as it is an internal library this solution works well for us at this point in time.

Another reason why the Publish module needs a reference to this Navigation module is to account for other navigation requirements throughout the app. Moving forward we will not be restricting this to library modules only – as we move to decouple more features within our application, these will use navigation interfaces within their corresponding packages (even if not yet modularised) – so having this reference to the navigation module helps us to achieve that.

When it comes to this Onboarding module, as previously mentioned, it will define an interface (lets call it the OnboardingNavigator) that defines the the required navigation. This might look a little something like this:

interface OnboardingNavigator {

    fun showSignUpForm(activity: Activity)

    fun showSignInForm(activity: Activity)
}

We pass in the activity here so that the onboarding module can completely handle the navigation to the next destination. We do not want to navigate here using string declarations of activity paths, so some form of context is required here.

Within our Navigation module we’re going to want to provide an implementation of this interface – this allows us to implement the required functions and launch the required activities when those functions are triggered.

class OnboardingCoordinator @Inject constructor() : OnboardingNavigator {

    override fun navigateToSignUp(activity: Activity) {
        activity.startActivity(...)
    }

    override fun navigateToSignIn(activity: Activity) {
        activity.startActivity(...)
    }
}

Here we actually use the path of the activity to satisfy the intent. This way we do not need a reference to the base app module of our project within the Navigation module, the same goes for the other destinations that we might navigate to.

Then, when it comes to the library module, we can initialise a new instance of our coordinator inside of where it is used. If you are using dependency injection, you can inject this into the library module by injecting the interface type and then access the functions of that interface as desired:

@Inject lateinit var onboardingCoordinator: OnboardingNavigator

Now that we have access to the OnboardingNavigator reference we can make use of it to trigger the required navigation. So for example, here we have a click listener set on one of the buttons within the onboarding screens – when that button is clicked we can call the corresponding interface function.

button_new_user.setOnClickListener { onboardingCoordinator.navigateToSignUp(this) }

When it comes to these library modules in our applications, we will generally have the kind of structure that is stated below. This outlines the dependencies that will occur between the Navigation module and the specified feature Library module.

But for any feature library module that we have implemented, we’ll end with a similar approach regardless of what the feature is. With that in mind, we can summarise that the navigation for these modules will take on a general structure when it comes to handling navigation.

To conclude the above approach to handling navigation when working with android library modules, overall we end up with something that looks like this:

In this diagram it may look like there is a lot going on, but some of these boxes just represent the functions that we define within the interfaces, along with the implementations. We can see here the different steps of this article pieced together into a complete solution. To summarise this we end up with:

• A library module that defines the navigational requirements in the form of an interface
• A navigation module that is responsible for implementing the navigation that is defined within that library module
• An app module which configures this navigation module for dependency injection and also to provide navigation should any other parts of our project need it (this part is subject to your project structure)

With the three core concepts in mind, we can see that we now have a clear separation of responsibilities when it comes to navigation, keeping our classes more lightweight and focused. We also see the benefits when it comes to testing too, now our tests for our activities / fragments no longer contain checks for specific intents being launched – however, we can still test these behaviours for our coordinator classes. For example, we could write a small test for our coordinator to ensure that the correct navigation remains in place:

@Test
fun showSignUpFormNavigatesToEmailConnectForSignUp() {
    main.launchActivity(null)
    Intents.intending(IntentMatchers.anyIntent())
        .respondWith(Instrumentation.ActivityResult(Activity.RESULT_OK, Intent()))

    onboardingCoordinator.showSignUpForm(main.activity)
    intended(allOf(hasComponent(Activities.EmailConnect.className),
        hasExtra(EXTRA_CONNECT_TYPE, 0)))
    Intents.release()
}

We can also do a similar thing from UI tests. Whilst we are not testing here to check whether a specific activity is launched, we can still verify that the navigator instance was interacted with as expected:

verify(onboardingNavigator).showSignUpForm(...)

As we move forward with features we will take the same approach that has been taken above to satisfy navigation throughout our app. Even for features that are not yet modularised (or able to be modularised, for example due to tight coupling) we will be able to take a similar approach by using the navigation module whilst experiencing all of the advantages listed above.

At the same time it’s important to note that this may not be applicable to all applications, before putting this in place (like any technical approach) it’s important to ask if you need this. When it comes to library modules where we cannot access specific classes it feels appropriate as we need some form of interface in place anyway. But for the case of your applications without this modules in place, yes it’s a benefit but is not the be all and end all.