DevOps Zone is brought to you in partnership with:

I'm a developer who is passionate about agile development practices, dynamic languages, open-source software contribution and test driven development. I'm currently working with Ruby/Rails, Python, Java, .NET and PHP, but I'm always interested in working with new technologies. Lately I've been experimenting with functional languages and server-side javascript. Drew has posted 1 posts at DZone. You can read more from them at their website. View Full User Profile

Continuous Backwards Compatibility

03.10.2012
| 8108 views |
  • submit to reddit

We want to make it as easy as possible to integrate with Braintree and we think one of the best ways to do this is by providing client libraries. They help our customers integrate quickly, provide idiomatic code in the customer’s language of choice and allow us to better troubleshoot integration issues.

While there are a lot of advantages to providing client libraries, one of the challenges is having confidence that new features we develop for our gateway won’t break any of the released libraries. At the time of writing we have 51 supported library versions across 5 languages which makes maintaining backwards compatibility no small task.

Overview

Our development team has two main streams of work: our gateway and our client libraries. As we develop new features for the gateway we maintain both master and release branches in git. The release branch is used for bug fixes and tracks the code currently in production. The master branch is where new feature development happens.

The development workflow for client libraries is more complicated but for the purposes of this post we’ll assume a master and release branch for each language. We also tag our code before each release. For a more detailed discussion of our client library release process see our previous post.

git overview

General Solution

Our solution to maintaining these libraries is, in a word, builds. We’re big believers in continuous integration and like to automate anything we can. We use hudson as our continuous integration tool and have automated test suites for our gateway and all of our client libraries.

The builds we’ve created for our client libraries fall into two groups: small builds with a quick feedback cycle and large builds with a slower feedback cycle.

Small Builds

Each client library has a single small build associated with it. This build runs the tests on the master branch of the client library against the master branch of our gateway code. These builds are triggered in two ways:

  • Any time code is pushed to the master branch of a client library, the small build is run for that library
  • Any time code is pushed to the master branch of the gateway, the gateway build is run followed by the small build for every library

small builds

These builds have quick feedback cycles and gives us confidence that the code we’re pushing to the gateway or our client libraries haven’t broken any current functionality.

Big Build

The small builds are great because they run quickly and frequently, but we’ve still got a problem. What if that small change we’re pushing to the gateway breaks the python library we released 3 months ago?

One strategy would be to create a small build for each tag of each client library we’ve ever released, but this has a couple of problems. First, it’s annoyingly manual and would take a significant amount of time. Second, it leads to a massive number of builds. With 50+ builds, failures can get lost in the noise.

The approach we chose to take was to create one big build that runs nightly. In a nutshell, this build runs every tag of every client library against the gateway’s release and master branches. Below, we’ll walk through the build step by step and show some code samples in ruby. We’ve tried to keep the code simple enough that you should be able to follow along with little to no ruby knowledge.

git overview

First, we clone the git repositories for the gateway and each client library.

sh "git clone git@gitserver:gateway.git"
LANGUAGES.each do |language|
  sh "git clone git@gitserver:client-library-#{language}.git"
end

Next, we switch to the release branch of the gateway by changing to the gateway’s directory and checking out the appropriate branch.

Dir.chdir(GATEWAY_ROOT) do
  sh "git checkout origin/release"
end

For each library we now iterate over all released tags, use git to check out that tag and run the default rake task. This rake task runs that library’s test suite against the release branch of the gateway.

def release_tags
  `git tag -l`
end
def clean_working_copy
  sh "git reset --hard"
  sh "git clean -d -x -f"
end
LANGUAGES.each do |language|
  Dir.chdir("client-library-#{language}") do
    release_tags.each do |tag|
      clean_working_copy
      sh "git checkout #{tag}"
      sh "rake"
    end
  end

Finally, we switch to the master branch of the gateway and repeat the previous step.

Dir.chdir(GATEWAY_ROOT) do
  sh "git checkout origin/master"
end

Managing Build Time

We call it the big build for a reason – it takes quite a while to run. As we release updates to our client libraries the build time will continue to grow. One way we control the time of the build is by managing an end of life list for each client library. This list includes tags which are not being used in our sandbox or production environments. Because these libraries aren’t supported and we proactively block them from accessing the gateway, there is no need to test these tags in our build.

def end_of_life_tag?(language, tag)
  {
    "dotnet" => %w[tag_1 tag_2],
    "java" => %w[tag_1 tag_2],
    "php" => %w[tag_1 tag_2],
    "python" => %w[tag_1 tag_2],
    "ruby" => %w[tag_1 tag_2]
  }.fetch(language, []).include?(tag)
end
LANGUAGES.each do |language|
  Dir.chdir("client-library-#{language}") do
    release_tags.each do |tag|
      next if end_of_life_tag?(language, tag)
      clean_working_copy
      sh "git checkout #{tag}"
      sh "rake"
    end
  end
end

Wrap Up

This approach give us confidence that the code we’re releasing maintains backwards compatibility with all of our supported client libraries. The combination of small and big builds gives us both quick feedback and exhaustive testing of all library versions. We think it’s a flexible solution that can be used by anyone providing client libraries to ease interaction with their API.

Published at DZone with permission of its author, Drew Olson. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)