Testing with Riak at Tapjoy

by Aaron Pfeifer

At Tapjoy we put a lot of effort into building effective tools and environments for testing.  This includes everything from performance to continuous integration testing to fully integrated environments for manually testing.

Today we're open-sourcing one of the tools we use at Tapjoy called fakeriak: an in-memory driver for Riak for testing and machines without Riak.  Let's talk a little bit about why we built fakeriak and how we're using it.

One area that is often challenging here is how to test code which has dependencies on external services.  These services could be anything from data stores like MySQL, Redis, and Riak to third-party services like Stripe and MailChimp.  While we want our code to be using the real service for staging and production environments, using them as part of automated tests is often untenable.

This is typically where a technique like method stubs is often used.  Instead of using the actual library itself, the API is stubbed out with the return values that you expect.  For example:

This technique has a lot of upside to it.

  1. Performance.  If you have a lot of tests that are making use of the external service (say on the order or thousands), stubs can increase the speed of your tests by a large margin.
  2. Test Isolation.  By removing the external dependency, tests are able to run in isolation.  This allows tests to run in parallel, which is particularly useful on continuous integration servers where you could have several different versions of the same test suite being run in parallel.

However, it also has a few disadvantages.

  1. Lack of confidence.  Adding a layer of abstraction can cause your tests to pass when they in fact fail against the actual library itself.  Remember, at this point you're now testing the stubs you've built in your specs rather than proper integration with the library.  How do you validate that your stubs are correct?
  2. Maintenance.  There's a point at which the number of stubs needed for each test becomes a maintenance headache.  It's not impossible to do, but it also no longer becomes convenient -- particularly when the library is upgraded and subtle changes to its API are introduced.

The points made above are the reason we use things like fakememcached and fakeredis -- and the reason we've built fakeriak.  Instead of stubbing methods, you're interacting with the actual library itself which also means no need to maintain those stubs as well!  Your tests end up looking like so:

fakeriak achieves this by providing an in-memory implementation of the riak client's backend driver.  We've found this to work great for many of our projects and helps to bump up confidence in the code we're building.

If you're interested in using this for your own projects, just jump over to the githubo repo and get started!  Happy testing!


Think this is interesting? We're hiring! See our current openings here.