Building External Service Mocks for Testing

by Elaine Uy in


For the past couple of months, my team has been transforming our internal architecture from a monolith into a sleek service oriented architecture. We’ve settled on using JSON Hyper-schema to define our service APIs. Each service is going to host one of these schemas to expose what resources are available, what attributes each resources has, and what kind of calls you can make. We then build a client to consume these schemas and act as the gateway for access to the external resources. In the end, we hope to have independently deployable applications that expose whatever data they need to through internal apis.

ex ad test copy
ex ad test copy

Though there are benefits to breaking up our large single application into services, there are a lot of real problems this conversion surfaces. The first one that comes to mind is, we need to figure out if our isolated service works the same as before. There are several tests we need to do to make sure we have parity, but at the unit level, we need to convert all our specs from relying on having access to all of the models to making external api calls for models that were no longer internal to the isolated service.

To accomplish this spec conversion, we need to be able to quickly mock out our service clients, and the resources returned from them preferably without a connection to a running remote service. (This approach is great for helping getting over the bar of internally testing a newly isolated server, but should not be used when doing interoperability functional testing.) There are a lot of approaches on how you should do your testing when it comes to making external API calls, but I’ve found what we’ve done here really jump started our testing framework and is making our move from single app to several services much smoother. Since we’re moving in the direction of supporting several internal services, it only seemed sensible to build out some kind of infrastructure to support our growing collection of services with a uniform approach that aligns with our client development.

When we look at this from the perspective of our specs, we are shifting from locally accessible model classes and local datas to just having access to a resource’s response data. Even though we’re moving the model code and datastore location out of scope, we are still handling test data that complies exactly to its JSON Hyper-Schema local to the client. Creating fixtures for a well defined set of resources is surprisingly a lot like functionality from a ruby library that already exists to set up test data.

Enter Factory Girl. Since it has a well known DSL, and it’s already used in the codebase, it seemed like a natural choice for mocking our service resources if we could get it to work. As it turns out, you can use Factory Girl without a backing ORM, which is exactly what happens when your datastore lives in another service. We had to do a little tweaking to skip creation calls. For more information on the nitty gritty details of how to do this, I found this blog post extremely helpful. Factory Girl still needs a class to base its test objects on, and with the information from our the JSON Hyper-Schemas, we can actually generate what we need.

In our client, we are using Structs to wrap our JSON responses in a nice container object. Structs let us conveniently access our data with two kinds of syntax: the dot syntax and the hash syntax keyed by attribute name (both by symbol and by string). We’ve found this a lot more flexible to use instead of a straight up parsed JSON, and consequently, we can use the struct objects our clients already make to factorize for our test data. As a bonus, Factory Girl works really well when your object already has attribute_name= setters and attribute_name accessor methods readily available.

We decided to create a task to auto-generate all of these client resource factories based on the struct classes for everything our local app needs to consume. Using both the external schema and knowledge of how our client is initialized, we can actually build the stubs of our client calls in our factories in an after (:create) hook and return our factory girl generated objects.

Our client call pattern is something like this:

Services::ServiceName::resource_name.method

where on initialization of an app, we create constants of the clients named after the ServiceName. Each client has resource objects attribute readable on them with all of the schema’s specified properties and links as methods on that resource object.

We can factorize any generic method call like this:

I ended up packing the above code and a skip_create into a trait. We want to provide the minimum skeleton factory to replace existing model factories in the specs. The biggest pro for doing this is to preserve existing factory calls in the huge test suite, since your specs are agnostic of how an object is persisted. However, it’s a little dangerous to stub too many method calls since it’ll mask real behavior, but if you’re creating a resource object, you’re guaranteed at least a look up. In case tests don’t need the look-up stubbed, I added an ignored attribute to toggle the stub creation. Ignored attributes in factories let you pass a parameter into factory creation that isn’t set on the class, but can be used in the factory’s scope.The finished generated factories look something like this:

There are several ways you can expand these generated factories in tandem with more json hyper schema definitions. For example, add min and max values as on an attribute can translate directly to default values on the factory itself. You could even make sequences out of any regular expression statements used to specify an attribute as well. If you wanted a complete dummy, you could stub every link on a resource to return your struct mock, but that approach is a little extreme and as mentioned above risky. In general, you should only stub what you know you’re going to call.

There are definite drawbacks here. At the moment, we’re not packaging these libs in the client, so it’s up to the local app to maintain, update, and own its factories. Additionally, if developers extend the generated client factories, they would be responsible for porting their extensions when a new schema was published. We kind of view these factories similar to the effectiveness of a rails generator. It’s good to get you started, but you still have to shell out some work for it to be what you want as your behaviors get more complicated. Every application is going to use a service’s resources differently, so it’s fair to say that each application should have unique code. For loading the test environment, we added the functionality to load clients from a locally stored json hyper schema instead of having to reach out to a remote client which makes the testing much less annoying.

other resources:

http://blog.newrelic.com/2014/05/05/using-new-relic-refactor-care/

http://nerds.airbnb.com/testing-at-airbnb/

 

Think this is interesting? We're hiring! See our current openings here.