Here at Tapjoy, we’ve recently started embracing Go for everything from greenfield projects to core application logic rewrites. Because most of Tapjoy’s codebase is written in Ruby (Ruby on Rails, Sinatra and pure rack apps), I came into Go with some specific expectations around how testing might work. However, I quickly discovered that routing specs in Go weren't as straightforward as I had hoped.
Looking at rspec, routing specs are fairly simple. According to the rspec-rails docs a simple routing test might look something like this:
For our application logic tests we’ve primarily been using ginkgo and gomega, and we love taking advantage of the built-in Go benchmarking feature. Where we need to mock interfaces to test calls on other packages, we use gomock. Unfortunately those sites didn't have any clear documentation on how to write router tests, so I started trawling the web looking for more descriptive guides.
The First Pass
The first helpful article I found was Kyle Truscoff’s blog post, Mocking HTTP Responses in Golang. While his application didn’t exactly mirror ours, it clued me in on how a lot of the httptest package works in conjunction with the http package to allow for testing response objects. It was also very helpful as a guide for how to test handlers.
I took a basic stab at implementing what I had learned through the post and uncovered an issue in the way that we had written our own application routing code!
Our HTTP router is initialized within a versioned package for our HTTP APIs. This package describes an HttpAPI object which contains the route and handler (we use gorilla) definitions for our server (which we serve using grace). When we call New() in this package, we create a new HttpAPI object, assign handlers to routes, and generate a server object (for passing to grace). All of this information is then attached to the HttpAPI struct which was being generated.
Multiple Registration Panics
There are some routes that exist purely to make integration and end to-end testing easier; therefore we do not want to expose them in a production environment. One of the first tests I wrote was to test this behavior. I had written a few ginkgo-styled test cases that were structured with Context calls and BeforeEach sections that would build out production and non-production routers.
Imagine my surprise when my tests would panic with:
http: multiple registrations for /
So what was happening?
The case for a Refactor
To understand what was going on here requires a bit of an understanding of how the Go http library works. When handler functions are added (with either http.Handle or http.HandleFunc, for example), those handler functions are associated with the http.DefaultServeMux. The http.DefaultServeMux is then assigned to the http.Server as the Handler.
Our test code was actually creating multiple duplicate handlers for http.DefaultServeMux every time we were calling New()! In what seems to be a more and more regular occurrence for our team as we feel our way through Go, our experience testing led us to refactor our code.
With the handler assignment now safely isolated from server creation, it was now possible to create new routers multiple times in the same test without hitting the same panic that was blocking me before. Now the BeforeEach blocks looked something like:
Because the test server is created each time using a different returned router object (see line 4, above), we had eradicated the panic condition. But wait, there was a new issue!
Some requests would succeed, but others would mysteriously fail, with:
dial tcp 127.0.0.1:51041: getsockopt: connection refused
What was going on? It took some head scratching, but I finally realized my mistake. The server was shutting down before receiving a request! It hadn’t occurred to me that exiting the BeforeEach function would trigger the defer. After much trial and error, the solution I settled on was removing the defer server.Close() from the BeforeEach call, and instead running it in an AfterEach call. BOOM! My tests were off to the races.
Router Tests: The Final Form
After all that, the structure of our tests looked something like:
I was then free to write my tests using the server.URL, which negated the need for a custom Transport within the httpClient. Hooray!
A “Bonus” FileServer gotcha
At this point all of the router test cases were finished, save one. Here at Tapjoy, we use JSON Hyperschema to define a lot of our server-to-server endpoints.
In order to easily handle the JSON schema serving, we have been using http.FileServer. The line in our NewRouter() function that added the FileServer was something like this:
It seemed straightforward enough, and I could verify that I was able to load schemas when I was testing locally. But for some reason, every time I had a test case that attempted to verify the ability to route to the schemas, I was getting a 404.
The issue here happened to be the current directory. All of our package tests reside side-by-side (per ginkgo’s recommendation) inside their respective packages. This means that our router tests were being run from inside `api/http/v1/` while our main program ran at the root level. Where this becomes interesting is the relative file path for the FileServer.
While running the main loop, api/http/v1/schemas was the correct path on disk to the schemas. When running the FileServer from inside the httpv1_test package, api/http/v1/schemas didn’t exist!
For now, this means that we don’t have a working schema test in our httpv1_test package. To solve this problem, we would need to add this test to the test suite for our main package in order to verify that we are routing to the files properly.
Clearly there is more exploring to be done, but we are excited to share what we feel is a pretty straightforward way to get router testing done in Go!
At the end of our effort to add routing tests, what we discovered was that the tests themselves weren't the most valuable result of the work. In the course of writing effective tests, we were led to improve our code, which in turn provided a clear model for future Go work at Tapjoy. It is great to have tests, but it is even better to have a pattern to follow for writing testable code in Go.