Testing a Golang gRPC API with RSpec

The title of this blog post might appear weird for those in the Ruby and Golang communities. Rarely do you hear of developers writing an API in Go and testing it using the darling of the Ruby testing world, RSpec.

With RSpec’s wonderful tooling and support, we figured we could write more readable tests than with any of the Go testing frameworks. We were mainly interested in testing the API as a whole unit in itself; seeing it as a blackbox that accepts a request and returns a response. This way we are able to make large changes and dramatic refactors to the internal implementation without breaking any tests. We can still be confident it does exactly as we expect when it comes to the database layer.

So how are we even testing this?

We’re creating a new application built on top of an existing Microsoft SQL Server database, so naturally we need some way to connect to that. We need to be able to query the DB, as well as iterate over any results we receive back. FreeTDS is the standard piece of software in the industry for allowing applications to connect to a Microsoft SQL Server, and we use TinyTDS which is a Ruby wrapper around FreeTDS.

TinyTDS comes with many benefits, such as converting SQL Server datatypes to lovely Ruby primitives, and allowing you to iterate over the results returned from a database call.

The first part in our specs is to open a connection to our test database. Because TinyTDS does not differentiate between a database connection and being able to accept queries, we assume that any errors mean no database connection could be made, and therefore no queries can be submitted

Once a DB connection is established, we can start setting up our test database so it’s in a clean, consistent state on each test run.

Here we are executing some SQL queries to the test DB through a Factory. Our Factory class provides a range of methods that clears and inserts values into the DB before the specs run. In the case above, we want to ensure all of the data in the tables that the supplier references through a foreign key are removed, hence the .clear methods invoked in the before do block, which return the id of the supplier just created in this case.

Our Factory class provides a nice abstraction for executing queries to our test database before our specs run, so our spec files don’t need to have knowledge of the database schema and can focus only on setting up assertions on the data returned from the API.

Using TinyTDS we can pass through our db_client(which is our TinyTDS client instantiated through our open_database method above). This TinyTDS client now has methods attached to it that can execute SQL queries to our test database. The typical approach is to use the execute function of the TinyTDS library, pass in the SQL as a string, then call either insert if you’re inserting into the DB (.insert), or do to ensure that we execute the statements synchronously in order to prevent race conditions.

How Does RSpec communicate to the Golang API?

This is where things become interesting (as if you didn’t find the stuff above equally as interesting). How do we communicate to the Golang API through our RSpec test suite? At some point once the test database is set up and appropriately seeded, we need to make a call to the appropriate APIs to actually execute the action under test.

We’re using gRPC for the API, a Google RPC (Remote Procedure Call) framework. Remote Procedure Calls are a technique used in computer science for calling functions in a different address space; so essentially you can have a server on one machine, and a client that communicates with the server by calling functions defined on the API as if they were local. This pattern is becoming increasingly popular as the industry moves towards a microservices over monolithic architectural pattern.

With gRPC you start with a proto file, which defines exactly how the client and server can communicate and with what information.

Above we define some remote procedure calls that can be called from the client to the server, what messages the request takes and what the server will respond with. We define this within the Neptune service (our internal application).

These response messages typically contain a repeated field of errors and a payload with the contents of the response returned back to the client. In the case above, when we call the GetSuppliers RPC, we receive back a response with possible errors or a payload containing information pertaining to the Supplier; like name, id, country etc. The numbers (e.g = 1 and = 2), represent the unique “tag” that the specific field uses in the binary encoding. So the name of the field itself does not get transferred over the wire, but its binary representation does.

gRPC uses protocol buffers (created by Google) to pass information across the wire. Unlike JSON, protocol buffers are a binary Interface Definition Language (IDL) which make them impossible to read by humans, but much faster for computers to process.

Especially for microservices where you have multiple services talking over a network, a binary transport format is much more efficient compared to transporting potentially large JSON strings or XML everywhere. Compared to XML, Google claim that protocol buffers are 3–10 times smaller and 20–100 times faster. Google have now gone full proto, claiming that protocol buffers are now their “lingua franca for data — at time of writing, there are 48,162 different message types defined in the Google code tree across 12,183 .proto files”

Google’s protobuf compiler allows us to auto-generate client and server stubs which take care of a lot of work for us. We don’t have to worry about serializing and deserializing the requests and responses. The client / server code necessary for a client to talk to an API is all generated for us through the protoc compiler and placed in their appropriate directories. In our case, we direct them to the Golang API and the RSpec api tests directory.

protoc --ruby_out=lib --grpc_out=lib --plugin=protoc-gen-grpc=`which grpc_ruby_plugin` ./neptune.proto
protoc --go_out=plugins=grpc:pb ./neptune.proto

Now that the stubs have been generated for both the Golang API and the Ruby client in RSpec, we can now call the RPCs defined in the Go files through our spec files, just as if they were local Ruby methods. The automatic code generation done through the protoc compiler makes this extremely easy to do. A get_suppliers method will be generated in a Ruby file that, when invoked, will call the GetSuppliers function in the Golang API.

Remember this example from above:

Our RPC_STUB here is defined in the spec_helper and sets up a connection to the Golang API so we can call RPCs on it with ease.

In this example test, we want to test functionality on the Golang API for getting suppliers. We want to ensure that when we call the get_suppliers remote procedure call, we receive back the suppliers in a protobuf message that contain the suppliers. Under the hood, the get_suppliers Ruby method call is calling the GetSuppliers function defined on the Go API

In summary, this function is exposed to our client through gRPC. It doesn’t take any parameters (represented by google_protobuf.Empty), and returns a a *pb.GetSuppliersRespoinse and any possible errors. This goes deeper into our store package which eventually executes the necessary SQL query to fetch the suppliers from the Microsoft SQL Server database. Once the results are retrieved they’re appended to the response payload and returned to the client.

In the case above where we test the API that returns us a list of suppliers, the error messages are just as clear as they would be if the API were written in Ruby. We make an assertion on what we expect to be returned from an RPC call, and if the returned value does not equal our expected value we’re told exactly what the difference is.

In this case, I removed the “Name” field from the Supplier struct on the Go side and our failure message clearly specifies what the difference is. Bugs are usually very easy to track down

It isn’t all sunshine and daffodils when using this structure. One notable thing we’re missing is the ability to debug the API on the test run. We can’t just binding.pry into the API, or halt execution as easily as we could if we just used Ruby.

Measuring test coverage has also been an issue. We can’t just install SimpleCov and get a quick idea of where we’re missing tests. There’s no simple way to do automated mutation testing either, so it’s difficult to assess the quality of our tests through automated tooling.

Overall we feel we made the right choice despite the drawbacks. Golang has been wonderful to program in for this API, and RSpec has been a breeze to test with as always. If you do have an idea of how we can solve our test coverage problem, then do get in touch. We are hiring after all.