Building Admin APIs for Rails Apps

At Cloud 66, we are strong advocates of building tools to simplify and automate tasks handled by humans. This approach has allowed us to scale our operations with a small team. We often open-source these tools to help other companies. You can see a list of our open source projects on our website.

Background

Our main application is written in Rails. Some of the support we provide to our customers involves inspecting and sometimes modifying Rails models in the app.

For example, transferring the ownership of an application: this is something our support team does for our customers. To make this change, we need to load an application model, run some validations on it (like the types of Cloud accounts connected to it, etc.), and then re-assign it to another account. While those checks are done in the code, starting this operation is something we do manually using our Support Toolbelt, which we call OpX.

OpX is a Ruby gem that communicates with our main Rails application through its Admin API - which is what I'm writing about today.

Admin API

With all of the business logic residing inside of our Rails app, we built a separate and private API endpoint to handle operations that only our support team can perform. This might include tasks like adjusting payments, moving applications between accounts, or checking on the status of a long-running job during a deployment to find any issues.

Admin API Generation 1

The 1st generation of this AdminAPI was a RESTful set of endpoints on the main Rails application. Authentication used an oAuth2.0 mechanism, similar to the one used for our public API.

While writing a RESTful API is relatively simple, REST falls short when it comes to admin operations. Let's go back to the example of transferring the ownership of an application. Ownership transfer involves not just modifying the "owner" parameter on the application, but also a lot of other long-running preparation steps, including checking Cloud API signatures, credit card validations, seeking manual agreement from the transferee's account and more.

In a RESTful world, you would write a PUT (edit) endpoint for your application domain model, which checks if the owner has changed and initiates the whole transfer process. In reality, this quickly turns your controllers into a mess:

1. Each attribute change can trigger a bunch of different processes.
2. Multiple attributes changing at the same time can have more complex outcomes.
3. Some operations will return immediately while others will return an ID for an asynchronous job.

We quickly abandoned the RESTful Admin API  and moved on. While REST is an excellent methodology for many use cases, it does not suit some admin operations.

Admin API Generation 2

Next we looked into using GraphQL for our Admin API. GraphQL seems better positioned for specific tasks based on mutations. It also lets the caller decide which fields to query, which is very useful for large domain objects with many dependencies. We ended up not using GraphQL for two reasons:

1. GraphQL has a somewhat steep learning curve and is nowhere near as easy as REST. Successful adoption of a technology in the team depends on the ease of learning, particularly when used for internal tools. Making mistakes in our Admin API can have serious consequences.
2. GraphQL is a data request, manipulation, and transfer technology. It doesn't have asynchronous operations built into it. Some admin operations can be either synchronous or asynchronous on the same object, depending on the type of task.

Our 2nd generation Admin API used GRPC. Google started GRPC as a language-agnostic RPC protocol. It is widely widely, particularly in projects written in Go, but many implementations exist for other languages, including Ruby. GRPC also supports bidirectional messages as well as async and streaming channels, which makes it a great candidate for what we needed.

However, our 2nd generation Admin API didn't last more than one year. Here is why:

1. GRPC relies on Protobufs, a binary and compact representation of models that are good for interoperability between multiple languages, has built-in backward compatibility paradigms and has an expressive DSL that helps with documenting models. However, we found that Protobufs added too much burden to our development process (converting the DSL to Ruby code, taking care of mapping them to native Rails models, etc.) This complexity is not justified when both ends of the call are entirely in your control and are in the same language. JSON would do just fine in a case like this!
2. GRPC relies on HTTP/2 and other technologies in the transfer application, which means it sits "next" to your Rails application instead of a proper mounting in the Rack sense. While libraries like Gruf do an excellent job of getting GRPC work with Rails, the result is a mess that's difficult to maintain.
3. This side-by-side approach takes our Rails authentication middleware out of the flow, which means our Admin API would need an authentication application of its own.

Admin API Generation 3:

By now we had a good idea on what we needed. We needed an RPC that:

• Was Rails friendly
• Had built-in authentication and worked with Rails authentication.
• Supported async and sync operations natively.

These  specifications are simple enough to build, especially for internal use behind dependable security layers. That's why we developed Unrestful, and today we are open-sourcing it!

While Unrestful works on the above principles, you can choose your own authentication framework for it. We use it with JWT, which means we can lock it down using Auth0, Cloudflare Access, or Google Identity Aware Proxies (Beyond Corp).

I encourage you to take a look at Unrestful example and source code, send feedback, and please feel free to make PRs!