Web Service API Essentials

After spending the past 15 years of my life building web applications, many of which included some capacity for client-server messaging beyond the browser, I’ve seen a lot of attempts to solve the puzzle. Some folks fail utterly at this task, while others merely miss the mark. Occasionally, I come across a solution that is not only functional, but also elegant in its design and implementation. I could rant for pages, perhaps scores of pages, about the failures and how they lead to developer frustration and slow development to a crawl, but I’ll save that for another time. Now, I’d like to talk about the techniques that bring about graceful, refined web services that are easy to learn, use, and integrate into other apps.

Know your user, the developer.

It took me ten years at least, working in the information systems world, to realize that the key to great design is to understand the people who use the product and their core motivation. Knowing what is most important to the end user is critical to managing development projects. As a result, a significant part of the development effort is invested in pursuit of a basic understanding of the user’s core needs. One can not simply ask for these things. The user may not be conscious of their needs. They may even lie to you, as they do to themselves, believing they need something they really don’t need or even something counterproductive. In API development, though, your user is another developer, so you might think you can assume a minimum level of knowledge about how web services work.

Don’t do that. If you assume anything about your user, set the bar as low as possible. Document everything as plainly and thoroughly as you can. Developers love reading documentation, learning the rules, so they can turn around and apply them to their relevant need. Remember, a developer is reading your documentation specifically because they intend to integrate your API into their system. This will likely make your company money, and in many cases will make their company money as well. This is often the purpose of the API integration project – to increase revenue by attracting new users with a new feature.

Embrace naming standards.

Wherever you stand on the convention vs configuration debate, there are some really valuable lessons to be learned from the convention camp. At an architectural level, a web service is typically an interface layer for interacting with objects or collections of objects. If a significant class in the object model is Song and another is Playlist and you’d like to expose the ability to add and remove Songs from a Playlist, it makes sense that GET /playlists/1/songs shows me only Songs associated with Playlist 1. If I want to see the same collection in XML or JSON, I simply append .xml or .json, respectively, to the path. This is significantly better than GET /cgi-bin/my_servlet.jsp?cmd=list&playlistId=1&format=xml.

There are also situations where custom actions beyond basic Create-Read-Update-Delete (CRUD) are desirable. Sometimes, these actions can be simulated by simply updating an object. For example, instead of having a custom publish action, one might simply update the record, setting the published_at column. Adding a non-standard action adds complexity to the controller and functional tests, as well as the web client code for interacting with this resource. In contrast, using existing standard actions means less code required on the client.

Provide details. Be specific. 

Remember you’re building a request processing system, probably a synchronous one. Your user (the developer) needs to know exactly what your system expects and what it will return, in success and failure cases. If you expect a given schema, with nested objects connoting their associative relationship, you must provide this schema, preferably with an example of sending a request and receiving a response. If there are required fields, note them in the schema. If there is any validation on the request data, describe it in the context of the relevant field. If there are query string parameters your action accepts, describe them and how they are used by the controller.

Paginate collections.

In some rare exceptions, it’s not necessary to paginate a collection. For example, if you have an object model validation that eliminates the possibility that there could be more than ten objects in the collection, pagination is unnecessary. In almost every other case, though, it’s a requirement. Pagination sets an upper bound on database overhead, server memory, and processing time of every collection request. The additional development time on the server and client sides are well outweighed by the benefits to bandwidth, availability, and overall performance of your API.

Minimize and document hidden and/or triggered behavior.

There are many good reasons to introduce hidden or triggered behavior. Maybe you want to update an associated record when another is updated. Maybe you need to create stub placeholder records to satisfy data integrity when creating a central record. Maybe you want to trigger some background task to perform some aggregation function. Whatever you do, document it. If creation of a Song automatically creates a Playlist with a default name and adds the Song to the Playlist and you don’t document it, I promise you a developer (or a hundred) will email you to ask you why there’s a phantom Playlist in their account. They might even think they’ve been hacked. You can’t predict or overestimate the shit storm you will bring upon yourself if your API is not crystal clear and easy to use. Every line of documentation you write will save you support time.

Use OAuth.

If the purpose of your API is to give your users the ability to allow your app to interact with other apps on their behalf, you would be a fool to use anything but OAuth. It is the gold standard for B2B integration. Authorizations are completely controlled by the user. A user need only authenticate once to grant indefinite secure access to a certified 3rd party, and they can cancel the approval at any time. You will need to build the OAuth provider and a management portal for consumers to register, but as with pagination, the investment is well worth the benefit.

Test your controllers. Include all supported formats.

Most important of all, publish a functional API. If your API doesn’t work as advertised, your developers may jump ship before they even get settled in. I can say from personal experience (the inspiration for this piece, as it turns out) that trying to work with an API pales in comparison to actually working with an API. If my first impression of your API is “I did what your docs say, and it’s borked,” or worse still “I can’t figure out how to sign in because the docs say ‘to sign in, you must be signed in’ and provide no explanation of how to do that,” there’s a real chance I won’t contact you to work through it. Instead, I’ll move on to your competitor, or worse for you, I’ll build a competitor app.