Micro-service APIs With Some Swag (part 1)

Every aspect of the API matters to some Client.

Jim des Rivieres, Evolving Eclipse APIs

It is fascinating that the quote above is 14 years old now. It was coined by the Benevolent Dictator of Eclipse APIs Jim des Rivieres in the days when we defined how Eclipse Platform APIs were to be designed and evolved. Of course, APIs in question were Java, not the REST variety that is ruling the API economy these days. Nevertheless, the key principles hardly changed.

Last week when I wrote about the switch to micro-services by SoundCloud, I noted that APIs are predominantly a public-facing concern in monolithic applications. There is no arms-length relationship between providers and consumers of functional units, enabling a low-ceremony evolution of the internal interfaces. They are the ‘authorized personal only’ rooms in a fancy restaurant – as long as the dining room is spotless, we will ignore the fact that the gourmet meals are prepared by a cute rat that sounds a lot like Patton Oswald.

Put another way, APIs are not necessary in order to get the monolithic application up and running. They are important the moment you decide to share your data with third-party developers, write a mobile app, or enable partner integrations. Therefore, monolithic applications predominantly deal with public API.

Things are much different for a micro-service based distributed system. Before any thought is put in how the general public will interact with the system, micro-services need to figure out how they will interact with each other.

In the blog post about Node.js clustering, I pointed out that Node is inherently single-threaded, and clustering is required just to stretch to all the cores of a single server, never mind load balancing across multiple VMs. This ‘feature’ essentially makes clustering an early consideration, and switching from vertical to horizontal scaling (across multiple machines) mostly a configuration issue. Presumably your instances have already been written to share-nothing and do not really care where they are physically running.

Micro-service APIs are very similar in that regard. They force developers to start with a clean API for each service, and since a complex system is often built with several teams working in parallel, it would turn into a total chaos without clean contracts between the services. In micro-service systems, APIs are foundational.

Internal APIs – an oxymoron?

In the previous post, I put forward a few rules of writing a micro-service based distributed system that concern APIs. Here they are again:

• Rule 3: APIs should be the only way micro-services talk to each other and the outside world.
• Rule 4: Internal APIs should be documented and otherwise written as if they will be exposed to the open Internet at any point.
• Rule 5: Public APIs are a subset of internal APIs with stricter visibility rules, rate limiting and separate authentication.

The aforementioned Jim des Rivieres used to say that “there is no such a thing as internal API”. Interfaces are either firm contracts exhibiting all the qualities of APIs or they can change at any time without warning. There is no mushy middle ground. I tend to agree with him when it comes to monolithic systems, where ‘internal’ refers to ‘written for systems’ internal use only’. However, in distributed systems ‘internal’ refers to traffic between services, or between systems behind the firewall. It is more to do with ‘things we say to the members of our own family’, presumably versus ‘things we say to the outside world’.

In this context, ‘internal APIs’ is a legitimate thing because ‘internal’ refers to the visibility rules, not the quality of the API contract. Rule #4 above explicitly states that – there is nothing different about internal APIs except visibility.

Presenting unified API front

If APIs are the only way micro-services should communicate with each other and the outside world, the consumers need to be presented with a cleanly documented contract. Documenting the APIs cannot be an afterthought – it needs to be built with the micro-service, sometimes even before the documented endpoints actually work.

The fact that our distributed system is composed of micro-services is a great feature for us and our ability to quickly evolve and deploy the system with little of no downtime. However, API consumers can’t care less about it – they want one place to go to see all the APIs.

There are multiple ways of skinning that particular cat, but we have decided to do as follows:

1. Proxy all the APIs to the common path (e.g. https://example.com/api)
2. Expose the API version in the URL (I know, I know, we can yell at each other until the cows come home about how that is great or stupid, but many popular APIs are doing it and so are we). Thus the common path gets a version (e.g. https://example.som/api/v1)
3. Reserve a segment after the version for each micro-service that exposes APIs (e.g. /projects, /users etc.).
4. Provide API specification using a popular Open Source API doc solution

On the last point, we looked around and considered several alternatives, finally settling on Swagger by Wordnik. It is a popular solution, with a vibrant community, fairly well defined API spec, a reusable live API UI that can be included in our UI, and with a path forward towards version 2.0 that promises to address currently missing features (the current version is 1.2).

A micro-service based system using Swagger to define APIs could look like this:

Each micro-service that provides APIs will make a Swagger API doc resource available, describing all the endpoints, verbs, parameters and request/response bodies.  Documentation micro-service can render these in two ways – using Swagger Live UI and rendering static docs.

Swagger Live UI is available as an Open Source project and allows users to not only read the rendered documentation, but enter values and try it out in place. To see it in action, try out the Pat Store sample.

The UI is all client side, which makes it stack-agnostic and fit for being served by a multitude of platforms, but if you are aggregating your definitions like we do, you need go around browser’s Single-Origin limitation. You can either proxy the API definitions or use CORS. In our case, it helps that we proxy all the services to the single external URL root, which is on the same domain as the doc UI – problem solved.

I can stop now while I am ahead – this being the part 1 of a multi-part article. In the next installment, I will walk you through an example of two micro-services – one providing API for Projects, another for Users. We will spec out the API, document the spec using Swagger, write a Node.js app to serve the UI from these definitions, and also render an alternative static version of the API doc.

See you next week, off to write some API micro-services.

© Dejan Glozic, 2014