OpenAPI and Vapor/Kitura

After reading that a shared OpenAPI library is an area of interest for the SSWG I thought I would see what the general wants & must-haves were in this area. I am by no means an expert in the OpenAPI spec, but I've spent a fair amount of time with it in varying capacities and I have a lot of appreciation for what a full API spec can offer not just in documentation but also automated testing and a unified front on top of any number of micro services that need not even be written in the same languages much less using the same technology stack.

I would be interested in contributing to such a shared library, potentially. Although I doubt my personal goals of the past align perfectly with everyone's, I may even have some code that could be contributed (with whatever modifications made it more of a general-purpose fit). At this point I have a pretty well fleshed out Codable implementation of the spec (https://github.com/mattpolzin/OpenAPI) and I am fairly proud of how well the spec's rules are represented at the type-level. I've spent a little time making it read alright declaratively, a little time thinking about interfaces that allow other types to generate OpenAPI components, and quite a bit of time just getting the majority of the spec built out (including most of the stuff borrowed from JSON Schema).

Again, I don't think my library is likely a good fit for pitching to this group, but perhaps some of my hard work is not too far off the mark to be useful in saving some time for whatever shared implementation is on the horizon.

Excited to hear what others are interested in with respect to OpenAPI integration into server side projects!

10 Likes

Thanks for sharing and for taking the time to build this. I'm also specifically enthusiastic about using formal API descriptions to avoid hand-writing API interface code. We might say a good API code generator would be like an "Interface Builder for Data."

But in practice it's hard. Until recently I led a team at Google that wrote code generators for gRPC APIs. gRPC APIs are described with the Protocol Buffer language, which is generally more opinionated than OpenAPI and explicitly designed for code generation. Even with that, it took a lot of effort to generate clients that came close to the usability and quality of handwritten API wrappers.

I've found OpenAPI clients to be harder because a broader range of things can be expressed in OpenAPI and some API descriptions aren't well-suited to code generation. Also many APIs don't have definitive OpenAPI specs, so the specs that we have are consumer-written "fan fiction" that are often incomplete or incorrect.

Despite this, I've seen talks from people at Microsoft, IBM, and Oracle who've used OpenAPI code generators successfuly by making sure their API descriptions follow strict style guides. IBM has Swift clients for the Watson APIs that I believe are generated with a variant of swagger-codegen.

Matt, you might find it helpful to find a way to automatically compare OpenAPI representation against a JSON Schema of the spec. To my knowledge, there's not an official JSON Schema for OpenAPI 3, but here's a schema that I generated by scraping the Markdown OpenAPI spec and used to generate a Protocol Buffer representation for a Go tool that I wrote.

Regarding client generation, I think it could be helpful to collect some "model" handwritten clients for APIs with known OpenAPI descriptions so we can work backward to write generators. Along with making HTTP requests and message serialization, asynchrony and error/retry handling are important, often overlooked, and usually need lots of review to get right.

p.s. If you're making and publishing an API, please make it follow a formal description! Going spec-first is a great way to do that.

4 Likes

Thanks for sharing your experience! Your post talks a lot about generating clients from OpenAPI specs, which in my opinion (and yours, clearly) is the "right" way to go about it, as opposed to generating OpenAPI from a client's code, which makes the code the source of truth instead of the contract between the server and client. That being said, the nature of OpenAPI and the respective tasks does make it much easier to generate some form of valid OpenAPI documentation from a particular client codebase than it is to generate some form of valid client codebase from any valid OpenAPI. My priorities for my own OpenAPI project, based on complexity mostly, are roughly:

  1. Finish my implementation of the spec
  2. Provide hooks for generating OpenAPI from Swift,
  3. Provide hooks for generating Swift from OpenAPI

Of these three, I am about 90% there on (1) and I've already done quite a bit of preliminary work on (2). I am of course aware of efforts like those described in the other OpenAPI thread that provide (3) via existing tooling, which I think is great, but does not particularly demotivate me from working towards 100% swift implementations.

I'm also specifically enthusiastic about using formal API descriptions to avoid hand-writing API interface code [...] But in practice it's hard [...] because a broader range of things can be expressed in OpenAPI.

This is definitely difficult, and in particular with OpenAPI, as you point out as well. Not only can OpenAPI represent a broad range of things, but it represents those things in a relatively flexible way. For example, realistically any given endpoint has 1 set of possible parameters it accepts and yet OpenAPI lets you define those parameters in multiple locations. The flexibility makes the spec more usable as an author, but more cumbersome as a consumer.

One of my express goals for my own OpenAPI project is to give access to "canonical" representations of OpenAPI components (i.e. a new endpoint structure that would be guaranteed to fully encapsulate the spec for a single endpoint). This will come after I have finished a full representation of the spec (which I am nearing completion of). At that point, it at least becomes easier to write a tool that does whatever with the various OpenAPI components.

you might find it helpful to find a way to automatically compare OpenAPI representation against a JSON Schema of the spec

To my knowledge, the reason there is not an official JSON Schema representation of the spec is that JSON Schema cannot properly represent all of the rules in the spec -- I don't have any examples prepared, I've just read this before. Regardless, this may be a moot point for the OpenAPI implementation I have here because I have represented the rules of OpenAPI natively in Swift types so there are very few places where it is even possible to represent invalid OpenAPI using my library; that, combined with Codable support, means that the act of decoding an OpenAPI document using my library provides the same validation you would be seeking by matching a document against a JSON Schema representation of the spec. That being said, the human-readability of error reporting upon failure to decode in my library is an area for improvement.

generate a Protocol Buffer representation for a Go tool that I wrote

This is really cool! I've not had time to dive into protobuf so it's all a bit foreign to me, but that looks impressive.

2 Likes