OpenAPIKit

Pitch

Include OpenAPIKit in the official and recommended SSWG projects.

Motivation

OpenAPI is a broadly used specification for writing API documentation. OpenAPI documents can be used to generate interactive documentation, automate testing, generate code, or just provide a solid source of truth and a contract between a client and a server.

As linked above, a lot of great tooling already exists around the OpenAPI specification. The aforementioned code generator even supports Swift with improvements being actively discussed.

OpenAPIKit fits into the existing ecosystem as a relatively low level library, with the intention of supporting other libraries and tools on top of it. It currently captures nearly all of the specification in Swift Codable types. Thanks to Swift's type system, OpenAPIKit validates OpenAPI documentation simply by decoding it and it guarantees that OpenAPI documentation it encodes meets the spec as well.

In short, OpenAPIKit is a foundation for any Swift code that aims to read or write OpenAPI documentation. My hope is that this spec implementation saves time for others interested in writing tooling or frameworks with a higher level of abstraction.

Project Status

The OpenAPIKit library currently implements approximately 90% of the OpenAPI specification with approximately 95% test coverage. This includes substantial support for OpenAPI schemas, which are themselves close relatives of the very comprehensive JSON Schema specification.

Next Steps

The plan is to prioritize the following.

99% Spec Implementation

The first order of business is completing the implementation of the spec. To be perfectly honest, there are a few small things with particularly large time commitments attached that I will likely leave for later, perhaps needing to be motivated by request.

Decoding Error Legibility

This is a new addition, thanks to the comments from @lassejansen below.

The error output from failed attempts at decoding OpenAPI documents is currently often a bit of a mess. There is a lot that can be done, much of it without too much work, to improve that situation. Seeing as how reading JSON/YAML representations of OpenAPI documentation is a primary focus of this library, good error output should be a primary focus as well.

Canonical API Information

OpenAPI allows for an author to document the same API in numerous different ways. This flexibility can save time and offer convenience when authoring, but as someone consuming the documentation in order to transform it or analyze it somehow it can be cumbersome.

Easy examples of this flexibility include (1) the ability to define available servers at the top level of the document but also refine or add servers in the Path Items object and (2) the ability to define parameters in the Path Items object but also add them in the Operations object. OpenAPIKit should provide easy answers to questions like "what are all of the parameters for a given endpoint?" or "what is the full list of servers used by this API?"

Protocols to facilitate generating OpenAPI

I have begun to hone in on a set of protocols (and conformances for fundamental and standard library Swift types) to facilitate generation of OpenAPI schemas from arbitrary swift code. In addition, OpenAPIKit already provides a method for generating OpenAPI schemas from arbitrary swift types using reflection (this is the foundation on which response schemas are built for the Vapor example use-case below).

Example Uses

Neither of the first two examples are trivial as-is, but they would have been drastically larger undertakings without a Swift implementation of the OpenAPI spec on top of which to build.

Vapor API documentation generation

For those interested, I've created a proof of concept library and app using Vapor that takes advantage of OpenAPIKit to generate OpenAPI documentation from the same route code that serves up responses (granted, to truly take advantage of OpenAPI I needed to introduce some type information to the routes that Vapor does not require out-of-box).

JSON:API schema generation

Another example use-case of OpenAPIKit is this library -- it takes JSON:API types and generates OpenAPI schemas for them.

Personal side note: Perhaps ironically, I am a much bigger proponent of writing OpenAPI documentation first and then creating (or generating) endpoints that meet the spec. However, for small projects especially, it can be incredibly valuable to generate API documentation from the code instead of the other way around.

Writing OpenAPI documentation

Handwriting OpenAPI documentation may sound laborious, but I am actually not the least bit opposed to doing so in the right context -- in fact, I have written NodeJS tooling in the past to facilitate easy, repeatable, standardized OpenAPI documentation as part of contract driven API development. It's actually quite nice to write OpenAPI documentation using OpenAPIKit -- the declarative structure is reminiscent of YAML but you get type safety, declared constants without $refs, and reusability a la Swift.

9 Likes

I think it's a great idea to have a separate library that handles parsing, representing and exporting OpenAPI files. Looks like you already covered a large part of the spec!

I'm not sure though if it's a good idea to use Codable for parsing and generating files. My main concerns are (1) error messages when parsing yaml and json files, and (2) losing dictionary order of the source files.

(1) Error handling

OpenAPI files can get quite large and it's easy to make mistakes when editing them. In my opinion it's important to have meaningful error messages that contain the line number of the file where the error occurred. I'm not sure if that's possible when using JSONDecoder or YAMLDecoder + Codable.

Consider this minimal example:

openapi: 3.0.0
info:
  title: API
  version: 1.0.0
paths:
  /all-items:
    summary: Get all items
    get:
      responses:
        "200":
          description: All items
  /one-item:
    get:
      summary: Get one item

The error is that the second path (/one-item) must contain at least one response. The error at the moment looks like this (formatted for readability):

Swift.DecodingError.dataCorrupted(
  Swift.DecodingError.Context(
    codingPath: [],
    debugDescription: "The given data was not valid YAML.",
    underlyingError: Optional(Poly failed to decode any of its types at: "paths//one-item"

      JSONReference<Components, PathItem> could not be decoded because:
      keyNotFound(
        CodingKeys(
          stringValue: "$ref",
          intValue: nil
        ),
        Swift.DecodingError.Context(
          codingPath: [
            CodingKeys(stringValue: "paths", intValue: nil),
            _DictionaryCodingKey(stringValue: "/one-item", intValue: nil)
          ],
          debugDescription: "No value associated with key CodingKeys(stringValue: \"$ref\", intValue: nil) (\"$ref\").",
          underlyingError: nil
        )
      )

      PathItem could not be decoded because:
      keyNotFound(
        CodingKeys(stringValue: "responses", intValue: nil),
        Swift.DecodingError.Context(codingPath: [
            CodingKeys(stringValue: "paths", intValue: nil),
            _DictionaryCodingKey(stringValue: "/one-item", intValue: nil),
            CodingKeys(stringValue: "get", intValue: nil)
          ],
          debugDescription: "No value associated with key CodingKeys(stringValue: \"responses\", intValue: nil) (\"responses\").",
          underlyingError: nil
        )
      )
    )
  )
)

I'm sure the error message can be improved by evaluating the underlying errors, but I don't know if it's possible to add line numbers with this approach.

(2) Dictionary order

Personally I think it's important to preserve the order of the dictionaries in the source files. Consider the paths object. It often contains dozens of paths and operations and people tend to create a "semantic" order in the source file (e.g. first register, then login, then list all items, then create an item, the get an item, ...). Especially when generating documentation it's very helpful if this order is preserved. SwaggerUI will do this, and the Ruby library that I've used in the past preserves the order, too.

Another case is modifying OpenAPI files programmatically. If the order and formatting isn't preserved, a git diff will show large blocks of removed and added lines, even if only one item was changed by the program.

To be able to do this we would need event-driven JSON and YAML parsers I think, and an intermediate representation of the OpenAPI document that stores lines numbers and formatting and uses ordered dictionaries.

What do you think?

1 Like

Thank you, these are both incredibly useful points. In fact, they are both things I’ve noted (to myself) as shortcomings during various experimentation in the past but then forgot to bring up in my OP for this thread! Well, that’s what peer review is all about!

Error Handling
Now that you bring it up, I think “improving error legibility” deserves explicit mention in the “next steps” for the library. I recently underwent an effort to improve legibility of errors in a different library of mine and was thrilled with the results. However, I have not focused on line numbers in the past. I like the idea but am unsure off the top of my head how easy it would be to get there. I believe that even without line numbers the error output can be made easy enough to understand that someone at least familiar with OpenAPI nomenclature would quickly spot the problem.

For example, I’d consider the following human readable error message pretty darn good and totally within reach with a little more work: “responses key is missing for the GET operation under /one-item

Ordering
This actually should be readily solvable so I will create myself a GitHub issue and tackle it sooner than later. The trick will be using ordered dictionaries instead of dictionaries and neither the dictionary literal syntax (writing dictionary literals in Swift) nor the decoding process should stand in the way. [EDIT] This resulted in https://github.com/mattpolzin/OpenAPIKit/pull/8.

1 Like

Thanks for kickstarting this I thinks it's important to have a Swift building bloc for working with OpenAPI.

I use an OpenAPI code generator(Swaggen) at work regularly.

Splitting it in multiple part is the way to go.

What other part of it do you think should be part of a SSWG project ?

  • parser: what you are building
  • semantic: what you get is valid and proper OpenAPI
  • code generator: Template(with parser data) -> Swift
  • documentation generator: Template(with parser data) -> website
  • default template: Client, Server

How do you plan to test you parser(maybe these files could help) ?

Did you investigate existing tooling?

2 Likes

Good question.

Naturally I think the parser should be written in Swift and is a good candidate for the SSWG since we are discussing that here, but that's less a focus on parsing for me personally and more about having an easy-to-work-with syntax tree for OpenAPI that can be interfaced with directly from other Swift code. Being able to read/write (i.e. decode/encode, parse/generate) a JSON or YAML representation of OpenAPI natively in Swift is more of an ends to that means. Then, another thing that "just falls out of" an implementation based on Codable Swift types is the guarantee that if you can write Swift for a particular OpenAPI document then it will produce a valid OpenAPI document when encoded as JSON/YAML.

Code generation I think is a natural eventual horizon (for building on top of something like OpenAPIKit, not building into OpenAPIKit). This is not because the existing code generation tools built around OpenAPI aren't good enough, but because I think Swift code generation over time could benefit from leaning more and more heavily on things like SwiftSyntax.

Documentation UI generation seems less critical to me (as a native Swift project). Projects like Swagger-UI and Redoc and a few others I've looked into are doing a really good job of this already so as long as your OpenAPI documentation can eventually be represented as JSON/YAML, these tools strike me as a good fit for generating a user interface to the documentation. That said, I'm not about to tell someone not to write a native Swift powered UI. That sounds very cool, just less essential.

These are great. Thanks for digging up some good sources. So far I have tested my library against a couple of specs (one of them is my own, the other is proprietary to the company I work for) but having a corpus of specs to test against would be beneficial and I will not be surprised if a spec found under one of your links either exposes a bug in my current implementation or else proves to have a bug my current implementation finds. I'll work on a CI step that parses some of those specs and maybe asserts some things against the results in the near future!

I did look into some of what was out there. Most things I found did not take advantage of Codable or did not support OpenAPI 3 or were not a native Swift solution.

Kitura's OpenAPI support was for OpenAPI 2 (i.e. Swagger), if I recall correctly, but their Kitura-OpenAPI was inspiring as I began thinking about my VaporOpenAPI library (currently just a prototype/showcase, but I plan to develop it further in the future).

yonaskolb/SwagGen is a fantastic example of the sort of library I hope could benefit from SSWG adoption of a library like OpenAPIKit. I see that SwagGen gives attribution to a parsing library called SwaggerParser that appears to have had a similar goal to OpenAPIKit but did not make it past OpenAPI 2 support. The current OpenAPI 3 support in SwagGen appears to read JSON but not write it (after all, it doesn't need to write it out in order to generate code) and because it is not based on Codable it lacks YAML support without converting the YAML to JSON (in the case of OpenAPI, not a problem, just an inconvenience). Given the right implementation of a library like the one I am proposing here (emphasis on right intended to imply that my library is not necessarily the clear choice), SwagGen and others like it could skip over the substantial consideration of how to import the spec and just worry about the end goal (in this case generating code). I would be very interested to get feedback on my library's implementation from the creator of SwagGen.

openapi-generator, vapor-server-codegen, and swagger-codegen are awesome projects. I am really grateful users of those libraries have added support for Swift, but as projects not written in Swift, I think they do fall tangential to my motivation for OpenAPIKit. The fact that good code generation tools exist is one of the reasons I don't think there needs to be a native Swift code generation option immediately (but projects like SwagGen above are still very exciting to me).

Therein lies my thinking that more immediate examples of uses of OpenAPIKit might be OpenAPI generation (opposite direction from code generation), direct authoring of specs in Swift, or ingestion of the spec by Swift code to power whatever other behavior -- maybe a Swift app or Swift-powered server could accept an OpenAPI spec and automate some API tests or take the information from the OpenAPI spec and plug it into an integration config format native to that app (i.e. create an internal API binding by importing OpenAPI documentation instead of requiring manual setup). Then again, the existence of yonaskolb/SwagGen perhaps indicates native Swift implementations of OpenAPI code generators could be a more immediate desire than I had realized.

2 Likes

For example, I’d consider the following human readable error message pretty darn good and totally within reach with a little more work: “ responses key is missing for the GET operation under /one-item

Sound great!

This actually should be readily solvable so I will create myself a GitHub issue and tackle it sooner than later. The trick will be using ordered dictionaries instead of dictionaries and neither the dictionary literal syntax (writing dictionary literals in Swift) nor the decoding process should stand in the way. [EDIT] This resulted in https://github.com/mattpolzin/OpenAPIKit/pull/8.

Very cool!

This is sadly not the case for the Apple JSONDecoder as a side effect of it using JSONSerializer under the hood and that in turn using NSDictionary to back its keyed container.

This is something I ran into before, and a reason why I suggested not to use Codable. But you are right, that's an issue of JSONDecoder and JSONEncoder, not Codable itself.

I have not tried it yet, but one example of an alternative to the Foundation JSON decoder that does retain ordering is https://github.com/omochi/FineJSON. I wish it was more battle tested or at least had more unit test coverage but the underlying parser does appear to have better test coverage.

1 Like
Terms of Service

Privacy Policy

Cookie Policy