Update on Workspace’s Documentation Generation for Packages

I saw several posts around here wishing for a way of documenting packages (@martini, @yxckjhasdkjh), so I thought I’d post an update on the current state of Workspace.

Workspace can generate static HTML documentation for any Swift package.


Workspace is itself a Swift package. You can install it however you want, but make sure you do so with a tagged version (not master). (You can copy and paste this if you don’t already know what to do.)

Usage is straightforward:

$ cd Path/To/My/Package
$ workspace document

Set‐up is easy. The minimal configuration necessary for documentation is this:

// “Workspace.swift” in the package root.
import WorkspaceConfiguration
let configuration = WorkspaceConfiguration()
configuration.documentation.localizations = ["en"]

(The initial load of a configuration is extremely slow, but the result is cached (including intermediates), so subsequent loads are virtually instantaneous. SwiftPM support for target‐based resolution, product caching and mirroring will/can make a huge difference too.)


Documentation‐related features as of Version 0.23.1 include:

  • Written in Swift with packages in mind. Built on SwiftPM and SwiftSyntax. (Where relevant, pieces of functionality have even been upstreamed into SwiftPM at times.)
  • Fully supports Linux, in addition to macOS. (Its own documentation is self‐generated on Linux in Travis CI.)
  • Fully comprehends package structure: (See here for a complex example of all of the following.)
    • Documents which symbols come from which targets and products.
    • Documents only targets reachable through products, ignoring the rest.
    • Understands target dependency trees, and lists symbols inherited from package dependencies and core libraries. (But does not republish inherited documentation comments to respect copyright.)
  • High comprehension of Swift source:
    • Documents the #if conditions under which a symbol is available.
    • Understands overloading and leaves out optimized variants to reduce clutter.
    • Understands inheritance:
      • Groups symbols according to the superclasses or protocols where they originate.
      • Does not require conformances or overrides to be documented. (IDEs like Xcode simply reuse the parent’s documentation anyway.)
    • Protocol pages allow filtering conformance requirements, customization points or provided extensions.
    • Documents operators and precedence groups.
  • Supports some command line interfaces. If the package provides an executable product which uses SDGCommandLine (or just spoofs it), it will be included in the documentation. See Workspace’s own documentation as an example.
  • Fully supports localization. Symbols can be documented in multiple localizations or redefined in separate localizations. All pages are cross‐referenced between languages.
    • English (UK, US, Canada) and German are already directly supported. See Workspace’s own German documentation as an example. († Proofreading by a native speaker would be welcome.)
    • Arbitrary unsupported localizations can still be used, but receive only source‐like headings and navigation, such as saying only var where supported localizations would instead say one of Properties, Eigenschaften, etc.

Contributions are welcome at all levels, but please run any ideas by me in a GitHub issue first. One specific area where I could use some help is improving the appearance of the generated sites.

  • Right now, it just mimics the familiar documentation style of the Standard Library and Foundation. It would be grateful if someone more artistic than I am were to liven it up.
  • There is no way to customize the appearance (besides overwriting the CSS afterward). A means of customization would also be welcome if it can be designed well.
  • The CSS only really targets a computer monitor. Additions or adjustments to better support smartphones would be welcome.

Workspace is licensed under the same Apache 2.0 licence as Swift itself.

8 Likes

Thanks for the pointer. I just tried it and it doesn't work for me:

  1. It can only generate documentation for products. We have many targets (we're writing a very modularised application) and would want to avoid having to add every single one as a product.
  2. It outputs a number of "Loading inheritance from 'FooBar'" lines and then just fails with Illegal instruction: 4
  3. The workspace document command needs to download and compile the tool even though the CLI was already installed. That would be problematic for us because our CI builds are already quite slow and of course, docker only caches per-command (we also have bad experiences with our CI not really using the docker cache a lot and redoing everything from scratch, but that's a separate issue; if we could however frontload more of the effort by putting it into a base image, it would help at least). Presumably, the parsing would also take a while.

Thanks for the reply.

Yes. It is designed with vended packages in mind. In those cases you just want to publish your public API, not the symbols of internal modules. If that is not desired, making Workspace capable of documenting everything would be relatively easy; it would just be a matter of adding a configuration option that turns off the filter.

If this is a deal‐breaker for you, then you can ignore the rest of my response, which is mostly for the sake of other readers.

Not knowing what “FooBar” actually is, I cannot say for absolute certain. But Swift 5.1 introduced some bugs in Swift’s CommonMark, where it reports invalid UTF‐8 offsets that lead to crashes. It has been fixed (here and here) but the fixes haven’t made their way up through the dependency tree and into a release yet. In the meantime, as a workaround:

  • If it happens in a dependency (“Loading inheritance from...”), configure that dependency to be ignored (example, explanation). Doing the same with other dependencies can also have a massive effect on parsing speed, at the expense of dropping any inherited symbols.
  • If it happens in a source file of the package you are trying to document (“Parsing...”), it is harder to narrow down. Known triggers include:
    • Missing indentation in multiline Markdown callouts:
      /// - Note: ...
      /// ... ← Indent this and the problem goes away.
      
    • Unicode characters at the end of code voice:
      /// ...`...☺`... ← Drop the code voice and the problem goes away.
      

Using Workspace 0.22.1 or earlier with Swift 5.0 is always an option too.

It doesn’t recompile the entire tool. But it does compile the configuration file using the Swift package manager (which admittedly suffers from the continued absence of SE‐0226). Since the whole thing is cached, setting CI up to cache these directories between runs should make all the slowness disappear for subsequent runs.

† Unless you were applying the option •use‐version x.x.x, in which case that is exactly what you asked it to do.

1 Like

Hello,

I'm eager for alternatives to the jazzy documentation generator, so I ran workspace document on GRDB. It gives this error:

$ workspace document
Generating documentation... (§1)

Loading “Workspace.swift”...
error: the product 'WorkspaceConfiguration' requires minimum platform version 10.13 for macos platform
error: the product 'WorkspaceConfiguration' requires minimum platform version 10.13 for macos platform

✗ Failed to generate documentation. (See ⌘F “§1”)

Loading “Workspace.swift”...
error: the product 'WorkspaceConfiguration' requires minimum platform version 10.13 for macos platform
error: the product 'WorkspaceConfiguration' requires minimum platform version 10.13 for macos platform

Setup:

  • Fresh clone of Workspace (b3c1731)
  • MacOS 10.14.6
  • Xcode 11.1
  • Minimal Workspace.swift as described in the OP

Is it possible to avoid this error?

EDIT Apologies: version 0.23.1 does not suffer from this problem. It is running right now :slight_smile:

Yes. It needs to point at a released version’s configuration module. Whenever master contains breaking changes to the configuration module since the latest release, master has nothing compatible to point at, so master cannot load real configurations (only the mocks it uses for testing).

@JK_Kross (in answer to your question on a different thread).

At the moment Workspace presumes a package, but you can still temporarily add a dummy Package.swift to any repository in order to describe its structure. The package does not need to build successfully for Workspace to scan it for documentation. Workspace already contains the scanned interfaces of the standard and core libraries for the sake of inheritance, so you could look at that scanning code to figure out what paths the manifest needs to point at, and what files need to be generated first, such as with GYB. If you do scan the Swift project documentation that way, make sure not to publish the result anywhere without sorting out the whatever licensing might be involved, since it wasn’t you that wrote those documentation comments.