Introducing Unidoc: scalable, multi-package swift documentation

hi all, i’m excited to announce Unidoc, a multi-package swift documentation engine, and successor to Swift Biome.


Unidoc is a scalable documentation engine for the Swift language. Unidoc can be thought of as a multi-target analogue to Apple’s local DocC compiler. It is designed for long-lived, centralized documentation indices that store, link, and serve multiple versions of documentation for many Swift packages at once.

Unidoc powers the Swiftinit open source package index!

Features

:feather: Small archive sizes

Unidoc servers are designed to store versioned documentation indefinitely. To achieve this, Unidoc uses a stable binary symbol graph format that can be up to two orders of magnitude smaller than an equivalent DocC archive.

Here’s a comparison for the (in)famous SwiftSyntax package, at version 508.0.1:

Archive Size File count
DocC (uncompressed, including synthesized symbols) 708.0 MB 84,619
DocC (uncompressed, stripping synthesized symbols) 155.0 MB 17,537
Unidoc (uncompressed, including synthesized symbols) 7.8 MB 1
Unidoc (tar.xz, including synthesized symbols) 611.4 KB 1

DocC numbers sourced from Slack

:arrow_up: Evolving documentation

You can regenerate Unidoc documentation from symbol graph archives without recompiling documentation from package sources, which historically was a major bottleneck in the DocC workflow. In many situations, this means you can easily upgrade Unidoc documentation to take advantage of new features even if the underlying symbol graph was compiled by an older version of Unidoc.

You can re-link Unidoc documentation against newer versions of their package (and standard library) dependencies as they are released, without recompiling the symbol graph from source.

Unidoc databases use a cellular architecture which allows you to stagger documentation upgrades across a package index without taking the server offline.

:link: Cross-package references

Unidoc can validate and resolve cross-package symbol links, including links to symbols in the standard library. This means you can link to String in your documentation, and Unidoc will automatically generate a link to the standard library documentation for String.

Normal “IDE-style” symbol references, such as links to Int within function signatures, are also supported.

:spider_web: Cross-package extensions

Unidoc can display extensions, including third-party extensions, directly in the documentation for the extended type. This means you can view Channel members originating from packages such as swift-nio-ssl and swift-nio-http2 from the swift-nio documentation itself.

In the future, we hope to support finer-grained control over third-party extensions shown in extendee documentation.

:revolving_hearts: Inherited symbols

Because Unidoc is a multi-package documentation engine, it can track and display symbols inherited from protocols in upstream dependencies, including the standard library, at negligible storage cost. This means types in third-party libraries that conform to protocols such as Sequence can display and link to Sequence API in their member lists.

:globe_with_meridians: Unified database

Unidoc servers maintain a combined database of all documentation in their index. This allows Unidoc to serve (or redirect) individual symbol pages on-demand, instead of requiring clients to download enormous Vue.js indices for client-side rendering. This provides better performance for clients, and greatly reduces cache churn on the server as documentation is upgraded.

:battery: Lightweight HTML

Unidoc generates lightweight HTML documentation that uses CSS for the majority of its layout and interactivity, and serves a very low number of additional assets. This means Unidoc pages are responsive, accessible, cache-friendly, and render with minimal content-layout shift (CLS).

:scroll: Readable signatures

Unidoc symbol graphs include line-breaking markers computed by SwiftSyntax, which allows Unidoc to display long function signatures in a readable, line-wrapped format. This makes it much easier to scan long lists of symbols with complex signatures, such as the member list of SwiftSyntax’s AccessPathComponentSyntax.

:flying_saucer: Documentation coverage

Unidoc can compute documentation coverage on a per-package and per-module basis. You can view coverage levels as pie-chart visualizations on package and module pages; see swift-atomics’s package page for an example.

:octocat: GitHub integration

Unidoc can periodically index Git tags by querying the GitHub API. This feature requires a GitHub app registration and app secret and is not enabled by default. Unidoc can also load repository metadata from GitHub, and use it to generate permanent links to source code on GitHub if the underlying symbol graph includes source map information.

The Unidoc compiler builds symbol graph archives with source maps by default, even if GitHub integration is not available.

:closed_lock_with_key: Secure administration

When GitHub integration is available, Unidoc can use social authentication to allow users to log in with their GitHub account and perform administrative actions.

As the Swiftinit index grows, we hope to allow package maintainers to claim ownership of their packages and manage their documentation directly through the Swiftinit website.

:mag_right: Search engine optimization (SEO)

Unidoc can generate, update, and serve sitemap files for search crawlers like Googlebot. This allows search engines to discover and index your documentation, and makes it easier for users to find your package.

Unidoc will make an effort to generate a <meta> description for every symbol in a package, even if the symbol has no documentation.

Unidoc avoids generating many copies of the same documentation, which can hinder visibility in search engines.


20 Likes

hi all, Unidoc 0.3 has been released!

along with improvements to query performance, Unidoc 0.3 includes a number of user-facing changes aimed at improving the usability of the rendered documentation.


:mountain_cableway: Per-symbol migration banners

The Unidoc server can now query successors for symbols in older (and prerelease) versions, and display a banner directing visitors to the symbol’s counterpart in the latest stable release of its package. This link is specific to the symbol, and comes with a corresponding <link rel="canonical"> element and HTTP header.

Example: https://swiftinit.org/docs/swift-nio:2.57.0/niocore/eventloopgroup

If the symbol does not exist in the latest stable release, Unidoc will only display the link to the package documentation root.

:telescope: Version management

Stale documentation is a perennial problem that afflicts all package indexes in the swift community, including swiftinit. Index sites can fail to serve up-to-date documentation for a variety of reasons, and these reasons are generally opaque to the reader.

  • The index may be unaware that a package it mirrors has released a new version.

  • The index may be backlogged by other releases in the documentation build queue.

  • The index may be unable to build documentation due to bugs in its documentation compiler, or even the swift compiler itself.

To increase awareness of new releases and decouple version indexing from the build infrastructure, Unidoc 0.3 now features deepened GitHub integration enabling Unidoc servers to track release activity in near-real time.

This information is now surfaced to the public through /tags dashboards, which can be reached from the landing page for the documentation of any package.

Example: https://swiftinit.org/tags/swift-asn1

:keyboard: DocC keyboard shortcuts

Unidoc’s front-end now supports the / keyboard shortcut for focusing the symbol search bar.

Although this is a relatively superficial change, it can be a significant ergonomic improvement, particularly if you have muscle memory from navigating DocC documentation.

image

The search bar can be defocused with the Esc key.

:open_file_folder: Collapsible ‘See Also’ sections

Very long ‘See Also’ sections are now collapsed by default. This is most relevant to packages with exceptionally bloated topics listings, such as SwiftSyntax.

Example: https://swiftinit.org/docs/swift-syntax/swiftsyntax/enumcaseparameterclausesyntax

:golf: Symbol disambiguation pages

Unidoc is able to serve symbol disambiguation pages under the 300 Multiple Choice HTTP status.

Although you should avoid creating ambiguous symbol links if possible, they are a natural occurrence as APIs evolve and overloads are added.

Example: https://swiftinit.org/docs/swift-certificates/x509/policyevaluationresult.failstomeetpolicy(reason:)

:sparkles: Miscellaneous changes

We fixed some symbol link parsing bugs in the Unidoc compiler, improved the performance of some documentation queries, upgraded the project to Swift 5.9, enabled HTTP/2 support, and hardened the security of some of the server’s administrative endpoints.

4 Likes