Use cases for combined documentation of multiple targets in Swift-DocC

[taylorswift]

@andrewtheis

to know exactly which version of the docs to point to - whether that’s a version tag or commit SHA.

versioning does not guarantee that docs will be compatible. in order of increasing likelihood:

• remote servers may go offline

• remote servers may change their URL scheme, globally

• package authors may move repositories between organizations

• package authors may rename packages

• package authors may force-push to repositories, disconnecting commit hashes from the tree

• package authors may re-use the same semver tag for multiple revisions

the last two items are way more common than many people realize.

@andrewtheis

Alternatively, and what probably makes more sense, is documentation could be generated locally somehow and pushed up as part of the commits to the package itself. This would prevent the documentation from ballooning in size as a separate copy for each commit wouldn’t be needed, it would just diff alongside the rest of the source changes.

this is correct, and has already proven to be a major problem for the swiftinit.org docs. doing a back-of-the envelope calculation for swift-syntax, which has a ~60MB symbolgraph and about 500 release tags, would require approx. 30GB in storage! just for one package!

diffing symbolgraphs would first require the generated JSON to be stable, which it currently is not.

diffing symbolgraphs is also incompatible with text compression, which is currently our most effective means of compacting symbolgraph data. it would also require the JSON to be pretty-printed (right now it is emitted in minified form) for line-by-line diffing to be effective, which would further balloon the file size.

Biome has an application-specific stable symbolgraph format which is VCS-friendly and yields an approx. 8x improvement in size when uncompressed, and 2–3x compressed. but this is not sufficient (yet), because we would need an improvement by a factor of maybe 100x for long-term storage of versioned documentation to be feasable.