Use cases for combined documentation of multiple targets in Swift-DocC

[taylorswift]

@andrewtheis

to know exactly which version of the docs to point to - whether that’s a version tag or commit SHA.

versioning does not guarantee that docs will be compatible. in order of increasing likelihood:

• remote servers may go offline

• remote servers may change their URL scheme, globally

• package authors may move repositories between organizations

• package authors may rename packages

• package authors may force-push to repositories, disconnecting commit hashes from the tree

• package authors may re-use the same semver tag for multiple revisions

the last two items are way more common than many people realize.

@andrewtheis

Alternatively, and what probably makes more sense, is documentation could be generated locally somehow and pushed up as part of the commits to the package itself. This would prevent the documentation from ballooning in size as a separate copy for each commit wouldn’t be needed, it would just diff alongside the rest of the source changes.

this is correct, and has already proven to be a major problem for the swiftinit.org docs. doing a back-of-the envelope calculation for swift-syntax, which has a ~60MB symbolgraph and about 500 release tags, would require approx. 30GB in storage! just for one package!

diffing symbolgraphs would first require the generated JSON to be stable, which it currently is not.

diffing symbolgraphs is also incompatible with text compression, which is currently our most effective means of compacting symbolgraph data. it would also require the JSON to be pretty-printed (right now it is emitted in minified form) for line-by-line diffing to be effective, which would further balloon the file size.

Biome has an application-specific stable symbolgraph format which is VCS-friendly and yields an approx. 8x improvement in size when uncompressed, and 2–3x compressed. but this is not sufficient (yet), because we would need an improvement by a factor of maybe 100x for long-term storage of versioned documentation to be feasable.

[ronnqvist (David Rönnqvist)]

Thanks for all the replies so far. They've both highlighted a few new use cases and brought attention to some important details for other use cases which is exactly the outcome that I'd hoped for.

It makes sense as a next step to continue with a more technical discussion about how to build this. I have more thoughts about this and am writing up a longer post with more information. I'm leaning towards posting that as a new topic to separate the technical discussion from the use case discussion. If you have more use cases for combined documentation of multiple targets, please continue adding them to this tread/topic.

[goncalo-frade-iohk (Goncalo Frade)]

Hi @ronnqvist. Hope everything is going well. Was there any update on this topic?

[ronnqvist (David Rönnqvist)]

After gathering use cases the discussion continued in this thread to talk about high-level design and implementation. After those high-level discussions I've had some conversations with people offline about more in-depth implementation details and prototyped a few future directions to explore areas where design decisions today could impact the ability to build on top of this. With the learnings from those discussions and prototypes we're shifting focus to implementing the initial feature set.

I'll let people know when it's in a state where they can try it out with their own projects.