Swift Package Index: Auto-generating, Auto-hosting, and Auto-updating DocC Documentation

Congratulations @finestructure! When finding and publishing documentation is this easy, there's no reason not to write great docs! This is huge step towards better documentation for the whole Swift community.

I'd be curious to hear your thoughts around documentation search—this is another area I think DocC could improve.

4 Likes

the goal of the swiftinit.org project was to serve ecosystem docs essentially like the feature the SPI just rolled out, but in a more integrated fashion. i had been working on it day-and-night for approximately 6 months, but the project hit major implementation snags several weeks ago when i tried to add support for versioning and documentation history, as this is a surprisingly deep and convoluted problem when it interacts with extensions on cross-package types. when Sven posted his announcement, i felt that my work had been made obsolete, which is why i did not welcome his announcement in the way that i should have.

I apologize to Sven and his teammates.

5 Likes

Sorry to hear that you felt like your work was made obsolete, @taylorswift ! Of course that wasn't our intention and we weren't aware of your intentions for the site. When we spoke at the Documentation WG meeting our plans were already well formed and it just so happened that the gap between meetings was large enough for us to complete the first part of this feature.

I believe there's merit in being able to link between packages but our goal wasn't/isn't to solve that at the package index level but rather adopt whatever linking mechanism DocC itself may support at some point.

I also think that while cross-linking might be interesting for some packages by specific authors, I suspect the vast majority of packages will at most want to link to standard library documentation and so are naturally siloed.

An easy way to see this is by looking at the long tail of packages that import little to no dependencies.

3 Likes

I believe the DocC version coming with Swift 5.7 will mostly take care of that by adding a sidebar with a search field to the documentation viewer.

There's a larger question around how that might integrate with the rest of the site but at least initially that should help navigate the docs for a given package.

1 Like

i understand. to be honest if i were you i also would have forged ahead. i had been so caught up in the stress of first-to-publish it didn’t even occur to me that you might have felt similarly pressured.

correct. this was the sinking realization that i never really confronted until today, that the silo-ing shortcut ended up being the winning strategy, if only because it makes the problem of serving package documentation tractable.

right, in hindsight i was spending all my time solving a problem that doesn’t really exist in the real world, probably because i maintain a ‘deep’ stack of dependencies that is not representative of how a majority of swift developers work. you had a better understanding of the market than i did.

i’m not really one to cry about getting outmaneuvered, so i suppose all i can do is congratulate you on a project well-planned. i certainly learned a lot from this experience…

3 Likes

Huge congratulations on launching @daveverwer @finestructure! The ability to find packages and quickly jump to their documentation is really delightful.

+1 on Ethan's point about not requiring packages to explicitly include the Swift-DocC plugin—this would reduce friction in publishing documentation even more. And as discussed with Dave and Sven, integrating the information declared by the .spi.yml file (i.e., what targets to build documentation for) into the package manifest directly would be a natural step forward. Maybe there are some sensible defaults we could use here, e.g., always publish documentation for products, but not tests.

Just some food for thought, Swift-DocC has an --emit-digest flag that produces an index of the documentation content available in a DocC archive. It would be neat to use that info to allow Swift Package Index's search field to also search for symbols within packages.

Big congratulations and I'm very excited to see more documentation!

4 Likes

Thank you, Franklin :relaxed:

We’ll be following up on those points you mentioned soon!

1 Like

Is there a way to link to existing documentation?

For example, I use a custom DocC-render and host the docs on GitHub Pages. Can I also get that "Documentation" button, and have it point to my existing URL?

That said, I also don't mind if SPI wants to render and host its own version of the docs. I don't see a downside to enabling it, especially if there might be better support for versioning or other benefits in the future. I don't use the plugin, though, so if I could notify SPI about the presence of DocC-compatible documentation solely through the YAML file, that would be nice.

1 Like

One of the next things we'll do is allow authors to configure the "Documentation" button to link to their own docs instead of having us generate and host it.

5 Likes

We haven't updated our docs/the blogpost yet but we've now updated the process to make the plugin entirely optional :)

5 Likes

The blog post is also now updated!

Thanks for everyone's kind words, too! We're really happy this has gone down well.

1 Like

Hello,

Now there are two online documentations about DocC itself:

I'm not sure how uncomfortable people feel with this.

  • Which url is canonical?
  • Which url should I use when I want to link to the documentation?
  • Which url contains the documentation of the latest stable release? Or is it the documentation of the main branch? There's no versioning visible anywhere.
  • How do search engines such as Google deal with such duplicated content?

I, personally, am confused. I know you all want the best, but do you have an opinion about those concerns?


EDIT - this is the same with TCA (and maybe others):

Sorry for those questions: they are already answered in the "Next Steps" paragraph of the announcement!

1 Like

A small suggestion

Now there are two online documentations about DocC itself:

It should be

Now there are two online documentations about DocC itself:

The developer Documentation site Apple Developer Documentation is not about swift-docc's doc itself, but some examples of docc usage.

For an example of Swift-DocC in action, check out developer.apple.com. Much of Apple's developer documentation, from Reference documentation to Tutorials, is built using Swift-DocC.
The latest documentation for the Swift-DocC project is available on Swift.org.

From README.md of swift-docc repo

1 Like

A small question:

  • swift-docc-symbolkit and many other repos does not have .spi file. So SPI does not build documentation for them.(Expected)
  • swift-docc and swift-markdown does not have .spi file. But SPI builds documentation for them.(Unexpected)

Is there a whitelist or something to control the documentation on SPI besided .spi file?

1 Like

Hey Gwendal!

All good questions, but I think they’re mostly answered by this feature being opt-in by package authors. If we don't find a .spi.yml file with the relevant keys, we don't generate docs.

As Sven mentioned above we’re going to be adding a feature where authors can tell us where their official docs hosting is and we’ll send people there. We don’t yet have issues created for everything that’s going to make up the “completed” feature (is there ever such a thing?), but this is certainly one that we’ll implement.

Yep, versioning is really important and as soon as we have it done we'll be switching the default "Documentation" link for a package to show the latest stable docs, rather than those from the default branch.

Well spotted!

In cases where authors prefer not to add the .spi.yml file to their repository and would still like to have their docs hosted by us we have a mechanism in place to do so.

1 Like

for completeness, swiftinit.org also hosts package references which overlap with the ones served by swiftpackageindex.com, swift.org, and apple.com. many projects like SwiftNIO and Vapor also host their own package-specific references.

although basic historical archiving is straightforward to acheive, true documentation versioning is extremely difficult to implement when you add URL stability as a requirement.

moreover, certain packages like swift-syntax contain both a large number of symbols, and a large number of tag releases, which makes indexing and serving this kind of content a significant engineering task in and of itself. although most packages do not vend enough API or release often enough to strain the documentation service provider, a handful of “fat” packages like swift-syntax are currently blocking this feature, at least for swiftinit.org.

i believe this was previously discussed at the last meeting of the documentation working group.

Just to update this thread as well: we've shipped versioned documentation last week. Please find out more in our announcement blog post!

6 Likes