Swift-DocC plugin for SwiftPM

Hi all!

I’m excited to announce the development of a new Swift-DocC command, being built as a plugin for Swift Package Manager (SwiftPM). This new plugin will improve the experience when generating documentation for Swift Package libraries and executables from the command-line.

It’s been really great to build upon the awesome work the Swift Package Manager team has done with Package Manager command plugins to make integrations like this one possible.

The plugin’s repository is available here and is under active-development. We anticipate releasing a 1.0 version of the plugin aligning with the release of Swift 5.6. While it won’t be ready for production use until then, we would love your feedback if you’re interested in trying it out early.

Note that Swift 5.6 is required in order to run the plugin. Development snapshots that include Swift 5.6 can be found on Swift.org.

To use the Swift-DocC plugin with your package, first add it as a dependency:

let package = Package(
    // name, platforms, products, etc.
    dependencies: [
        // other dependencies
        .package(url: "https://github.com/apple/swift-docc-plugin", branch: "main"),
    ],
    targets: [
        // targets
    ]
)

You can then invoke the plugin from the root of your repository like so:

swift package generate-documentation

For more details on how to use the plugin to generate documentation for hosting online or how to preview documentation locally, see the plugin’s documentation hosted on GitHub Pages here.

38 Likes

Excellent, great to see one of the prime use cases of command plugins kick off :slight_smile: I honestly can't wait to move documentation of our open source packages to this, thanks!

On the usage side... I'll continue to bring up the theme of "all those commands are too long and scary looking" as I have during development of plugins themselfes, and other options for docc.

Really, why do all our commands have to look so complex when compared to other ecosystems:

Documentation IMHO is as important as unit tests for a good (open source especially) project for it to gain adoption and by simple changes such as engraining it deeper and "simpler looking" we give it a bit more of the gravitas it deserves.

Would you consider changing the command to swift package doc? One might argue for "docc" but I don't think that's good -- why would someone new to the language know or care about "that extra c" - it just looks weird and is an implementation detail for the task of "i want to get documentation".

I'm honestly somewhat hung up on this -- those long names we have for some tasks truly make them feel like "other" rather than "absolutely basic thing you should always do" to me, coming from all of the above example ecosystems.

I had raised it'd be excellent to be able to contribute top level commands as well during the plugin review, so we'd be able to get to the holy grail of swift doc eventually I'd hope, but I understand this is outside of this proposal's pitch and would be an SwiftPM discussion again.

22 Likes

Seems like this comes from Swift's tradition of being explicit about exactly what you're doing. In the IDE this is OK because we have things like autocomplete, but on the command line I can see the argument for terseness. That said, I might be wrong but I believe swift-argument-parser has some mechanisms for generating completions. I haven't used the myself ever but maybe if we make that more apparent/easier to take advantage of we can get the best of both worlds.

1 Like

Hi Konrad,

Really glad to hear you're looking forward to moving your open source packages to the plugin. :blush:

I definitely see the argument for supporting swift <documentation-verb> to align with swift test and swift build but agree that we should discuss that in a different thread as it's more of a general Swift Package plugins discussion.

For the specifics of the "generate-documentation" verb, my bias is towards slightly more verbose commands that are more readable but I can understand reasons for the opposite. There's a strong argument to just follow the precedent of other popular languages and support swift package doc. However, doc doesn't really make sense as a verb to me. I think I would prefer something like swift package build-docs to convey that this is a build action that produces documentation.

That being said, the "generate-documentation" verb is not defined by the plugin. The Swift-DocC plugin just declares itself as a documentation generating plugin in its manifest. The actual "generate-documentation" verb is defined by SwiftPM and affects all plugins that are declared with the documentation generation intent.

I think its definitely worth considering if we should adopt a more concise verb here but, within the current structure of SwiftPM command plugins, this wouldn't be specific to the Swift-DocC plugin.

2 Likes

Hmmm... I see, thanks.
I'll check in with some folks and hope we could improve the user experience here.

It is incredibly repetitive and just weird looking at all known to me other ecosystems to have such long names there, IMHO, so worth revisiting while we still can.

5 Likes

As someone who doesn't already know what those various doc commands do, they all seem like bad names. Do they generate docs? Do they view docs? Do they add docs to my code? Do they bring up my package's docs or the language's docs? What does it do? No way to know without investigating, where generate-documentation is much more clear. Perhaps SPM should have an alias system to allow those who prefer short commands to add short aliases.

6 Likes

Commands like generate-documentation will make PowerShell users at home🤣

And to take the experience from PowerShell, I believe the keys to better UX are: unified convention and helpful auto-completion.

Swift already has an API naming convention (proposed with the API Design Guideline), and that helps us read, memorize and distinguish various Swift APIs. I believe such thing will also help when it comes to the naming of SwiftPM plugins.

Better and wider auto-completion support will bring CLI experience to a new level. Rules like swiftpm gdswift package generate-documentation not only make a similar length to npm run doc, but are also more general and useful.

3 Likes

First off, kudos to the docc team for pushing this forward, looking forward to trying this out.

I agree with @ktoso that it is slightly verbose syntax at times. I understand that it is a bit out of scope for this thread, but just want to mention that the (to me) obvious bike shed spelling would be:

swift build doc
or
swift build documentation

it would preclude having targets with that name, but that seems to be a fair tradeoff for the ergonomics provided I think, and it removes the uncertainty @Jon_Shier pointed out of whether it is a matter of viewing or generating documentation.

Just 0.02c

1 Like

Could be a bit misleading as it’s not writing the documentation for you, but what about swift package document, as in “please give me documentation for this package”.

That’s worse imho since it’s an actual verb “document this package”.

Cool! I'll definitely check this out.

As for the command name, I agree with @ktoso.

CLI interfaces are full of abbreviations - and while it may not always be completely clear, I much prefer typing "cp -R" over "copyFile --recursive", or "mv" over "moveFile". Even with things like compiler flags, I much prefer the brevity of "swiftc -O" over "compileSwiftCode --enable-all-optimizations" or whatever it would be if we dogmatically chased clarity over brevity even here.

It's important to consider the audience - which kinds of people will even use the CLI interface, and for which purposes? And are those people likely to favour longer, verbose names, or abbreviated names in that context?

IMO, it's one of those things where, abstractly, sure - something like swift doc or swift package doc is not 100% clear. But if we went with that name, people who are used to CLI interfaces will appreciate it, and I expect we will get basically zero complaints about it.

Auto-complete is nice, but (in my experience) it's slow, doesn't work everywhere, and I'm not sure it's really practical for command plugins (where the set of available commands depends on the package you're in).

P.S. Also, it's worth remembering that verbose names are particularly difficult for users who do not have English as their first language. The name generate-documentation has very little meaning to them, and is just a pain to type; like if you were forced to repeatedly type the command name "apfelstrudel-mit-sahne"

10 Likes

In terms of CLI options specified with -, these aren't mutually exclusive, -O could just as well be a shorthand for --enable-all-optimizations, and -R for --recursive. It's not that hard to support both options. Swift Argument Parser (not sure if DocC uses it though) supports automated creation of shorthand variants out of the box.

But in terms of command naming, I definitely vote for doc. Given that the majority of tools for other languages already use it, this would follow the principle of least surprise. I don't think anyone would expect doc to abbreviate anything other than something related to documentation.

1 Like

This point makes no sense to me. Using Swift at all requires some level of English proficiency, if only from the keywords. Plus we have already have a number of commands with longer names. So I'm not sure how a command which has a clear English name is somehow worse than one that uses an abbreviation that doesn't even mean documentation when you look it up is better.

3 Likes

To some of the earlier comments, I feel strongly against shortening parameters and stripping them of their semantic meaning.

Looking at swift package generate-documentation it says exactly what this command is going to do and does — it generates the documentation of a swift package.

A command like go doc is indeed shorter to type in but it leaves a lot of unanswered questions — for example "what does this command do?". doc could be referring to a Go document, Go documentation, or Go doctor. Is this command creating a new Go document? This kind of syntax, if anything, discourages beginners.

It's likely that some audiences would enjoy short syntax like, say, tar -xcvf but it feels that a command being short for the sake of being a short command doesn't align well with the rest of Swift APIs and there is no immediate need for shorter command line parameters.

1 Like

Not that I would advocate for single-letter options, which are truly ambiguous and must be memorized, swift docs or swift package docs seem reasonable to me. It's possible I am being careless or inconsiderate, but I generally refer to "docs" and not "documentation", never mind the pluralization.

I don't understand why grammar comes into the picture for these things... this is a command line, not prose. I get that some want APIs to be complete phrases because they are read more than written. Surely writing is more prevalent at the terminal rather than reading and indicates a need for brevity, right?

"Computer, Earl Grey, hot." It's not an imperative verb but it gets the point across.

I know some may not agree on its importance but, speaking purely from a UX perspective, it is important to consider how a brief command feels, so long as it is the "right kind" of brief. When too long, it will feel like the operation will be more complex than it really is. If too short, it will feel downright cryptic (the tar scenario). Neither makes people feel like using Swift is a low resistance activity. I think folks who use other command line tools know that delightful balance of conciseness when they see it. This is something I have been considering a lot lately and involves some of the work I'm doing to make Swift easier to approach and learn.

5 Likes

I'm excited about this new plugin and am looking forward to making it work with package snippets so DocC can use them as code listings. I'm wondering if there will be a way for the plugin to depend on SymbolKit so symbol graphs can be generated before calling DocC.

Is there an intermediate executable that this plugin calls or does it eventually just execute the DocC command line? What work can the plugin perform before that call happens?

3 Likes

Hi @bitjammer!

I'm really glad to hear you're excited about the plugin. It would be great to get snippet support integrated here.

SwiftPM command plugins, in their current incarnation, cannot depend on other libraries. From SE-0332:

Since SwiftPM currently has only a single-layered package dependency graph, it isn't feasible in today's SwiftPM to allow a plugin to define its own dependencies on packages such as SwiftArgumentParser.

But the proposal does indicate this may be supported in the future. As it stands, a plugin can guarantee access to executable products that it depends on and executables that ship in the Swift toolchain.

The Swift-DocC plugin is essentially glue between SwiftPM and the docc executable that ships in the Swift toolchain. The plugin itself is a kind of intermediate executable that SwiftPM calls but there's no further indirection between the plugin and docc.

The plugin can ask SwiftPM to do work. So, in this case, it asks SwiftPM to generate a symbol graph for the specified targets and then passes those symbol graphs to docc.

All that to say, I think the best way to approach this would be to teach SwiftPM to generate the necessary symbol graphs and then (if necessary) augment the Plugin API so that when documentation plugins ask SwiftPM for symbol graphs, they can also request symbol graphs that include snippets. From what I can tell, your existing work on Emit snippets into symbol graphs by bitjammer · Pull Request #3953 · apple/swift-package-manager · GitHub should directly apply here.

Semi-related question @ethankusters -

if we wanted to tweak up the docs to include 'internal' and 'private' content - the kind of thing you might want to do for internally focused App docs - would it be acceptable to add a CLI option to this plugin to allow that?

I dug into the packageManager.getSymbolGraph, found the relevant options - and it looks like it already includes the capability within those options to ask for private symbols to be generated (swift-package-manager/PackageManagerProxy.swift at 744d13afed80ae3d73631051d052217bce9b2f16 · apple/swift-package-manager · GitHub). I'm guessing by maybe ammending or updating the options from what's returned by target.defaultSymbolGraphOptions(in: context.package).

Do you think that would be an acceptable path to enable the "include private" functionality?

1 Like

Interesting. It’s hard for me to see what the benefit of the plug-in is if all it can do is execute processes or call an API defined by the plug-in host. If you can’t add dependencies, what’s wrong with SwiftPM calling DocC itself, if it’s in the toolchain?

As for the dependencies, it poses some potential difficulties for utilizing SymbolKit, because adding a new link to SwiftPM seems potentially too disruptive, and parsing and understanding Swift code doesn’t seem to be under DocC’s umbrella, but then making yet another tool just to ask SwiftPM about its snippet targets seems a bit much.

1 Like

Hi @Joseph_Heck,

I think a CLI flag makes a lot of sense here. We should start with a bug on bugs.swift.org and a specific forums thread to work through the naming but I think that would be a great enhancement.

Long-term, I'd like to expose some kind of static configuration for this so you wouldn't need to pass it as a flag every time if, for example, your DocC catalog is specifically designed around internal framework documentation. But the question of where that configuration should live is a can-of-worms I'm still thinking about :slight_smile:

The plugin actually already does this to some extent, it will automatically include internal symbols for executable targets since its unusual for symbols in those targets to be marked public.

You can see that code here:

2 Likes