Swift-DocC plugin for SwiftPM

In terms of CLI options specified with -, these aren't mutually exclusive, -O could just as well be a shorthand for --enable-all-optimizations, and -R for --recursive. It's not that hard to support both options. Swift Argument Parser (not sure if DocC uses it though) supports automated creation of shorthand variants out of the box.

But in terms of command naming, I definitely vote for doc. Given that the majority of tools for other languages already use it, this would follow the principle of least surprise. I don't think anyone would expect doc to abbreviate anything other than something related to documentation.

1 Like

This point makes no sense to me. Using Swift at all requires some level of English proficiency, if only from the keywords. Plus we have already have a number of commands with longer names. So I'm not sure how a command which has a clear English name is somehow worse than one that uses an abbreviation that doesn't even mean documentation when you look it up is better.

2 Likes

To some of the earlier comments, I feel strongly against shortening parameters and stripping them of their semantic meaning.

Looking at swift package generate-documentation it says exactly what this command is going to do and does — it generates the documentation of a swift package.

A command like go doc is indeed shorter to type in but it leaves a lot of unanswered questions — for example "what does this command do?". doc could be referring to a Go document, Go documentation, or Go doctor. Is this command creating a new Go document? This kind of syntax, if anything, discourages beginners.

It's likely that some audiences would enjoy short syntax like, say, tar -xcvf but it feels that a command being short for the sake of being a short command doesn't align well with the rest of Swift APIs and there is no immediate need for shorter command line parameters.

1 Like

Not that I would advocate for single-letter options, which are truly ambiguous and must be memorized, swift docs or swift package docs seem reasonable to me. It's possible I am being careless or inconsiderate, but I generally refer to "docs" and not "documentation", never mind the pluralization.

I don't understand why grammar comes into the picture for these things... this is a command line, not prose. I get that some want APIs to be complete phrases because they are read more than written. Surely writing is more prevalent at the terminal rather than reading and indicates a need for brevity, right?

"Computer, Earl Grey, hot." It's not an imperative verb but it gets the point across.

I know some may not agree on its importance but, speaking purely from a UX perspective, it is important to consider how a brief command feels, so long as it is the "right kind" of brief. When too long, it will feel like the operation will be more complex than it really is. If too short, it will feel downright cryptic (the tar scenario). Neither makes people feel like using Swift is a low resistance activity. I think folks who use other command line tools know that delightful balance of conciseness when they see it. This is something I have been considering a lot lately and involves some of the work I'm doing to make Swift easier to approach and learn.

5 Likes

I'm excited about this new plugin and am looking forward to making it work with package snippets so DocC can use them as code listings. I'm wondering if there will be a way for the plugin to depend on SymbolKit so symbol graphs can be generated before calling DocC.

Is there an intermediate executable that this plugin calls or does it eventually just execute the DocC command line? What work can the plugin perform before that call happens?

3 Likes

Hi @bitjammer!

I'm really glad to hear you're excited about the plugin. It would be great to get snippet support integrated here.

SwiftPM command plugins, in their current incarnation, cannot depend on other libraries. From SE-0332:

Since SwiftPM currently has only a single-layered package dependency graph, it isn't feasible in today's SwiftPM to allow a plugin to define its own dependencies on packages such as SwiftArgumentParser.

But the proposal does indicate this may be supported in the future. As it stands, a plugin can guarantee access to executable products that it depends on and executables that ship in the Swift toolchain.

The Swift-DocC plugin is essentially glue between SwiftPM and the docc executable that ships in the Swift toolchain. The plugin itself is a kind of intermediate executable that SwiftPM calls but there's no further indirection between the plugin and docc.

The plugin can ask SwiftPM to do work. So, in this case, it asks SwiftPM to generate a symbol graph for the specified targets and then passes those symbol graphs to docc.

All that to say, I think the best way to approach this would be to teach SwiftPM to generate the necessary symbol graphs and then (if necessary) augment the Plugin API so that when documentation plugins ask SwiftPM for symbol graphs, they can also request symbol graphs that include snippets. From what I can tell, your existing work on Emit snippets into symbol graphs by bitjammer · Pull Request #3953 · apple/swift-package-manager · GitHub should directly apply here.

Semi-related question @ethankusters -

if we wanted to tweak up the docs to include 'internal' and 'private' content - the kind of thing you might want to do for internally focused App docs - would it be acceptable to add a CLI option to this plugin to allow that?

I dug into the packageManager.getSymbolGraph, found the relevant options - and it looks like it already includes the capability within those options to ask for private symbols to be generated (swift-package-manager/PackageManagerProxy.swift at 744d13afed80ae3d73631051d052217bce9b2f16 · apple/swift-package-manager · GitHub). I'm guessing by maybe ammending or updating the options from what's returned by target.defaultSymbolGraphOptions(in: context.package).

Do you think that would be an acceptable path to enable the "include private" functionality?

1 Like

Interesting. It’s hard for me to see what the benefit of the plug-in is if all it can do is execute processes or call an API defined by the plug-in host. If you can’t add dependencies, what’s wrong with SwiftPM calling DocC itself, if it’s in the toolchain?

As for the dependencies, it poses some potential difficulties for utilizing SymbolKit, because adding a new link to SwiftPM seems potentially too disruptive, and parsing and understanding Swift code doesn’t seem to be under DocC’s umbrella, but then making yet another tool just to ask SwiftPM about its snippet targets seems a bit much.

1 Like

Hi @Joseph_Heck,

I think a CLI flag makes a lot of sense here. We should start with a bug on bugs.swift.org and a specific forums thread to work through the naming but I think that would be a great enhancement.

Long-term, I'd like to expose some kind of static configuration for this so you wouldn't need to pass it as a flag every time if, for example, your DocC catalog is specifically designed around internal framework documentation. But the question of where that configuration should live is a can-of-worms I'm still thinking about :slight_smile:

The plugin actually already does this to some extent, it will automatically include internal symbols for executable targets since its unusual for symbols in those targets to be marked public.

You can see that code here:

2 Likes

@bitjammer That makes sense. Would it be reasonable (or even feasible) to add a new executable tool to SymbolKit that can do this? The plugin would have access to any executables vended by SymbolKit if it depended on it. And SwiftPM command plugins have an understanding of the package model so the Swift-DocC plugin should know what files to point that hypothetical SymbolKit executable at.

For the same reasons as DocC, I don't think SymbolKit should be responsible for parsing for understanding Swift files.

Ah right. Agreed. :slight_smile:

An alternative would be to add an executable product to the Swift-DocC plugin that itself depends on SymbolKit. Then the actual plugin code can just invoke the executable.

At that point though- it seems like we should just create another Package that's dedicated to that purpose and have the plugin depend on that. That would also allow other tools (besides the Swift-DocC plugin) to build upon snippet symbol graphs.

Oh, so you can have other kinds of targets in the plugin package? I thought the entire package wasn't allowed to have dependencies. Hm, perhaps that's something to consider. It still seems strange to use an executable for use for that, though. We can revisit this soon though! Thanks for clarifying.

1 Like

Is this currently possible? Because I'd love for a plugin to call its own executable (even though it's sandboxed). That'd lower the barrier of integrating codegen such as sourcery by a lot.

Yeah, it is possible to make a plugin depend on a binary executable, it looks something like this:

// swift-tools-version: 5.6
import PackageDescription

let package = Package(
    name: "SwiftGen",
    targets: [
        /// Package plugin that tells SwiftPM how to run `swiftgen` based on
        /// the configuration file. Client targets use this plugin by listing
        /// it in their `plugins` parameter.
        .plugin(
            name: "SwiftGenPlugin",
            capability: .buildTool(),
            dependencies: ["SwiftGen"]
        ),
        
        /// Binary target that provides the built SwiftGen executables.
        .binaryTarget(
            name: "SwiftGen",
            url: "https://url/to/the/built/swiftgen-executables.zip",
            checksum: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
        ),
    ]
)

It's the SwiftGen example from the plugins proposal: swift-evolution/0303-swiftpm-extensible-build-tools.md at main · apple/swift-evolution · GitHub

3 Likes

Hi all! Quick update here.

The Swift-DocC Plugin has just been updated for the latest API changes in the SwiftPM command plugin infrastructure. I think a few of you noticed that the Swift-DocC plugin broke with the latest Xcode beta (CC: @Joseph_Heck), so if you update to the latest version on main, you should find things working again.


The SwiftPM command plugin UX has been updated so that options like --target and --product are passed directly to the plugin instead of to the package manager itself.

This means that instead of invoking the Swift-DocC plugin with something like:

swift package --target SwiftMarkdown generate-documentation

you would now do:

swift package generate-documentation --target SwiftMarkdown

Besides being (in my opinion) a generally more user-friendly command-line experience, this allows plugins to be a little smarter about choosing which targets to build. For example, now when a user invokes the basic

swift package generate-documentation

the plugin will generate documentation for all targets used in your package, including those defined by dependencies.

This also allows users of the plugin to specify products and targets of dependencies which makes doing something like

swift package --disable-sandbox preview-documetnation --product ArgumentParser

possible for packages that import ArgumentParser.

I think this will be great for improving the experience of learning about a new package you're depending on.

Please let me know if you run into any issues! As we approach the release of Swift 5.6, we're also working towards the 1.0 release of the Swift-DocC plugin so any and all feedback is very much appreciated.

- Ethan

7 Likes

The Swift-DocC Plugin has officially reached version 1.0 along with the release of Swift 5.6! :partying_face:

A special thanks here to everyone who provided feedback as we worked towards the first stable release. I'm looking forward to continuing to refine the plugin with you all in future versions.

In the meantime, you can add the the plugin to your own package with:

let package = Package(
    // name, platforms, products, etc.
    dependencies: [
        .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
    ],
    targets: [
        // targets
    ]
)

And start generating Swift-DocC documentation with:

swift package generate-documentation

As always, see the repository on GitHub and the documentation site for more information.

- Ethan

11 Likes

Amazing work, this is super exciting!

1 Like

Is it the current limitation that I cannot generate the documentation for my package that uses UIKit. While generating I get:

... error: no such module 'UIKit'
import UIKit

Same as when running swift build but in this case, I can pass options to properly build

swift build -Xswiftc "-sdk" -Xswiftc "`xcrun --sdk iphonesimulator --show-sdk-path`" -Xswiftc "-target" -Xswiftc "x86_64-apple-ios13.0-simulator"

I tried to pass the same params to generate documentation but it didn't help

swift package generate-documentation -Xswiftc "-sdk" -Xswiftc "`xcrun --sdk iphonesimulator --show-sdk-path`" -Xswiftc "-target" -Xswiftc "x86_64-apple-ios13.0-simulator"

Update:
After changing the order of the build options the build completes but the documentation is not generated (from Xcode works though)

swift package -Xswiftc "-sdk" -Xswiftc "`xcrun --sdk iphonesimulator --show-sdk-path`" -Xswiftc "-target" -Xswiftc "x86_64-apple-ios15.3-simulator" --allow-writing-to-directory ./docs generate-documentation --target UIEnvironment
Building for debugging...
Build complete! (0.09s)
Generating documentation for 'UIEnvironment'...
Building for debugging...
Build complete! (0.07s)
error: 'UIEnvironment' does not contain any documentable symbols or a DocC catalog and will not produce documentation
1 Like

To build docs for iOS you need to run xcodebuild:

xcodebuild docbuild \
    -scheme UIEnvironment \
    -destination 'generic/platform=ios'

Via Building multi-platform documentation with DocC | Daniel Saidi

3 Likes