[Pitch] Swift Snippets

Evolution Pitches
, , ,
1

Hi everyone,

I've been prototyping work on Swift Snippets, which we previously discussed, and I'm excited to share a proposal pitch with you now.

Here is a link to the proposal text:

Swift Snippets Proposal

The proposal centers around a convention for writing snippets in a Swift package, building and running them, and using them in DocC documentation. There is certainly a lot more we can do with them and this is just a start. Let's hear your thoughts!

Trying it out

To try it out, I have a few branches that you can clone.

Swift Markdown: acgarland/snippets
This Swift Markdown branch has some example snippets and an article called Snippets. It uses an in-development branch, Swift DocC Plugin: acgarland/snippets, to build its documentation.

You'll need to use a fresh toolchain, or clone the main branch of SwiftPM and DocC. Here's a run-through of what you can do at the command line to try it out:

# Build SwiftPM
git clone https://github.com/apple/swift-package-manager swiftpm
cd swiftpm
swift build
cd ..

# Build Swift DocC
git clone https://github.com/apple/swift-docc
cd swift-docc
swift build
cd ..

# Get Prebuilt Swift DocC Render
git clone git@github.com:apple/swift-docc-render-artifact.git

# Build Swift Markdown docs
# Substitute your platform for `x86_64-apple-macosx` below as needed.
git clone --branch acgarland/snippets git@github.com:bitjammer/swift-markdown.git
cd swift-markdown

export DOCC_HTML_DIR=/path/to/swift-docc-render-artifact/dist
export DOCC_EXEC=/path/to/swift-docc/.build/x86_64-apple-macosx/debug/docc

/path/to/swiftpm/.build/x86_64-apple-macosx/debug/swift-package --disable-sandbox plugin preview-documentation --target Markdown --enable-experimental-snippet-support

From there, open the preview site (http://localhost:8000/documentation/markdown) and navigate over to the Snippets article, which I've temporarily placed as the first item in the Topics section.

22 Likes
2

This looks really great! I'm excited to start using these in docs.

I just had a few questions but I'm definitely a +1 here.

For the integration with Swift-DocC:

@Snippet(path: "my-package/Snippets/Snippet1")

The path argument consists of the following three components:

  • my-package : The package name, as taken from the Package.swift manifest.
  • Snippets : An informal namespace to differentiate snippets from symbols and articles.
  • Snippet1 : The snippet name taken from the snippet file basename without extension.

This path structure makes sense to me as an absolute link but I'm wondering if we should allow relative linking here? A DocC catalog is generally associated with only a single Swift Package so we should be able to infer the package name. And if I'm explicitly using an @Snippet directive, I'm not sure the Snippets namespace in the path should be required either. I like having the full absolute path as a measure of future-proofing (what if we want to allow linking to Snippets in a regular <doc:> link in the future for example), but it seems like we should support something like:

@Snippet("Snippet1")

in addition to

@Snippet(path: "package-name/Snippets/Snippet")

to better support the common scenario.

For the SwiftPM portion, I'm a little concerned about not exposing any controls for the "Snippets" directory.

SwiftPM will automatically find snippet files with the following pattern:

  • ./Snippets/*.swift
  • ./Snippets/*/*.swift

Should we be exposing API in the Package model that allows for customization of where this directory belongs? I like having the top-level Snippets directory as a default to match Tests and Sources. But with those other target types, I'm able to customize the directory using the path argument.

//! Line comments prefixed with ! will

I don't have a very strong opinion here but I wonder if this syntax will be confused with Objective-C and other C-based language's doc comment syntax. Someone coming from Objective-C might assume this is just another flavor of Swift's doc comment syntax (and interchangeable with /// and /**) when it's explicitly not.

Should we consider using something like //: instead?

Again, this is looking really great. Having the ability to actually regularly compile code examples that are shown in documentation is going to really impactful. (And I know this is just the beginning of what we can do with Snippets.) Thank you!

3 Likes
3

will snippets be highlighted?

will snippets have type identifiers resolved to USRs? what happens if multiple snippets declare types with the same name?

what happens if a snippet references or imports a module that is not a dependency of the project?

4

So you hit on just the thing about future proofing, specificallywhere modules might emit their own snippets. I'm actively looking into this. What would be the mechanism for communicating that package name? Would it have a more general meaning?

1 Like
5

I feel like this feature is unique enough to not need customization at this time, but I'm curious if anyone else has a need.

1 Like
6

I believe that's what Playground comments used, right? I think that would be a reasonable alternative.

2 Likes
7

Yes, they will have the same (syntactic) highlighting that other tagged code listings have in DocC. In the future, we could employ the compiler to perform semantic highlighting and provide tokens instead.

Semantic information isn't published with snippets at this time although we could add that. They are each their own executable target, so they are free to use whatever names they want.

They are built the same as any executable target in a package, so the import would fail without some new features to pull in dependencies on the side. Is there a particular use case you're thinking of here?

8

can snippets reference each other? if i am writing a tutorial, can i interject a paragraph in between two connected code blocks?

a common use of snippets in documentation is to demonstrate interoperability with other packages and APIs that are not actually dependencies of the project itself.

for example, an article author may wish to compare a project’s API to that of a competitor.

9

I'm not sure we would need to actually communicate the package name to Swift-DocC beyond the way we are today in the symbol graph. I'm imagining this working how relative links work in Swift-DocC today, but since we have the explicit @Snippets directive, scoped down to just Snippet symbols.

So as long as you don't have any conflicts (more than one Snippet from different Packages with the same Snippet name): during the link resolution phase, DocC should be able to resolve "Snippet1" to "package-name/Snippets/Snippet". But as soon as there is a conflict, you would need the full path or the link would fail to resolve.

10

Snippets need to be available in without surrounding context, so they can be quickly gleaned or appear in search results. They aren't fragments of a larger piece of code and meant to replace all "inline" code blocks. However, you could do the following in a tutorial-like article:

First paragraph

@Snippet(...)

Second paragraph.

@Snippet(...)

Or just use the snippet's description for the prose content, and just do:

@Snippet(...)
@Snippet(...)

It's best if each of those snippets were useful on their own, analogous to a flash card.

So I am super excited to see snippet-centric packages in general, actually. Sometimes there are interesting snippets on combining the capabilities of two libraries intentionally in ways that the original libraries' authors didn't anticipate, or there might be a sort of smorgasbord package of algorithms that leverage several packages.

Snippets should already have access to import libraries defined in the package itself, but there isn't yet a way to add dependencies just for snippets. Probably we could eventually add that. For this first version, I was trying to avoid changes to the manifest and tackle the case of enhancing package documentation but I am sure we will want to add the ability to use outside dependencies soon.

Since we're on that subject, how might you imagine adding those to the manifest?

1 Like
11

The idea is good but I'd like to make sure we can test snippets.

In my previous work all our code examples were in test code like this: akka/ActorDocSpec.scala at main · akka/akka · GitHub

And by surrounding various pieces of the code with the same #some-name one would include them into sources. This has the upside of them just being tests.

This needs to handle re-indenting the sources such that if the #name is inside some function for example, the indentation gets fixed to start "left most" when included etc.

7 Likes
12

But, with two different module's symbol graphs in the documentation context, how would you know which module to look for a snippet identifier?

For example, say I'm documenting the Markdown module, and the package is swift-markdown. There are two symbol graphs that go into DocC, one for the Markdown module which contains the regular library symbols, and one for the swift-markdown "module" (i.e. package). If I just say @Snippet("Snippet1"), what in the documentation context says which root in the topic graph to check?

13

it sounds like snippets would be expected to be very short, since longer runs of code would require prose to break it up. but at the same time a snippet is important enough to live in its own file, instead of being an inline code block embedded in a doccomment. so that suggests that snippets would be expected to be very long. how should we reconcile this?

i agree, this sounds like a good idea. would snippets be built by default when running swift build?

14

Hm that is a good point. I'll need to think about it a little more :smiley:

My initial thought is that, at least for this initial version, we can mark a topic graph as containing snippets or not (containsSnippets: Bool). Then in the common case, we would only have one valid topic graph to check when resolving snippets.

15

What part of a snippet would you want to test that isn't already tested in a library's tests? The snippets should reflect the same usage of public API. Can you give a more concrete example for a Swift library?

One thing I want to maintain and stress with snippets is that they don't require sewing together or thinking about multiple files. That said, you could hide assertions or perhaps even XCTest cases with the // MARK: Hide markers and simply run the snippet, or build/run them with a particular configuration to make certain regions of code active.

Absolutely, this is already implemented since it's possible to only show what was nested code in the original snippet source.

16

i don’t like testing snippets, or elevating them to the same level as the library’s actual tests. snippets can be a maintenance burden, and a library author may wish to take on some technical debt by allowing the snippets to lag a project’s API somewhat.

1 Like
17

Yes, that's right. Snippets are meant to be bite-sized and appear en masse. I don't think snippets living in their own file suggests how long they should be necessarily, though. I don't know if I would want to be rigid on this, authors might sometimes write a "long" snippet just by nature of some process being very long and serial.

I'm wondering if the dominant context in which these things tend to appear will be enough. For example, what I would love to see, sooner rather than later, is a package's snippets showing up on a package index online by default. Every package's doormat should get its own, personal goshdarnswiftui. Or, I could see a place to just search specifically for snippets and having them show up in the results in full length. Organically, maybe a feedback mechanism can let snippet authors know what readers think.

It's an interesting problem that might need a social solution rather than a technical one at the end of the day. If you think of anything, I'm all ears!

2 Likes
18

Yes, and that's the current implementation. A big benefit of snippets is that they at the very least don't go stale in the package.

3 Likes
19

How "snippets" work out in real big projects is that: unless a snippet breaks when something is changed in the implementation, such as behavior, then it won't be updated.

Snippets must be tested because they are even more important than tests - they are what people learn from. Is a behavior change makes a snippet bad, this is terrible because people learn the wrong thing then.

11 Likes
20

if a snippet imports a large framework, or a framework that takes a relatively long time to compile (like NIOWebSocket), will this clog the CI? will compiling with explicit swift build --target become commonplace?