SE-0356: Swift Snippets

Evolution Proposal Reviews
1

The review of SE-0356: Swift Snippets, begins now and runs through May 16, 2022.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager or direct message in the Swift forums).

What goes into a review of a proposal?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift.

When reviewing a proposal, here are some questions to consider:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at https://github.com/apple/swift-evolution/blob/master/process.md

Thank you for helping improve the Swift programming language and ecosystem.

Tom Doron
Review Manager

17 Likes
2

This is looking great and will be a great addition to the Swift ecosystem. The integration with Swift-DocC looks really neat and is a great solution for ensuring that code snippets always build.

Could the proposal go into more detail into how you use API from other targets in the package? You mentioned in [Pitch] Swift Snippets - #34 by bitjammer that targets need to be imported, so I think the proposal should reflect that since this will be a common workflow.

1 Like
3

I'm not entirely sure I understand the question as referenced in the pitch thread. Snippets are just executable targets–if there is a library target Foo in the same package, you just write import Foo in the snippet like any Swift file, nothing more than that. The only thing that is different about snippets in this regard is that you don't need to manually write dependencies in the manifest.

For future scenarios like a snippet-only package that wants to link against libraries outside the package and SDK, there will be a need for something explicit to tell SwiftPM to download a dependency from somewhere. Until then, in terms of importing other APIs, they are no different from executables.

4

I expect most snippets to import the library they're demonstrating, so keeping import statements in the snippets may add noise when rendered in documentation, since they'll appear in every snippet by default.

For example, when writing snippets for the Markdown library, will the line import Markdown appear in each of my snippets, and I need to hide them using MARK: Show and MARK: Hide? For example:

// MARK: Hide
import Markdown
// MARK: Show

someMarkdownAPI()

Have you considered ways to automatically hide these import statements?

3 Likes
5

+0.75

I would like to see Snippets be explicitly mentioned in the Package manifest, and allow authors to specify the path, much like we can our Sources/ and Tests/ directories - I'm not a fan of starting to have a mix of bespoke required paths and others being overridden.

It's something I'm currently unhappy with DocC, and I don't want to see us carrying it forward

8 Likes
6

Yes. It’s important to keep in mind that snippets won’t necessarily always be embedded in a framework documentation context, even ones that were written to illustrate a single framework. For example, they might be embedded in another site. In another example, snippets may show up in search results that don’t necessarily navigate to a page.

In addition to that, it’s important for a reader to know which modules are necessary for the code to work, especially when it involves more than one framework or package. I view imports as a part of the snippet.

It might seem repetitive when there are a bunch in a row, but I would rather wait and see on this. It might end up being be a thing that is automatically hidden when rendered in certain contexts. If it really becomes important, we can even move import statements into their own section in the data model.

1 Like
7

I'll toss in a note that I also prefer to have the explicit import for examples such as these. In some cases it can be repetitive, but for any more complex examples (such as when you're using more than one framework), its critical. For those cases where it's brutally repetitive and adding no value, there is the option to hide that content from anything exposed in DocC using // MARK: Hide and // MARK: Show as @franklin showed earlier, which seems reasonable to me.

Additionally, it makes it clear what the name of the framework is you need to import. Most Apple frameworks match names 1:1 (a notable exception being UniformTypeIdentifiers), but outside of the Apple provided frameworks the names of the packages to import aren't always matching so cleanly.

2 Likes
8

Overall evaluation +1

@Mordil has a great point that swift tooling overall is bouncing back and forth between convention and explicit configuration, and much of the DocC oriented tooling has leaned more heavily into convention. That said, the convention chosen for where to land Snippets isn't onerous to me, and I prefer having it explicitly outside of either Sources or Tests. I lean towards having a convention as a good default, and allowing it to be specified. That addition would be an improvement, but I don't see it as a blocking failure.

Prior to the snippets proposal, I was leveraging Tests for verifying example content worked as expected, which came with the additional manual burden of replicating content into DocC articles or references as needed. This proposal (and initial implementation) improves that situation significantly, providing a place for examples that I can verify both work and continues to operate as expected as the library evolves. It also has the added benefit that I don't need to copy content all over the place (into articles, or reference doc areas) to expose the examples in documentation.

My previous experience with this sort of tooling has been primarily with Python, which had some of the most literate examples, but suffered with being awkward to describe any more complex use cases, which this proposal deals with by having example be its own small executable.

3 Likes
9
  • What is your evaluation of the proposal?
    This is great + 1
  • Is the problem being addressed significant enough to warrant a change to Swift?
    Yes, documentation is important, runnable documentation is even better
  • Does this proposal fit well with the feel and direction of Swift?
    Yes
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
    I have never implemented runnable documentation myself, but this proposal makes it look easy so if this is approved I will start. I do not know if it's this easy in other languages.
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
    I read end-to-end.
10

Uff, took me a while to wrap my head around this but as someone who wrote a ton of library documentation with such snippet engines I felt I should give this a proper read -- hope the feedback can be fuel some productive discussion!

Thanks for the proposal, this is a great topic that could–together with docc–finally lead to great improvements in consistency and quality of documentation of swift libraries.

Having that said, I have some concerns about the current shape of what is being proposed.

The proposal states that:

  • Complete sample projects
  • Bits of code displayed inline within documentation

Both of these are critical tools, and snippets aren’t intended to replace either of them. Instead, snippets offer an additional method for teaching with code, that fits somewhere between both ideas.

(Emphasis mine); My primary problem with the way this proposal positions itself is by not aiming to fix the "bits of code displayed inline" which are often written in-line in documentation in Swift documentation today, which leads to them being outdated (snippets don't even compile), or slightly wrong (snippets compile but do the wrong thing).

Shouldn't snippets aim to replace such "random snippets which don't even compile"? I'm sure you know that because you explain the issues with them in the Motivation section after all.

I'd like to improve on the

Each snippet is a single file, making it easy to think about as an author and as a reader, but it is also a fully valid program.

part of the proposal, based on my prior experience in writing a lot of reference documentation, first using rst and later on building our own documentation engine paradox that was used a lot by various lightbend/scala ecosystem libraries...

In one sentence my point here is that a snippet not just a file, but a file contains many snippets which are purposefully deliniated by the documentation author. The focus isn't on "hiding" but on showing concrete important pieces.

In many projects that are a bit more advanced than a collections library or something similar, you often end up interleaving "snippet" with "explanation", and then the "next step snippet" and so on.

In the current model, I would have to replicate all the "setup" to show the "next step" into many files -- as many, as I have steps in my explanation.

Rather, I would suggest adopting a "snippet is an identifier of a section of code" approach, like this:

//# imports // whatever the syntax might be
import Library
//# // "end current snippet", or tagged to "end a concrete snippet" with // snippet: imports

//# setup
let x = [...]
//#

//# step-2
x.example()
//#

This would be contained in a file like Snippets/Example.swift. Then we'd include them like:

@Snippet("Intro", sections: "imports", "setup", "step-1", "step-2")

It is also worth discussing the @Snippet as it is proposed today, as it uses "path" which is a bit misleading; because it is not a real "path" (like file path). And since we're going to assume "Snippets/" anyway might as well simplify it a bit, and just ask for the snippet name, and if we have to disambiguate, the module name:

@Snippet(/*module name*/"MyLibrary", /*file name*/"Intro", chunks: "...") // "chunks" or "snippets"

This "opt-in sections to be shown" approach also solves the question about imports really, since it is up to the snippet author to decide wether to include them and WHICH imports to include.

For example, I may be using my testkit library but I don't need the users to know about it since I'm using it in verification of the snippet:

//# imports
import Library
//#
import LibraryTestKit // NOT shown in snippets

//# sample
something()
//#

// @Snippet("...", sections: "imports", "sample")

The proposed use of // MARK: HIDE seems weird to me. Again, i'd propose the opposite -- sections which are to be referenced to from prose documentation should be annotated, not the other way around. This way we can:

@Snippet("Intro", sections: "imports", "setup", "step-1", "step-2")

Where in our Intro.swift we can have tests between all those steps:

//#imports
import CoolLibrary
import CoolLibraryExtras
//#
import XCTest

//# step-1
let thing = initializeThing()
//#
assert(thing.something == 2)

//# step-2
let result = await thing.performAction()
//#
XCAssertEqual(result, "something")

This way we're able to show the steps of an unfolding code snippet, and in the end show the entire thing if we wanted to.

Don't forget to import...

@Snippet("Hello", sections: "imports")

Then you can...

@Snippet("Hello", sections: "step-1")

And finally, we're ready to ...

@Snippet("Hello", sections: "step-2")

### Complete listing

@Snippet("Hello")

Nesting restriction

A single level of subdirectories is allowed to balance filesystem organization and further subdirectories for snippet-related resources, which are expected to be found informally using relative paths.

The "single level of subdirectory" seems like an arbitrary restriction, I'm not sure it's worth imposing it. You'd have to properly diagnose if a deeper structure is detected, and there's no real benefit from banning those. Why forbid Cluster/SpecificConfiguration/[...].swift examples :-)

Testing

I have concerns about how testing is not really thoroughly discussed for snippets.

Effectively this proposal claims either "just compile them" or "just run them" and that would be enough to verify them... but the practical side isn't fleshed out.

Compiling snippets (swift build --build-snippets as being proposed) we know isn't enough, as we need to run them as well -- but to do so, we have to list them and run them all "one by one", and the proposal does not offer any "test my snippets" solution.

Having to list all samples in some bash script to make sure they're tested on CI is not really acceptable, there must be an automatic way to make sure when I add new snippets that they're going to be tested already.

This is why usually such snippet systems I worked-with/built in the past, rely on tests to BE the source of snippets. So the discovery and "make sure they're all run" is solved by test runners. I don't get the argument about XCTest being platform specific as a reason for not being able to use these -- test discovery works on both Linux and Darwin and same do all assertions.

While XCTest may be lagging behind a bit on features sometimes (that's a different issue), I don't see why disallowing snippets from tests? E.g. we totally can / could:

//#imports
import DistributedActors // cluster lib
//#
import DistributedActorsTestKit
import XCTest

// test taken from http://github.com/apple/swift-distributed-actors
final class ClusterAssociationTests: ClusteredActorSystemsXCTestCase {
    func test_snippet_join() throws {
        let other = self.setUpNode()
        
        //#joining-a-cluster
        let cluster = ClusterSystem()
        cluster.join(other.cluster.node)
        //#
        
        // Assert using lots of unit-test utils that would have to be made work in XCTest,
        // as well as "in app for snippet" if snippets cannot be tests...
        try assertAssociated(other, withExactly: cluster.uniqueNode)
    }
}

While this is attempted to be addressed in "Tests acting as snippets" it doesn't really discuss it in depth enough IMHO.

The proposal, as is, offers a sub-par solution -- there's no "just test my snippets". And the only argument made against tests as snippet sources is that that simple top level script files are simpler, but that isn't really a very strong argument...

"MARK: HIDE"

The proposed snippet design where we can "hide" but not "select what to show" also makes it harder to show things to not do, which are often as useful to show as proper usage.

For example, with selecting snippets, I could:

func dontExecute() {
  //#bad
  thing.mode(.one)
  try thing.two()
  //#
}

func good() {
  //#good
  thing.mode(.two)
  try thing.two()
  //#
}

but rather we'd have to make a complete new file for the "bad" case, and then dance around with // MARK: HIDE

// MARK: HIDE
func dontDoThis() {
  // MARK: SHOW
  thing.mode(.one)
  try thing.two() // don't do this
  
  // MARK: HIDE
}
// MARK: SHOW

I also find it rather weird to use the MARK syntax for this... as it results in outline elements in the "minimap" that are nonsensical (just a bunch of HIDE)...

Swift Package Manager

This section is a bit light on content; A new target type "snippet" is mentioned but not explained more, I think this deserves more details as introducing a new target type is rather substantial.

How are dependencies handled for snippets? Say I need to include some metrics implementation for my snippets to make sense; where do I declare that dependency? Is that overriding a SnippetGroup in the Package.swift, or somewhere else?

Finally, the proposal mentions "Multiple snippets per file." in future directions, but to me this is the heart of such engines, as explained above in depth, and I find it very lacking and limiting setting us off from the "wrong foot" onto this journey I think, personally -- documentation authors want to show exact pieces of a file to users, and not just dump an entire file at them (again, based on my experience documenting large/advanced libraries).

// edit: typo

26 Likes
11

Thank you very much for taking the time to read the whole proposal and offer your thoughts, @ktoso! I really appreciate it.

Snippets could stand in well for code blocks but they were designed to be more of a "flash card" that can be searched, browsed, found, copied, and pasted. However, they do come with certain encouraged methods and what I hope will be some new, important consumption scenarios, some of which are outside the traditional documentation experience. I intentionally left these examples out of the proposal because I wanted to focus on authoring.

What you have described here is a tutorial though, not a snippet. DocC already has facilities for creating a sequential narrative, as if building up a piece of code. I think it's totally fine to judiciously include regular developer (//) comments in a snippet, though!

It helps to think about where snippets could appear to readers–not just inserted into prose but also showing up in their entirety in search results on the web or in an application. That's probably not a place where even a moderately sized tutorial should appear.

It behaves exactly as a path and is directly inspired by existing DocC and URL terminology. It is effectively the same as a DocC symbol link or block @Image in Markdown. See:

This argument presumes that snippets should exist in an environment where other imports are necessary for non-snippet code. A snippet already knows which imports to include–all of them! Since a snippet is a file, one would only write the imports necessary for the file to compile. And there is no reason to hide the imports of a snippet as they are valuable information, especially when snippets show up outside a "single framework documentation page" scenario.

Perhaps DocC may want to hide imports if there are many interspersed throughout an article in the future. If someone is just documenting Foundation, then it might seem a bit funny to see import Foundation five to ten times in an article. That said, what might be obvious to us may not be to another developer–if we put ourselves in the shoes of a newer programmer who is looking for a kind of functionality, they may get a snippet in search results that uses Combine. In that case, showing the import is helpful to them. DocC can either look for import statements at the beginning of a snippet, or we can push imports into a separate field in the data model so that different kinds of presenters can choose whether to display them.

Consider another interesting use case: a package existing solely to illustrate compositions of APIs between SwiftUI and Combine–showing the imports becomes especially important.

In the scenario you presented with LibraryTestKit, if we are talking about testing, there isn't anything that indicates that the import couldn't be appended at the bottom of a snippet, along with assertions, under a single hide flag. If we are talking about helper libraries that have demo functionality, it's not yet clear that we need some sort of "pre/setup hide" functionality.

Again, what you have described here is a tutorial–and snippets are not trying to compete with that use case. Guiding a reader through something step by step can be valuable but that's not [edit: missed the “not” here in this reply] the goal of snippets.

Perhaps what might be needed here is a kind of start/end scope for test-related things. Straw example, but something like:

let x = foo()

#test {
  // do some hidden test stuff
}

print(x)

That would help alleviate the fast, tick-tock rhythm you are suggesting. The mark comments are optimized for one or two switches in a file, or larger chunks, but it really depends on the shape of the code being tested. I believe that the majority of code in a snippet will actually be shown.

This was meant to provide a simple way for folks to put further accompanying subdirectories of resources without naming restrictions next to their snippets. That is, if someone needs a resource file with a .swift extension, the search can stop at the second level and a new snippet won't accidentally appear.

It also encourages authors to not create too deep of a hierarchy, making it difficult for readers to find snippets. A perfectly legitimate way to find and consume snippets, some may find their way via a GitHub file browser. Deeply curated hierarchies are very easy to create and always seem like a beneficial thing–however, I find them to be a strong anti-pattern in documentation, making documents (or code in this case) difficult to find as a reader's sense of organization or terminology may not match up with the author's. In a sense, overly deep hierarchies, whether it's files or documents, are more often for the author than the reader, practically speaking.

This problem occurs both in conceptual documentation like articles, and in sample application projects and was something I was trying to avoid based on my evaluations while working on documentation systems at Apple.

The first thing that comes to mind here is, "How does one test an executable?" because that's exactly what snippets are. My answer to that would not necessarily be XCTest. More on that below.

Not all snippets can be run in an automated fashion, such as those that involve user or network input. This was left open to allow developers to choose what level of support they want to give snippets.

A large portion of the verification comes from building snippets. Honestly, if we had just that, it would be a huge leap forward. Another large chunk comes from the existing tests for library code being called–library authors should and are already writing these. What remains is testing behavior for the compositions not anticipated by library authors. I believe those can be covered by preconditions referring to the natural declarations that exist in a snippet for now.

This was addressed in the proposal but perhaps should've included an example. The example code you posted is more or less what I was thinking about trying to avoid. It's not easy for me to read on its own. Think of a newer developer who is not familiar with the project–they are not going to bother looking for that code in the project and they are much less likely to contribute changes to a snippet back to a project. This also points back to snippets being in their own file, which frees them from a particular point of view (the library developer's, in this case).

There are platform-specific differences between Darwin and non-Darwin platforms when using XCTest, but really the reason I didn't propose putting snippets in tests is that they require a specific execution scenario that is difficult to reproduce. For example, XCTest assertions require a certain runtime context in order for test failures to be collected and logged properly. That is not really an appropriate restriction to put on what I see as a general purpose executable. I am all for using the de facto standard test library! I would much rather wait until there is a way to include XCTest assertions in plain executables, log test failures as they occur, and automatically return nonzero before exiting, however. That still doesn't indicate to me that snippets should exist primarily in test cases.

As mentioned in the future directions section, it might be possible to extract snippets from any kind of target tests and even libraries. For this first version, I wanted to focus on snippets as a new kind of documentation unit in a new context–a small executable was the perfect stage to make that happen. In order to keep that simple for this first version, informal testing with precondition seemed like the best place to start, along with compilation and library tests that should already exist today.

This is an interesting idea, I have seen these sorts of good-bad scenarios–WebKit's Code Style Guidelines come to mind. I don't think snippets which show poor ways to use a library will be very frequent. A snippet is supposed to be exemplary–something a reader can copy and paste into their own project with few to no changes. So, for this example which you posted, I would simply write:

//! Use thing mode ``.two` to...

thing.mode(.two)
try thing.two()

In order to call out an anti-pattern with some stateful code as you mentioned, I would use a warning aside that exists in DocC:

> Warning: you must match ``mode`` changes with their accompanying method. For example, match ``Mode.one`` with ``one()``.

I don't think this is a contradiction for hide flags, since the snippet is saying "don't do this" –snippets should say what developers should do for the reasons stated above.

We can be flexible in this both ways–there isn't anything sacred about using "MARK", so we can change it, and Xcode can adjust for these as well.

Apologies, I think a sentence or two got accidentally deleted here. To clarify, snippets automatically depend on all library targets defined in the Package manifest in order to make it easy to write many snippets without each having to have their own entry in the manifest. The fact that snippets are a new target type isn't important except for that behavior–in all other ways, they are the same as an executable target.

While I agree that authors may want to point out the interesting pieces of a larger file, I am offering a different perspective–that we can create example code where the majority of it is interesting, exemplifying a common task. Snippets do naturally suggest a reasonable length and level of complexity. If a snippet needs to be 100 lines or more, that suggests that the what is being explained is a complex, multi-step scenario, and there are better tools for that. Perhaps it might even inform a library designer that new API is needed (with proper feedback on a snippet from readers)...

4 Likes
12

This is an idea that was kicked around quite a bit, and referencing Sources/ is apt. The "Future Directions" section gets into this line of thinking, specifically multi-file snippets, and extracting snippets while building. The idea is that, in the future, snippets of code could be discovered and processed anywhere within your package, even in Sources/. This would add considerably flexibility, and align with the general behavior of SwiftPM.

This is in "Future Directions" because these ideas involve some fairly long threads to pull. In the meantime, the simpler implementation means documentation and package authors can get started right away, knowing that snippets they write should work moving forward.

1 Like
13

My opinion on the overall design is largely unchanged from the pitch thread. I think this is an important problem, it's something that I personally encounter very often and I struggle with it. There are ways to work around bits of it, but there's just no overall good answer for keeping code snippets in documentation up-to-date right now.

But when I try to imagine implementing the proposed feature, there are just too many practical problems. I'm not fond of the need to have a parallel hierarchy of separate files, and some of the issues were also touched upon by Konrad (e.g. testing that snippets don't just compile, but actually produce the expected result). I mentioned them in the pitch thread.

Anyway, the reason I'm commenting is this:

One thing that struck me is how little we specify when it comes to things like paths. It's not unique to this proposal, but this is another example of it. This is what is says about the path:

Swift-DocC (or other documentation tools) can then import snippets within prose Markdown files. For DocC, a snippet's description and code will appear anywhere you use a new block directive called @Snippet , with a single required path argument:

@Snippet(path: "my-package/Snippets/Snippet1")

The path argument consists of the following three components:

  • my-package : The package name, as taken from the Package.swift manifest.
  • Snippets : An informal namespace to differentiate snippets from symbols and articles.
  • Snippet1 : The snippet name taken from the snippet file basename without extension.

Which sounds okay... but wait - did you say file names?

  • Which characters are allowed? For example, backslashes are allowed in file names on Apple OSes but not on Windows - should that be enforced on all systems using some basic filename restrictions?

  • Do they respect Unicode canonical equivalence? If I write caf\u{00E9} in my source code, will it match a file named cafe\u{0301}? Note that the native behaviour of Apple OSes and ChromeOS tends to differ from Windows and Linux here.

  • Are they case-sensitive?

Again, lots of proposals fail to specify details like this. Do we just leave all of that unspecified, or is it worth pinning down (perhaps in some kind of broader "infrastructure" proposal defining concepts such as paths and how we expect filenames appearing in source code to be interpreted)?

2 Likes
14

DocC already derives identifiers from file names and the same rules would apply in terms of what characters are allowed, which is largely influenced by what is allowed in URLs. Perhaps the proposal could’ve been more clear on that.

Curious about this. What about this is unappealing? Is it more about the file location or the location of the snippet contents?

15

I think this falls into the trap of the "enemy of the good", because with that proposal, I'd also want to see it generalized to allow having Tests be able to be mixed in with Source files, which increases the scope and adds unnecessary delay.

If we can today, I want less magic than proposed. If we can't, for some technical or resource constraint reason, then I suppose I can live with it - but if we're just putting it off to avoid more churn in package manifest specification... No.

16

Thanks for the detailed reply; I'll get back to the other points...

I think you made some good points, like some of what I'm asking for is actually tutorials; that's fair points.

This is my issue though I think? I can't make a snippet that depends on some other thing that in combination with my library is "the snippet I want to show to people". I build highly "pluggable" systems and would want to be able to show people "this is how you plug stuff together".

This isn't a tutorial case; this is "here's some snippets how you can configure this library to use metrics and tracing and..." it really seemed to me like the best use of snippets out of all I had mentioned; but I'd need to control their dependencies to do this.

But if we never have the explicit target at all, where would I add the 3rd party dependency I want to show in the snippet (but my library does NOT depend on)?

Thanks for discussing the various perspectives :slight_smile:

7 Likes
17

Ah, so you are talking about snippets using external dependencies. Yes, that’s a great point. One thing that I’m hoping to see in the future are snippet-only packages that illustrate a new way of combining two packages. I was thinking that would be further out but we can track it in the future directions section of this proposal if that sounds good.

The way I was originally going to implement that was to allow snippets access to library products of any package dependencies in addition to the libraries defined in the package.

2 Likes
18

Yeah I guess I'd do it as a separate package; We had such in the past. This one specifically illustrates the case I had in mind: swift-distributed-actors/Samples/Sources/SampleMetrics at release/0.x ¡ apple/swift-distributed-actors ¡ GitHub

So actually... it would be possible if I made such samples package; and have that depend on some metrics thing same as my above sample app... and that'd depend on the things, and the snippet would have it as well I guess...?

It is not super clear to me if that already just works, or extra work is required :thinking:

19

If I understand it correctly, as proposed, snippets can only depend on the targets provided by the current package. It is not possible for snippets to depend on other packages, even if they are listed as dependencies in Package.swift.

@bitjammer: Is my understanding correct?

20

Review manager note: With the discussion still going, we will extend the review period until May 24th.

2 Likes