Document Extensions to External Types Using DocC

Hi everyone! I'm excited to announce that I'll be working on [SR-15410] Can't document extensions with DocC - Swift!

Since most of you probably haven't seen my name tag before: My name's Max, I'm studying informatics in Munich, Germany and I'm very excited to be contributing to the Swift ecosystem!

The question about DocC and how to document extensions to external types originated in the forum thread DocC and extensions? and it quickly became apparent that there are some important UX questions to be solved. Therefore I thought I'd kick of my contribution efforts with a pitch focused on the UX of this new feature.

First off, I want to thank everyone who participated in the forums post linked above, you'll probably find some of your ideas in my concept! Special thanks to @franklin for guiding me trough this process and his initial feedback on this pitch!

Introduction

DocC does not include extensions to a type defined in an external module in the documentation catalogue, even though the extension and its contents are declared in the documented module.

Consider the following potential addition to the SlothCreator package:

/// A type that generates sloths.
public protocol SlothGenerator {
    /// Generates a sloth in the specified habitat.
    func generateSloth(in habitat: Habitat) throws -> Sloth
}

public extension Collection where Element == Habitat {
    /// Generates one ``Sloth`` per ``Habitat`` in the collection.
    ///
    /// - Note: Unfortunately, neither this comment nor the function itself is included in
    /// the documentation catalogue yet.
    func mapToSloth(using generator: SlothGenerator) throws -> [Sloth] {
        try self.map(generator.generateSloth(in:))
    }
}

/// A type that generates names for sloths.
public protocol NameGenerator {
    /// Generates a name for a sloth.
    ///
    /// - parameter seed: A value that influences randomness.
    func generateName(seed: Int) -> String
}

/// An array of strings conforms to ``NameGenerator``. Each time
/// ``generateName(seed:)`` is called, it returns the element identified
/// by the given seed.
extension Array: NameGenerator where Element == String {
    public func generateName(seed: Int) -> String {
        self[seed % self.count]
    }
}

Neither of the two extensions is added to the documentation catalogue. That is, both Collection/mapToSloth(using:) and Array/generateName(seed:) are not listed. Furthermore, they cannot be referenced using their identifiers and therefore it is also not possible to include them in the documentation catalogue using manual curation. Finally, Array (with Element == String) is also not listed among the Conforming Types of NameGenerator.

Motivation

Swift encourages us to use extensions on external types and therefore we should also be able to document such extensions accordingly. As I already mentioned, this capability has also been requested and discussed on the Swift forums before.

While there are possibly infinite use cases, I think that especially the growing ecosystem of SwiftUI packages could benefit hugely from this addition. A large part of their public API surface may be made up of view modifiers, which are usually exposed as function-extensions on SwiftUI's View type.

Proposed Solution

We extend the default top level sections in the sidebar (e.g. Structures, Classes, or Functions) by one element named Extensions. This section lists one element per external extended type (not per extension). Extensions to locally defined types are not listed there. An alternative naming suggestion would be "Extended Types". This is a bit longer and maybe less elegant, but I think it gives a more precise description of the content. Furthermore it might prevent users from looking for extensions to internally defined types in that section.

The documentation page for these external extended types only lists its locally defined members and conformances. The grouping/positioning of these members follows the usual rules. One can still manually curate the type's page to achieve whatever grouping is desired, of course.

This approach roughly follows the precedent set by the documentation framework Jazzy. You can check out a Jazzy documentation here.

Note that this also means that documentation added to extension blocks (not to the elements inside the extension block) is ignored. This would e.g. apply to the comment on Array's conformance to NameGenerator in the example above.

Future efforts therefore might include incorporating such comments if they belong to a protocol conformance or there is only one extension block for the respective type, as well as improving the automatic grouping using various heuristics. Those could include whether or not two members are defined in the same extension block and if such block states a protocol conformance.

Member Identifier

Swift allows us to shadow imported types by declaring a type with the same name locally. Therefore just using TYPE_NAME/MEMBER_NAME might be ambiguous:

/// This is a custom structure called ``Int64``, which is different
/// from the `Swift/Int64` from the standard library.
public struct Int64 { }

public extension Swift.Int64 {
    /// An extension to the `Swift/Int64` type from the
    /// standard library.
    public static var one: Self {
        1
    }
}

I think the most sensible solution is to prefix the external type with its module name (MODULE_NAME/TYPE_NAME/MEMBER_NAME), i.e. in the above example Swift/Int64/one would be the valid identifier, whereas just Int64/one would be invalid, as the local Int64 type does not have a property called one.

One could allow usage of the simpler identifier without the module name if the external type is not shadowed locally. However, this option could be added at a later point in time without source compatibility problems, therefore I don't consider it a priority right now.

Type Identifier

The more interesting question is with the identifier for the external type itself. Currently, the solution trivially seems to be MODULE_NAME/TYPE_NAME. Unfortunately this could cause conflicts with [SR-15431] Support DocC references to symbols defined in another module - Swift down the road. The description of this feature request already suggests usage of the exact same identifier for the type definition in the external module. That is, we have to decide whether e.g. Swift/Int64 should link to the page in the local documentation catalogue listing the local extensions or to the external documentation catalogue with the original type declaration.

My preferred solution would be a local first strategy, i.e. if the link only refers to the type itself, it links to the local documentation catalogue's page listing the locally defined extensions. This page then always contains a link to the respective page in the external documentation catalogue (once [SR-15431] Support DocC references to symbols defined in another module - Swift is implemented), e.g. in the "Framework" section of the right column. Naturally, if there are no local extensions to the external type in question, the link directly refers to the type's page in the external documentation catalogue.

Here's an example of how this would look like with the Xcode frontend (of course the proposed changes are not exclusive to Xcode):

Alternatives Considered

Specific Syntax for Differentiating Type Identifiers

An alternative to the semi-ambiguous MODULE_NAME/TYPE_NAME identifier would be to augment either of the two identifiers with a special flag, e.g.:

  • Swift/Int64-swift.extension links to the local extension page (if valid)
  • Swift/Int64 always links to the types declaration in the external documentation catalogue

I dislike this approach as it results in a less intuitive syntax. Of course there is auto-complete in Xcode, but from time to time people also tend to use this syntax outside of Xcode.

"Extending Module Name" Based Identifiers

A third option for type and member identifiers would be to use the extending module name as a prefix. That is, the page listing all local extensions would be identified by EXTENDING_MODULE_NAME/TYPE_NAME (e.g. SlothCreator/Int64), whereas the original declaration of the extended type in the external module would be referenced by DECLARING_MODULE_NAME/TYPE_NAME (Swift/Int64). Locally defined members would then be identified by EXTENDING_MODULE_NAME/TYPE_NAME/MEMBER_NAME, whereas other members would be identified by DECLARING_MODULE_NAME/TYPE_NAME/MEMBER_NAME.

However, then we have to look at the shadowing problem again:

/// This is a custom structure called ``Int64``, which is different
/// from the `Swift/Int64` from the standard library.
public struct Int64 {
    // some members ...
}

public extension Swift.Int64 {
    // some members ...
}

/// This is an extension to a custom structure called `Int64`, which is different from `Swift/Int64` AND the local ``Int64`` types.
public extension SomeOtherPackage.Int64 {
    // some members ...
}

Assuming this example was set in the SlothCreator package, the local documentation pages of all three types would be identified by SlothCreator/Int64. The solution to resolve this conflict would be adding a unique code to the type name as it is already done in ambiguous situations for member identifiers, i.e. something like SlothCreator/Int64-59i2x. Identifiers for members would also require this postfix on the TYPE_NAME in certain situations.

While I would be open to this approach, I feel like it could be very confusing to some as it semantically differs from the way we reference the respective types and members in Swift itself.

One Page per Extension Block

We usually group code that is semantically similar into one extension block. Therefore, instead of collecting all extensions to one external type on a single page, one could also create one page per extension block.

In that case (type) identifiers would have to include some code unique to the respective extension block (e.g. Swift/Int64-61i0x) so that we can unambiguously identify the various pages belonging to the same external type. This would not be necessary for member identifiers.

What's beneficial about this approach is that comments above extension blocks always have an appropriate place in the documentation catalogue. Furthermore, there would be a clear rule for automatic grouping with predictable results that can be influenced by the grouping of extensions in the source code.

However, I feel like this would result in having very many extension pages with very little content each for most code bases. Furthermore, this concept would contradict DocC's concept of the "Topics" section on a type's page which is specifically designed to allow for discussion of different natures of the same type.

Discussion Points

Of course, please share any feedback or ideas you have! However, I'm especially looking for input on the following topics:

  • Opinions on identifiers. Do you agree with my choice or would you prefer one of the other options? Do you see problems with any of the options I oversaw? Do you see any better alternatives?
  • What do you think about the naming of the new sidebar section ("Extensions" vs "Extended Types")? Do you think many people would look for extensions to locally declared types in an "Extensions" section? Do you have any other suggestions on how one could reduce the risk of confusion there?
  • What is your opinion on comments above extension blocks? Is that something we should embrace in DocC, or do you think this is unnecessary or even bad style? I personally write such comments from time to time, especially if the extension block adds a protocol conformance to the type. This decision might have some influence on the implementation approach, so it might not be too easy to change later on.

Based on how this discussion goes I'll either iterate on this pitch to incorporate bigger changes or start exploring implementation strategies in more depth.

Either way, I'm very much looking forward to your thoughts on this!

~ Max

19 Likes

Hi Max, this looks great! I'm excited to start using DocC for extensions.

Regarding urls/identifiers specifically, could you share a more compact version of what you're proposing? In other words, given the following very edge case scenario (and any other examples you think are useful to show), can you provide all of the symbol identifiers and page urls for how I might link to these from within the MyModule docs and how I might link to the MyModule pages from the Swift docs (in a future where we have cross-catalog linking)?

// This code is all in the module `MyModule`

/// My type and member that shadow Swift's version.
public struct Int64 {
   public  var bitWidth: Int
}

public extension Swift.Int64 {
    /// This decl shadows the Swift.Int64.bitWidth decl
   public var bitWidth: Int

    /// This decl overloads Swift.Int64.advance(by:) that takes an Int.
    public func advanced(by: String) -> Int64
}

Also, to check my understanding, are you proposing that the identifier for the extension page for a type shadows the identifier for the extended type? I see two problems with that:

  1. It would no longer be possible to link to the extended type/shadowed member (in authored content or automatically in places like the declaration). I think this is problematic for the places shadowing occurs because that's probably where you want to link to the shadowed symbol.
  2. You couldn't host both packages on the same server because their URLs would overwrite each other.

All that said, I'm totally in favor of designing for the 99% case, if we also make it possible for the user to handle the 1% case even if for those cases it's less ideal.

3 Likes

@jack thanks so much for your input! You feedback is very valuable to me as identifiers/URLs really were the part of this pitch I felt the least confident about.

Let me first give you the the identifiers for your example as I proposed them in the pitch:

(I don't know how/why Swift should link to MyModule as this would be a cyclic dependency, but I added those identifiers anyway.)

The MyModule.Int64 type:
Identifier in MyModule: Int64
Identifier in Swift: MyModule/Int64
URL: hosturl/mymodule/int64

The MyModule.Int64.bithWidth member:
Identifier in MyModule: Int64/bitWidth
Identifier in Swift: MyModule/Int64/bitWidth
URL: hosturl/mymodule/int64/bitwidth

The Swift.Int64 type's extension page in MyModule:
Identifier in MyModule: Swift/Int64
Identifier in Swift: MyModule/Swift/Int64
URL: hosturl/mymodule/swift/int64

The Swift.Int64 type (in Swift):
Identifier in MyModule: identifier does not exist - linked in extension page
Identifier in Swift: Int64
URL: hosturl/swift/int64

The Swift.Int64.bitWidth member (defined in MyModule):
Identifier in MyModule: Swift/Int64/bitWidth
Identifier in Swift: MyModule/Swift/Int64/bitWidth
URL: hosturl/mymodule/swift/int64/bitwidth

The Swift.Int64.advanced(by:) member (defined in MyModule):
Identifier in MyModule: Swift/Int64/advanced(by:)-CODE
Identifier in Swift: MyModule/Swift/Int64/advanced(by:)-CODE
URL: hosturl/mymodule/swift/int64/advanced(by:)

The Swift.Int64.bitWidth member (defined in Swift):
Identifier in MyModule: identifier does not exist
Identifier in Swift: Int64/bitWidth
URL: hosturl/swift/int64/bitwidth

The Swift.Int64.advanced(by:) member (defined in Swift):
Identifier in MyModule: Swift/Int64/advanced(by:)-CODE
Identifier in Swift: Int64/advanced(by:)
URL: hosturl/swift/int64/advanced(by:)

So, yes, you are right, extended types, as well as their shadowed members cannot be linked. About your concerns:

  1. They type is always referenced on the type's extension page instead. As for the shadowed members, I didn't really consider this a problem as they are inaccessible from Swift code and therefore basically meaningless for our local module. I see the use-case you mentioned, though I am not sure if it justifies opting for one of the IMHO less intuitive alternatives I presented in the pitch.
  2. As you can see in the example above, the URLs are not subject to ambiguity as the extension pages are hosted within the documentation catalogue the respective extensions are defined in. Basically the URLs always follow the pattern DOCUMENTATION_MODULE_NAME/TYPE_MODULE_NAME/TYPE_NAME/MEMBER_NAME, where the TYPE_MODULE_NAME is skipped if it equals the DOCUMENTATION_MODULE_NAME.

Looking at the URLs in more detail also gave me an idea for an intuitive syntax that allows referencing everything without ambiguity.

Basically, the ambiguity we're looking at with the type/type-extension-page and the shadowed members is about ambiguous relative paths.

Consider the example. The extension Swift.Int64.bitWidth (defined in MyModule) is identified by Swift/Int64/bitWidth. This is essentially a relative path starting at the documentation catalogue's root. However, the same relative path exists starting at the hosturl (referencing the shadowed bitWidth defined in Swift), i.e. the root of your hosting environment for DocC documentation catalogues.

One could consider this second relative path an absolute path as this "root of the hosting environment" is basically the root of the documentation catalogue world on your server.

Now, we could use this idea of absolute and relative paths in DocC's identifier syntax. In order to (unambiguously) reference a symbol outside the local documentation catalogue, one has to use an absolute path starting with a /, followed by the DOCUMENTATION_MODULE_NAME and the rest of the path inside that external catalogue.

Relative paths also can link to external catalogues if there is no conflict with locally defined symbols.

Based on that strategy, your example above would yield the following identifiers:

The MyModule.Int64 type:
Relative Identifier in MyModule: Int64
Relative Identifier in Swift: MyModule/Int64
Absolute Identifier: /MyModule/Int64
URL: hosturl/mymodule/int64

The MyModule.Int64.bithWidth member:
Relative Identifier in MyModule: Int64/bitWidth
Relative Identifier in Swift: MyModule/Int64/bitWidth
Absolute Identifier: /MyModule/Int64/bitWidth
URL: hosturl/mymodule/int64/bitwidth

The Swift.Int64 type's extension page in MyModule:
Relative Identifier in MyModule: Swift/Int64
Relative Identifier in Swift: MyModule/Swift/Int64
Absolute Identifier: /MyModule/Swift/Int64
URL: hosturl/mymodule/swift/int64

The Swift.Int64 type (in Swift):
Relative Identifier in MyModule: identifier does not exist - linked in extension page
Relative Identifier in Swift: Int64
Absolute Identifier: /Swift/Int64
URL: hosturl/swift/int64

The Swift.Int64.bitWidth member (defined in MyModule):
Relative Identifier in MyModule: Swift/Int64/bitWidth
Relative Identifier in Swift: MyModule/Swift/Int64/bitWidth
Absolute Identifier: /MyModule/Swift/Int64/bitWidth
URL: hosturl/mymodule/swift/int64/bitwidth

The Swift.Int64.advanced(by:) member (defined in MyModule):
Relative Identifier in MyModule: Swift/Int64/advanced(by:)-CODE
Relative Identifier in Swift: MyModule/Swift/Int64/advanced(by:)-CODE
Absolute Identifier: /MyModule/Swift/Int64/advanced(by:)
URL: hosturl/mymodule/swift/int64/advanced(by:)

The Swift.Int64.bitWidth member (defined in Swift):
Relative Identifier in MyModule: identifier does not exist
Relative Identifier in Swift: Int64/bitWidth
Absolute Identifier: /Swift/Int64/bitWidth
URL: hosturl/swift/int64/bitwidth

The Swift.Int64.advanced(by:) member (defined in Swift):
Relative Identifier in MyModule: Swift/Int64/advanced(by:)-CODE
Relative Identifier in Swift: Int64/advanced(by:)
Absolute Identifier: /Swift/Int64/advanced(by:)
URL: hosturl/swift/int64/advanced(by:)

What do you think about this approach?

Hi Max, thank you for this superb writeup! I appreciate how much depth you've gone into when designing your proposed solution.

I like the idea of grouping all APIs in extensions together under a single "Extension" page. This aligns with what DocC does for extensions to types defined in the same module, and anyway, as a reader of the documentation, you aren't really concerned about how these APIs are created in source. Also, the constraints on the extension are displayed alongside each method in the documentation. (This does make it more difficult for DocC to present extensions' doc comments in a meaningful way if there is more than one extension for the same type, though.)

I agree that the fact the section would not include extensions to locally declared types could be confusing. We could explicitly name the section with the module being extended and create a section per extended module. For example:

Swift Extensions
  - Array
  - Int64

Foundation Extensions
  - URL
3 Likes

Hi Max, this is a great writeup. Documenting extensions to external types is an extremely important feature, because as you correctly identified, SwiftUI view modifiers are normally implemented by extending View.

  • Opinions on identifiers. Do you agree with my choice or would you prefer one of the other options? Do you see problems with any of the options I oversaw? Do you see any better alternatives?

I agree with @jack's point here. I think that we shouldn't have any ambiguity in shadowing types vs the original. I would personally like an approach which aligns with what @franklin suggests where we prefix non-local identifiers with their defining module name. Something along the lines of:

// This code is all in the module `MyModule`

/// My type and member that shadow Swift's version.
public struct Int64 { // MyModule/Int64
   public  var bitWidth: Int
}

public extension Swift.Int64 { // MyModule/Swift/Int64
    /// This decl shadows the Swift.Int64.bitWidth decl
   public var bitWidth: Int

    /// This decl overloads Swift.Int64.advance(by:) that takes an Int.
    public func advanced(by: String) -> Int64
}
  • What do you think about the naming of the new sidebar section ("Extensions" vs "Extended Types")? Do you think many people would look for extensions to locally declared types in an "Extensions" section? Do you have any other suggestions on how one could reduce the risk of confusion there?

I agree this is a weird situation. I quite like @franklin's solution there. It disambiguates what the extension is extending but we could potentially get into a situation where we have many such sections at the "top level". I don't really have a good solution to offer here.

  • What is your opinion on comments above extension blocks? Is that something we should embrace in DocC, or do you think this is unnecessary or even bad style? I personally write such comments from time to time, especially if the extension block adds a protocol conformance to the type. This decision might have some influence on the implementation approach, so it might not be too easy to change later on.

I personally am not a fan of these comments but I wouldn't go as far as to say it's bad style. The reason for this is that when you extend a type or protocol you can not alter the meaning of the type or the requirements associated with the protocol. The only things I can see that can only be documented on the extension block as opposed to the members are the reasons why you are providing the extension, which IMO doesn't belong in the documentation in that it doesn't tell the user actionable information. I could be wrong here and if you think there is a genuine need for these I would love to hear the motivation.

1 Like

@daniel-grumberg thank you very much for your feedback! I see that you share @jack 's concern about unambiguous identifiers. This seems to be of high priority to many of you!

You didn't provide the identifiers for the (shadowed) members in the example. Would I be correct in that Swift/Int64/bitWidth links to the Swift.Int64.bitWidth definition from the Swift module? I find that very counter-intuitive, as writing Swift.Int64.bitWidth in code in MyModule gives the local definition from MyModule.

What do you think about my earlier suggestion in my response to @jack 's comment?

(Sorry, this response of mine above was removed by the spam bot previously, but it's back online now...)

@franklin I really like your suggestion! Having one section per extended module makes it really clear that local extensions won't be included there. Furthermore, I think it would make it easier to explore and find pages if you don't have a specific term to search for.

@daniel-grumberg I don't think this would create too many sections for most people, as we're only talking about public extensions here. Most frameworks try to narrow down their set of public dependencies as far as possible anyway. And one can always opt for manual curation if the automatic behavior performs badly in a specific situation.

This proposal makes sense to me. The one downside here is that this is no longer aligned with Swift syntax, i.e., MyModule.Swift.Int64.bitWidth is not valid Swift. However, the nesting that /MyModule/Swift/Int64/bitWidth conveys does seem natural to me, because to use Swift.Int64.bitWidth you would first need to import import MyModule. (side note: Should Swift allow you to write MyModule.Swift.Int64.bitWidth?)

I think the main downside here is that if MyModule declares a type called Swift, we would need a disambiguation suffix (-swift.module-extension). However, I don't really see why a library would declare a symbol of the same name of the module it's importing, although it's technically possible. I understand why you'd declare a symbol of the same name of its defining module, but I don't see the use case for using an imported module's name. (My initial thought was to name this page Swift Extensions instead (URL component Swift-Extensions), however I'm proposing below to use the "Module Extension" label which would make the word "Extensions" redundant.)

Also, this change where we now have module extension pages that automatically curate extended types raises additional questions:

  • How does this impact the extension pages in your original proposal (e.g., for Array)? We probably don't want to use the same eyebrow value (aka the gray text above the title) for Swift and Array. I wonder if we should make things explicit with "Module Extension" and "Structure Extension".
  • How does this impact automatic curation on the top-level framework page? Would we go back to the model of an "Extensions" group that automatically curates all the module extension pages?
  • I expect it to be possible to override the documentation/curation of extension pages (for both module and symbol extensions) via documentation extensions, e.g., via a documentation extension that has the title # ``/MyModule/Swift`` or # ``/MyModule/Swift/Array`` . Is that what you envisioned as well?

Awesome stuff -- just here to say such categorization sounds very very nice :slight_smile:

1 Like

I like this in principle, but as I am quite new to the project myself I will delegate to @Franklin about the feasibility of this. I think this is similar to what I was getting at, it is essentially a way of adding the defining module for a symbol as prefix in the path. However I don't think relative paths should be allowed to link to external catalogues, imagine the scenario where you have written a bunch of relative paths that would resolve to a page in an external catalogue and that someone then shadows that thing in the local catalogue, all your links would now point to the shadowed thing if I understand your suggestion correctly.

No, if you use relative paths to symbols that are not defined in the local module they first link to the respective symbols in the external module. If you now decide to shadow these external symbols using local symbols, those relative paths now point to the shadowing symbols, i.e. the extensions in the local module and not the shadowed symbols in the external module.

You are correct though, this indeed may result in the scenario that adding an extension to an external type changes the destination of DocC links. However, I think in 99% of the cases this is the desired behavior. After all, usages of the shadowed symbol's identifier in Swift code now also use the local definition in the extension, i.e. the shadowing symbol.

I'm glad you like the approach! You are right, this absolute identifier syntax does not (yet) exist in Swift, however, I think that is better than using a syntax that exists in Swift with a different meaning.

Do you propose an overview page for each publicly extended module? I didn't have something like that in mind. What would that overview page show? I mean, sure, it could list all of the extended type's names, but not much more. The automatically generated page wouldn't have a comment to describe the extended module itself or any of the extended types. Furthermore, the extended modules are listed on the left sidebar on the same level as the section headers "Protocols", "Structures", etc., which also don't have overview pages.

I'd instead suggest to add this content to the default framework overview page. Without manual curation that page just lists all top-level symbol grouped exactly the same as in the sidebar. I'd extend this page with topic sections "Swift Extensions", "Foundation Extensions", etc. with each section listing its extended types just as in the sidebar.

This would also resolve this downside. Technically one could still e.g. create a type Swift with inner type Int64, but honestly if anyone ever decides to do that I'm fine with them having to use your proposed disambiguation suffix.

Yes, manual curation of symbol extension pages (as I said, I don't think we need module extension pages) would be possible. However, you wouldn't have to use the absolute identifier in the title. You could also just use # ``Swift/Array`` . The absolute identifier really only is necessary if you want to link to a a symbol in an external documentation catalogue that is shadowed in the local module.

Generating a path component for a page that doesn't exist isn't something that the DocC model accounts for at the moment, e.g., for /Swift/Int64/bitWidth, it would typically expect a page for Swift, Int64, and bitWidth. This is how DocC curation is modeled, where each layer of nesting is represented by a page that curates its children. That being said, I'm open to exploring evolving that model!

I think the approach you're proposing worked well when we weren't considering nesting the extended module within the defining module (/Swift/Int64 rather than /MyModule/Swift/Int64) because the Swift page is defined by the docs for the Swift standard library module itself. With /MyModule/Swift though, we're talking about something that /MyModule owns (i.e., the extensions to Swift).

Some ergonomics/organization limitations I think we'll see if we don't generate an extended module page:

  • if you want to curate the entirety of the extensions of a module somewhere else in your curation tree, you'd need to manually curate each extended type, rather than curating the extended module's page. And when you introduce a new extension, you'd need to keep that curation up-to-date manually.
  • you can't easily curate extended types into their own topic groups, e.g., you can't have an organization such as:
- Swift
  - Collections
    - Array
    - Sequence

because Swift is a topic group rather than a page.

Another alternative syntax could be to use both the extending module and the extended module in the first path components similar to the naming of the extension symbol graph files. If we wanted to follow the naming convention of the extension symbol graph files (joining the names with an @ sign) then this first path component would be MyModule@Swift.

In this hypothetical syntax I feel that the @ sign would be suitable for the authored link but we may want to use another character (for example -) in the URL making the first path component of the url mymodule-swift.

1 Like

I don't mind the layering mechanism that was proposed, and would be fine with alternatives such as what David's proposing - but I'd absolutely prefer that whatever naming/extension mechanism is used be consistent between the URL and any symbol that you'd author in markdown.

Along those lines, 100% onboard with wanting to not have any ambiguity of any shadowed type vs. original types. I've self-inflicted enough shadow damage using ill-conceived names in the past, I'd not like to be able to stumble into that here as well.

No super-strong opinion on where extensions should land if they're not curated, but I'd absolutely want to be able to curate the extensions. I've had cases where extensions are a minor implementation detail in a library, and other cases where they're a critical aspect of the API that's exposed for developers to use and deserve more prominence. In any case, just having them exposed would be a tremendous win, so I'm loving this proposal at the heart of what it's trying to do.

1 Like

Where would the extension pages live in that scenario?

I guess either documentation/mymodule-swift or documentation/mymodule/mymodule-swift?

I'd find the first case quite weird, as pages in documentation/mymodule would still list and refer to content in documentation/mymodule-swift which is a different sub-folder.

In the second case, the mymodule- prefix wouldn't be redundant itself, but the fact that it states mymodule there is. That is, one could also use some constant there (e.g. the swift.module-extensoin @franklin suggested).

I am not too familiar with the inner workings of DocC yet, so I didn't have that constraint in mind. I wouldn't want to tamper with these foundations right now tbh. I think we can find a good solution based on your suggestion that fits well within the existing model.

I therefore want to try to answer your questions regarding the module extension pages:

I don't think the symbol graph files currently contain the type information of the extended type (i.e. we wouldn't know if it is an actor/class/enum/struct/protocol. If that information can be added easily, I'd do that of course as I'll probably have to tamper with the generation of those files anyway. I'd suggest the terms "Extended Module" and "Extension" (if that information cannot be added easily) or as you suggested prefixed with the respective symbol kind, e.g. "Structure Extension".

I'd add a section "Extended Modules", which lists all the extended module pages.

This would also apply for the sidebar. The section header (i.e. the one on the same level as "Structures") would be "Extended Modules". This section only lists the extended module pages, not the extended symbol pages. This list's items can be expanded to reveal items for the extended type pages, which in turn can be expanded to reveal the member pages.

Basically, yes, however, using the absolute identifier would not be necessary, i.e. just # ``Swift`` or # ``Swift/Array`` would also be fine.

Absolute identifiers are always an option, but they are only necessary to refer to the original declaration (in an external module) of a shadowed member symbol, or the extended type itself.

As you mentioned, this ambiguity is not solved by absolute identifiers. I'd go with the disambiguation suffix you proposed. For example, if MyModule were to define a type called Swift with an inner type Array and also extend the standard library's Array, we'd get the following identifiers in the MyModule documentation catalogue:

For referring to the external documentation catalogue for the standard library:

/Swift

For referring to the standard library's Array type's page in the external documentation catalogue:

/Swift/Array

For referring to the locally declared type Swift in the local documentation catalogue:

Swift

or MyModule/Swift

or /MyModule/Swift

For referring to the locally declared type Swift.Array in the local documentation catalogue:

Swift/Array

or MyModule/Swift/Array

or /MyModule/Swift/Array

For referring to the "Extended Module" page for the standard library in the local documentation catalogue:

Swift-swift.module-extension

or MyModule/Swift-swift.module-extension

or /MyModule/Swift-swift.module-extension

For referring to the "Extension" page for the standard library's Array type in the local documentation catalogue:

Swift-swift.module-extension/Array

or MyModule/Swift-swift.module-extension/Array

or /MyModule/Swift-swift.module-extension/Array

I think this approach for URLs/identifiers meets all the requirements. There is no more ambiguity (i.e. even shadowed types can be referred to in documentation) and all identifiers translate 1:1 into valid URL-suffixes. Whenever we take a valid Swift symbol name (e.g. Swift.Int64) and transform it into a DocC identifier by replacing the dots by slashes, the identifier resolves to the exact same symbol. IMO, the syntax is very intuitive. The one non-intuitive case (the -swift.module-extension disambiguation syntax) only comes up if one declares a type with the same name as one of the extended modules (which should be extremely rare).

This sounds great! I think it strikes a good balance between an intuitive link syntax for the 99% of cases where you don't declare types that shadow what you import, while still making it possible to link to the right symbols in the rare cases.

Since there were a few changes to the original pitch here, it would be super helpful to update it with the changes so that we're all on the same page, since it seems like the main points of feedback have been resolved :pray: Over time, you can also consider integrating it as documentation in the Swift-DocC repo itself (so that it shows up on Documentation).

Thank you for your amazing progress and diligence here, and working through these edge-cases! Super excited for DocC getting this feature.

This is totally fair, I think the Swift-DocC project could benefit from more documentation for contributors in the area of topic URLs. This is the current documentation: Documentation.

2 Likes

I am super glad you like the outcome @franklin. I am very happy with the improvements we could achieve over my original pitch, so a big thank you to everyone who contributed their ideas and opinions!

I 100% agree! I'll create a new write-up based on the outcome of this discussion and post it here in the next couple of days.

1 Like

Reiterating everyone's thoughts that it's awesome to get this feature and a big thank you to you for working out these edge cases. :smiley:

2 Likes