SE-0346: Lightweight same-type requirements for primary associated types

Evolution Proposal Reviews
41
  • What is your evaluation of the proposal?

Strong +1, it feels like a natural extension of the language from my perspective.

I appreciate all the thought put into the criticisms of this proposal, particularly:

  • the Sequence<.Element == Int> alternative
  • the "generic protocols" discussion

I feel the authors have done an excellent job explaining their choices in these areas. I find the resulting proposal very compelling.

  • Is the problem being addressed significant enough to warrant a change to Swift?

Yes, it has the potential to simplify a lot of declarations, especially in functions. I found the "equivalent to" examples in the proposal particularly compelling.
Having written my fair share of generic framework code for a large enterprise app, I can see myself preferring this to the existing where clauses.

  • Does this proposal fit well with the feel and direction of Swift?

Yes, I think so. But this is very subjective.

At first blush, it felt like a departure from Swift as I know it. I'm very familiar with using generics and reading their signatures, so at first this was admittedly difficult for me to interpret.

However, reading the proposal with fresh eyes, this feels like a very natural way to introduce "simple generic constraints" to beginners. I found the use cases very compelling.

I have taught Swift's protocols to other programmers experienced in languages like JavaScript and Ruby. I've noticed associated types are a frequent sticking point. I think the generics-like syntax will be more approachable.

That said, as evidenced by some of the reactions in the pitch thread, experienced programmers coming from languages like C++ and Rust might find the generics-like syntax more confusing. They seem to expect more sophisticated features like "generic protocols".

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I don't have experience with this feature or the "generic protocols" feature from other languages. As a result, I found some of the pitch discussion difficult to follow.

I did find this rationale from the proposal compelling.

We believe that constraining primary associated types is a more generally useful feature than generic protocols, and using angle-bracket syntax for constraining primary associated types gives users what they generally expect, with the clear analogy between Array<Int> and Collection<Int> .

Associated types (and this syntactic sugar for them) are one of the most compelling reasons to use protocols in Swift. The proposal opens up new possibilities for API design in this space.
I trust the proposal authors when they say the "generic protocols" feature could be implemented using alternative syntax.

I would prefer using this syntax as the proposal describes, not for "generic protocols"

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I've followed the original pitch thread, read the original pitch and this updated proposal. I've thought about how I might use this in my own framework code.

1 Like
42

I knew that, so it looks like I didn't make my point clear enough.
There is also

which feels absurd. For me, it would make more sense to rule out generic protocols once and forever rather than introducing a brand new syntax because the obvious spelling is used for something else.
Generic classes use angle brackets, generic structs use angle brackets, generic enums use angle brackets, generic functions use angle brackets — but generic protocols should be written in a completely different way?

That would be more confusing than not having the feature at all, and I wish we would be more decisive instead of stirring false hopes.

3 Likes
43

(T,U) : Convertible is not modeling the relationship as a "generic protocol", it is modeling it as a binary relation over co-equal types. (Writing the types as a tuple is perhaps an abuse of notation, but it avoids needing to invent something totally novel.) Some people may only be familiar with languages where protocols are inherently a unary relation, and so allowing protocols to be generic seems like the only option for modeling a binary relation. I can understand why those people are frustrated by this proposal, because they feel like this is foreclosing on the possibility of modeling binary relations in the future. But there are languages that can directly express binary relations over types, such as Haskell, and that has significant benefits, especially for a language that centers member access as heavily as Swift. For example:

  • It would be natural for a binary protocol to have requirements (or extension functions) that are members of either type, or of neither.
  • It would be natural to be able to express that a binary protocol is commutative in its operands, avoiding boilerplate reversed conformances.
  • etc.
14 Likes
44

Strongly agree with what John says here. I think one can reasonably like or dislike the sugar aspect of this proposal, but the improvement for opaque results is very real, and it is definitely not closing us off from ever having "generic protocols".

+1 from me, simply on the basis of the new functionality for opaque results. It's a real weakness, this fixes it, and happens to make some other things easier to spell as a happy side-effect.

3 Likes
45

I think my biggest problem with the "looks like generics" aspect of the syntax is that it doesn't behave like generics.

  • Array<String> is a type
  • Collection<String> is not a type, but a constraint on one
  • any Collection<String> is a type again
  • some Collection<String> is kind of a type but only allowed in some syntactic positions

As a newcomer to Swift, I'm expected to see code like:

struct MyStruct {
    var array: Array<String>
    init(array: Array<String>) {
       self.array = array
    }
}

And understand that to generalize this, I cannot write:

struct MyStruct {
    var collection: Collection<String>
    init(collection: Collection<String>) {
       self.collection = collection
    }
}

Instead I have to write:

struct MyStruct<C: Collection<String>> {
    var collection: C
    init(collection: C) {
        self.collection = collection
    }
}

or maybe even

struct MyStruct {
    var collection: any Collection<String>
    init(collection: some Collection<String>) {
        self.collection = collection
    }
}

... all of which have significantly different meanings and performance characteristics?

And that's just the low-hanging fruit. I'm sure there are any number more examples where the chosen syntax creates landmines of understanding for beginners and experienced Swift programmers alike.

I'm no great fan of the C++-inspired generic syntax Swift has, but at least if we stick to angle brackets for type parameters and where clauses for constraints, it remains more-or-less consistent and more-or-less learnable.

I'd love to see some pie-in-the-sky thinking for "what could generics & protocols in Swift look like if we got rid of angle brackets and where clauses and did something novel, consistent and expressive! But I feel like this proposal is fighting the fundamentals of the language, to lose a very few characters compared to a more consistent syntax.

And yes, we need a syntax that allows us to write -> some Collection where ReturnType.Element == String, but we don't have to create this much confusion to do so.

3 Likes
46

These statements are all true of Collection itself, or any protocol. It's not new to this proposal.

2 Likes
47

I think that that response actually helps me understand this proposal more, as:

  • the distinction between types and protocols and the constraints they create is already confusing as hell,
  • [presuming you agree about multivariable typeclasses getting some unrelated syntax or never existing] this piece of syntax isn't useful for another purpose, and,
  • in general, in the positions in which it will be legal, for the protocols where it will see use, it will mean what you would expect, by analogy to a generic type conforming to the protocol

In fact, you basically wouldn't expect to use this for a protocol where an implementing type wouldn't use a generic for this associated type. So if we made OptionalProtocol, Wrapped would be a primary associated type, but eg. CodingKey would never be a primary associated type of Codable.

Am I starting to get it?

6 Likes
48

This proposal doesn’t lay out guidelines for how API designers ought to decide whether something ought to be a primary associated type. I imagine we’ll consider those guidelines if this proposal is accepted and there’s a follow-up to adopt it in the standard library.

That said, I think you’ve suggested a very interesting possible guideline: would a generic conforming type be likely to have that associated type as a type parameter? So, for example, a generic collection type would likely be generic over its element type, but it’s not likely to be generic over its index or iterator types.

7 Likes
49

I just did a small search for this term: Zero results — in the whole internet. So compared to generic protocols, this seems to be a much, much more advanced concept.
While I can (I think) deduct the meaning, I fail to see obvious solutions for details, like how to specify requirements. On the other hand, I don't think anyone would be confused by a generic protocol which acts like any other generic type, and it is crystal clear how they should work.

That is actually the major quarrel I have with ruling out generic protocols:
It creates an exeption for one of the building blocks of the language, as protocols can neither be generic, nor nested, which are very useful capabilities shared by all other types.
This proposal would be the last nail in the coffin for a significant unification, and even a slightly more powerful alternative can't change this.

2 Likes
50

This part of the proposal seems like it could cause confusion:

Note that default associated type witnesses pertain to the conformance, and do not provide a default at the usage site. For example, with GraphProtocol above, the constraint type GraphProtocol leaves Vertex unspecified, instead of constraining it to String .

Having the syntax show a default between the angle brackets and then that be the only place when used that it isn’t defaulted seems to violate the other symmetries that this proposal provides.

Separating the definition of an associated type from making it a primary associated type would also restore this symmetry.

It would also then reserve the = default type syntax to be used for a potential future ‘default generic type parameters’ enhancement. To give a concrete example, say Publisher were to add primary associated types. This could be declared as Publisher<Element, Error = Never> and would allow some Publisher<String> to be used instead of having to repeat Never any time a non-throwing publisher is used.

4 Likes
51

To look at this another way, I did a search for "generic protocols" (as opposed to "generic traits") and found mostly questions with accepted answers of "you want to use associated types" and articles about how to use protocols with associated types. I think when most folks want to make a protocol "generic," this is what they mean, and I think it's a great argument for the proposal as it stands.

It's also understandable that John's theoretical description of a non-existent feature has zero results. "Primary associated types" had zero results just recently, but is already becoming searchable. I assume that if/when such a feature is pitched, it will become searchable as well.

14 Likes
52

This is an interesting question, because the proposal does not provide any rationale for this choice.

Yet, the Publisher example you gave raises two questions:

  • Why should the designer of Publisher choose Never instead of Error as the default Failure type? I mean that I can understand why someone would like a Never default, probably because one uses a lot a never-failing publishers. But why is this case more important than users who use a lot of Error-failing publishers?

  • The proposal comes with an example:

    typealias SequenceOfInt = Sequence<Int>

    I conclude that you should be able to provide your own Never-defaulted publisher type:

    typealias YesPublisher<Output> = Publisher<Output, Never>
func foo() -> some YesPublisher<Foo> { ... }

    Note that I'm not 100% sure that this would compile, because it's hard to keep track of all the discussed changes. But would this be acceptable to you (naming issue aside)?

53

The actual concept I was describing is a binary relation. The Wikipedia page for that comes up immediately if you search for that phrase. Tino apparently did an exact-phrase search just to make a point.

6 Likes
54

Yes - Publisher probably wasn't the best example because, as you say, there are reasonable arguments for what the default should be. My main point was that by doing as @Moximillian and @benrimmington have suggested and keeping the associated type declaration inside the protocol body (with it then being made a primary associated type by also mentioning its name inside the angle brackets), we keep the = Type syntax available for 'default at usage site' in the future.

2 Likes
55

Quick thought: for the same reason as why we don't annotate every non-throwing ordinary function with nothrows before the result type signature's ->.

5 Likes
56

And also because the only other available default, Error, would be a terrible default.

1 Like
57

Not with Lazy in its current form. You still need to surface enough information to drive the conditional conformance. In the case of LazyMapCollection, that conditional conformance is driven by a generic parameter Base, the collection the lazy wrapper wraps. When that wrapped collection is bidirectional, so is the lazy collection.

If you just returned an opaque some Collection<Element> that happened to be a LazyMapCollection<[Int]> then the information to drive the conditional conformance is not available to the caller (even if the compiler sometimes "knows" it for optimization purposes, if the function is inlinable), so no bidirectional conformance.

This is kind of a fundamental conflict. You cannot both keep secrets, and provide functionality based on those secrets. There has to be some give and take. There's a simpler example of this: if you are returning an [Int] but you actually return some Collection<Int> you're holding back some really useful information from the caller. They would probably love to have an Array, or at very least some RandomAccessCollection<Int>, but in order for you to preserve future flexibility to return something different, you're not giving information of which they could otherwise take advantage.

A middle ground could perhaps be to create a protocol that provides just enough information to drive the conditional conformance, while reserving other flexibility:

extension Collection {
  var lazy: some LazyCollection<Self>
}

// A protocol that provides lazy versions of common
// Collection operations.
protocol LazyCollection<
  Base: Collection
>: Collection where Element = Base.Element, Index = Base.Index {
  var base: Base { get } // unwrap the laziness of the collection
  func map(...) -> some LazyCollection<Self>
}

and then, with one additional language feature we don't yet have, do something like this:

// note this next part is not currently valid Swift,
// you can't retroactively conform protocols to other
// protocols, conditionally or otherwise
extension LazyCollection: BidirectionalCollection 
  where Base: BidirectionalCollection { }

This would allow you to get the conditional conformance benefits without exposing concrete implementations like LazyMapCollection , LazyFilterCollection.

Note that unlike Collection, this chooses a whole collection, Base, as the primary associated type for LazyCollection, not just the Element type. This allows for the unwrap operation to return the original eager collection, and follows the pattern we've a few times where primary associated types usually match generic arguments of concrete types implementing the protocol.

Of course, this is quite a lot of effort to go to purely to hide the type implementation details of things like LazyMapCollection. It may well not be worth it, especially as the Lazy types in the standard library are fragile so you don't get the ABI benefits, only the "hide the types" benefit. And the nesting e.g. some LazyCollection<some LazyCollection<some LazyCollection<[Int]>>> still gets exposed. But it may be more compelling in other similar cases.

6 Likes
58

Observation while drafting the adoption of this feature in the stdlib: having all associated types defined using consistent syntax is really valuable. It would be a shame to lose this consistency.

The associatedtype declaration serves as a useful place to attach API documentation, declaration modifiers like the (undocumented) override attribute, and it's just nice to have all of the associated types defined in the same way.

Example 1: Sequence (Doc Strings)

(This is the same issue as @allevato has raised above.)

We currently have:

public protocol Sequence {
  /// A type representing the sequence's elements.
  associatedtype Element

  /// A type that provides the sequence's iteration interface and
  /// encapsulates its iteration state.
  associatedtype Iterator: IteratorProtocol where Iterator.Element == Element
}

This is nice and symmetric, everything is consistent.

As proposed, this would now look like:

public protocol Sequence<Element> {
  /// A type that provides the sequence's iteration interface and
  /// encapsulates its iteration state.
  associatedtype Iterator: IteratorProtocol where Iterator.Element == Element
}

We no longer have an obvious place for Element's documentation.

Of course, we can still put it on the name itself, but this makes the declaration less readable:

public protocol Sequence<
  /// A type representing the sequence's elements.
  Element
> {
  /// A type that provides the sequence's iteration interface and
  /// encapsulates its iteration state.
  associatedtype Iterator: IteratorProtocol where Iterator.Element == Element
}

(Sometimes the doc string will have multiple paragraphs of text. Having it interrupt the protocol declaration would be a shame.)

One solution would be to integrate it into the doc string of the protocol itself in some way. This would work, but it makes the primary associated type even less similar to the rest of the associated types:

/// <Long discussion on what Sequence is all about>
///
/// - Associated type Element: A type representing the sequence's elements.
protocol Sequence<Element> {
  /// A type that provides the sequence's iteration interface and
  /// encapsulates its iteration state.
  associatedtype Iterator: IteratorProtocol where Iterator.Element == Element
}

Example 2: Collection (Associated Types with Modifiers)

Collection refines Sequence, and it wants to have the same primary associated type, Element.

In the current version of the stdlib, Collection redeclares Element with the (undocumented!) override modifier, to help associated type inference:

public protocol Collection: Sequence {
   // FIXME: Associated type inference requires this.
  override associatedtype Element
}

There is no obvious place to put such a modifier in the primary position:

public protocol Collection<Element>: Sequence {
}

Of course, we don't need such a modifier if Collection<Element> has the same semantics as the override associatedtype Element declaration. Which leads us to the last example:

Example 3: RangeReplaceableCollection (Inherited Associated Types)

RangeReplaceableCollection refines Collection, without explicitly overriding Element.

public protocol RangeReplaceableCollection: Collection {
   ...
}

If we want it to have the same primary associated type, we'll want to write:

public protocol RangeReplaceableCollection<Element>: Collection {
  ...
}

However, will Element here refer to the existing associated type that was inherited from Sequence, or will it introduce a new associated type with the same name?

Recomended Solution

My preferred solution would be to leave the existing associatedtype declarations as is. The bracketed syntax is a nice way to designate one of the associated types as the primary, but it should not define the associated type; it should just reference it.

public protocol Foo<Bar> { // error: Unknown associated type 'Bar'
}

For cases like Sequence, we would be able to keep the doc string where it belongs:

protocol Sequence<Element> {
  /// A type representing the sequence's elements.
  associatedtype Element

  /// A type that provides the sequence's iteration interface and
  /// encapsulates its iteration state.
  associatedtype Iterator: IteratorProtocol where Iterator.Element == Element
}

For cases like Collection, we wouldn't need to figure out a new way to attach modifiers:

public protocol Collection<Element> {
   // FIXME: Associated type inference requires this.
  override associatedtype Element

  /// A type that provides the sequence's iteration interface and
  /// encapsulates its iteration state.
  associatedtype Iterator: IteratorProtocol where Iterator.Element == Element
}

For cases like RangeReplaceableCollection, it would be clear that the declaration of the primary associated type isn't going to break ABI by adding a new associated type to the protocol:

public protocol RangeReplaceableCollection<Element>: Collection {
  ...
}
31 Likes
59

I can't let this passage go unnoticed:

However, duplicating the associated type declaration in this manner is still an error-prone form of code duplication, and it makes the code harder to read. We feel that this use case should not unnecessarily hinder the evolution of the language syntax. The concerns of libraries adopting new language features while remaining compatible with older compilers is not unique to this proposal, and would be best addressed with a third-party pre-processor tool.

(Emphasis mine)

Library authors should not be considered second class citizens. Their concerns ought to be considered just as important as the concerns of the clients of said libraries. Library maintainers are people too; there is a limit to how much pain they are willing to suffer.

Forcing us to reach for preprocessing tools means that we're going to lose basic modern software engineering features, like intelligent code completion / syntax highlighting, structural editing, etc. etc. This will lead to lower quality output.

It will also make library code be even less similar to regular Swift code, making contributions that much more difficult.

(That said, most Swift packages have the option to simply bump their required toolchain version in a minor release, which resolves this pain.)

25 Likes
60

6 posts were split to a new topic: Language downgrade tool for source libraries