[Pitch 2] Light-weight same-type requirement syntax

What you're referring to "generic protocols" are really more like "multi-Self" protocols, where there are multiple types involved in a conformance without a functional dependency between them, in contrast to the relationship from the Self type to associated types in a protocol conformance today. Although Rust uses generics syntax for these, I don't think that's necessarily the best choice, because it implies that one type is more important to the relationship, and that is the exact opposite of what the feature means. Generic argument syntax on the other hand already implies a functional dependency for non-protocol types—given any instance of Array, for instance, you can recover its Element type from that instance, since there is no value that is both an Array<Int> and Array<String>. By analogy, any generic value using a particular conformance has only one possible binding for its associated types, so it seems appropriate for primary associated types on a protocol to be notated that way as well. We can adopt this syntax now, and still consider other ways to express multiple-parameter conformances. (One strawman might be to declare such a protocol as protocol Convertible(from: T, to: U), provide conformances by extension Convertible(from: Int32, to: Int64), and express constraints as <T, U> where Convertible(from: T, to: U).)

11 Likes

At the protocol declaration, only a single primary associated type is allowed.

I don’t understand the motivation for this restriction. Why can’t I have a protocol Functor<Arg, Return>?

Right, as you mention, this would expose part of the concrete type's API through the opaque type, whereas normally the opaque type's behavior is defined entirely in terms of its generic requirements. I would rather we come up with a way of abstracting over the throwing effect via an opaque type than go down the route of exposing implementation details of the concrete type.

The gotcha is that using a some return type cannot change its effects. If it ever threw it may not become non throwing without breaking at the very least API contract (perhaps ABI) or visa-versa.

Right, exactly. This would defeat the purpose of an opaque type at least to some extent.

So unless we surface generic effects or some other solution in that space of determining the throwyness of a conformance I fear that this feature won't be able to be used meaningfully for any AsyncSequence work.

You would only be able to write code that assumed the AsyncSequence throws -- that doesn't seem like the end of the world, does it?

1 Like

At this point several people have pushed back. There's no real motivation for this restriction, except that it made my prototype implementation with the @_primaryAssociatedType attribute simpler. In principle I'm not opposed to generalizing it to allow multiple associated types (but at that point I'd like us to come up with a better term than "primary associated type").

11 Likes

Well by your own suggestion; the signature should be some AsyncSequence<T, nothrow> or something like that. So to me that feels like this is incomplete and should not be used yet for AsyncSequence yet the pitch uses that as one of the first justifications.

edit: clarification on my saltiness
I think that generally for non effectful types such as Collection etc this pitch is amazingly useful. I just wish we had a better way of expressing the types that have effects. some Sequence<String> is great syntax imho. I just hope we can get to a point very soon that allows for some AsyncSequence<String> too without much more overhead to the developer.

2 Likes

I am confused by this explanation. An expression that evaluates to an instance of Array must have a static type of Array<Element>, so “recovery” is trivial. For generic protocols, does “recovery” of type parameters occur only in the presence of existentials? Can the same restriction not be applied to generic protocols to require their types always be fully specified? E.g.:

protocol Convertible<To> {
  func convert() -> To
}

extension Int: Convertible<Float> {
  func convert() -> Float { ... }
}

extension Convertible<String> {
  func convert() -> String { ... }
}

let myNumber: Int = 1234
(myNumber as Convertible<Float>).convert() // 1234.0
myNumber.convert() // error: multiple matches for func convert()
myNumber as any Convertible // error: incomplete type Convertible<_>
1 Like

+1, I'm very much in favor of this change. If it means we can have opaque return values with specified associated types, this is absolutely a much-needed feature.

On the topic of "generic protocols" and Rust, I'm not sure if the feature I'm thinking of in Rust is the same as what people mean by "generic protocols". If I understand correctly, Rust's feature is more about how to express many-to-many ad-hoc type relationships. The canonical example being what types can be cast to what types.

For that feature, I don't think we want type-parameter syntax, because what we're defining is not parametric, it's ad-hoc. It's analogous to function overloading rather than generics. A subset of these would be a single type conforming to a protocol in multiple ways.

I've hit this need before with String, which is a type that provides multiple models of string: extended grapheme clusters, Unicode scalar values, and UTF-8 code units. If a type wanted to conform to a protocol dealing with String, but provide a different conformance for each of String's models, then currently that type must be parameterized somehow (e.g. over the Element of the corresponding view). Or, the type has to provide a unique type for each conformance and add API to access them.

For example, in the consumer/searcher prototype from way back when, I was trying to conform CharacterSet to a protocol over String. I wanted to support both scalar and grapheme cluster processing. For that, I had to make separate types. Funnily enough, while the fundamental limitation of not allowing multiple conformances is still present, allowing these types to be opaque return values could alleviate some of the bloat associated with these kinds of workarounds.

That prototype is graduating into something real. If this avoids having every single generic algorithm needing to define multiple public types (and symbols!), that honestly could be the difference between shipping this in the stdlib or not.

I've also hit this same issue in multiple different API I wanted to add to System. Without opaque return values with associated types, I couldn't justify shipping them.

3 Likes

Yeah, I was discussing this with @Joe_Groff privately and he explained to me that Rust's generic traits are analogous to Haskell's multi-parameter typeclasses. So rather than a type T conforming to a "generic instantiation" of a protocol MyProto<U>, it is more that a pair of types (T, U) together conform to MyProto.

4 Likes

Well, if that's doable it changes the calculus considerably. This is the first time I've even heard of the possibility of multi-self protocols. Would both a Convertible(from: A, to: B) and a Convertible(from: B, to: A) conformance be possible under this model?

1 Like

This illustrates a greater point that I'd like to raise.

On the syntax, I'm personally unsure and a bit sceptical, but there is one part of this proposal that I'm very, very unhappy with: that it is actually two, very big features, bundled in to one proposal. That's generally considered bad form, as it means we can't fairly evaluate either of them in isolation.

Perhaps we'd accept the sugar anyway, or perhaps we wouldn't, but we're sort of "held hostage" (no malice implied) because it seems we can only get constrained opaque types if we also take the sugar. It's like meeting a lost traveller in the desert and offering them Coca Cola or nothing - they'll take it if there's no alternative, but you can't walk away from that and claim "people dying of thirst prefer Coca Cola!"

Moreover, the available constraints on opaque types are severely limited. You'd only be able to use them with protocols whose authors have declared one associated type to be more important than the others, and you'd only be able to constrain that particular type, and only with a same-type constraint (possibly subtypes too? But that's speculative and not formally part of the proposal). Those are limitations of the sugar, but because they come together, it applies to the (IMO more important) functional enhancements around constrained opaque types as well.

I'll consider Joe's response tomorrow because I'm unfamiliar with Haskell and it's too late to look in to it in depth, but I wanted to briefly touch on this point. This thing should be 2 proposals, and combining them does each feature, the language, and the community a disservice by unfairly shielding them from proper and balanced scrutiny.

7 Likes

I'm not sure it would make sense to consider one without the other. If we allow one to write func foo<T : Sequence<Int>>() as shorthand for func foo<T : Sequence>() where T.Element == Int, why would we not also allow func foo() -> some Sequence<Int>, and vice versa?

The sugar applies in any position where the right-hand side of a protocol conformance requirement can appear; this happens to include inheritance clauses, where clauses, extensions, and opaque result types. I don't think there's much value in separating out into two or more proposals.

More general constraints on opaque result types can be accomplished with named opaque result types; something like this for example:

func foo() -> <T, U> where T : Sequence, U : Sequence, T.Element == U.Element

This pitch does not foreclose the possibility of introduced named opaque result types.

It's true that a protocol has to be designed with this usage in mind, but at the very least I'm pretty certain we can safely generalize to multiple generic arguments with the same syntax:

protocol DictionaryProtocol<Key, Value> { ... }

I plan on implementing that next week to experiment with.

I'm not suggesting we take the sugar before opaque type constraints: I'm suggesting we need constraints on opaque types before we consider sugaring them. Which means the sugar coming later.

That's not the concern. The concern is that we have evaluations of the proposal like this:

"If it means we can have opaque return values with specified associated types, this is absolutely a much-needed feature."

If I may rephrase to how I interpret that, it's saying "if I need to accept the sugar to get the feature I really want, I'll take the sugar". Which is not as glowing an endorsement as it might seem on first reading. It also means the order in which these features are added to the language affects how they are perceived. By combining the sugar with the functional enhancement it sugars, we lose the ability to fairly consider each of them separately on their individual merits.

12 Likes

Do you think that it could be confusing to have 2 or more primary associated typeS? Maybe. Ideally, it would be nice if we could come up with a name in which we replace associated_ with something else, so an associatedtype would remain the same as it is right now, and the proposal would introduce some that's, conceptually, entirely new.

Here's some ideas:
primarytype
requiredtype
parameterizedtype
characterizingtype
constrainedtype
identifyingtype
definingtype

1 Like

I cannot believe how hard this syntax is being pushed. It makes me really sad actually. The evolution of Swift isn't a democracy, yet the whole process is being very well "selective" as authors of the proposals often times skip and ignore messages from the community which collide with their ideas outlined in the proposals.

I use Swift on nearly a daily basis and I care about the expressiveness of the language a lot, especially on the generics front. This syntax is not generic and should not allude to be generic. The pitched syntax is jumping over way too many hoops at once, but at what cost?

(A) The language already made a partial promise on the ability to create constraints using the where clause inside a typealais declaration, but it still hasn't delivered that feature to us.

(B) A fair while ago we already extensively discussed the problems in that area and pitched an alternative syntax which should provide some improvements while also maintaining the flexibility on the users side. That syntax was Collection<.Index == Whatever>.

Now this pitch (C):

  • It tries to make it look like either (A) or (B) were mistakes and we need further improvements, although I as a Swift user never seen the daylight of any of those. You cannot judge on issues on something that never was delivered.
  • It completely removes the flexibility which we could have with (A) and (B). What do you want me the developer to write when for instance Collection has only a single primary associated type being Element but I still want to constrain Index as well?
    // How is this
    Collection<Foo> where Index == Bar
    // better than this (other than it being a few character shorter)
    Collection<.Element == Foo, .Index == Bar>
    
  • The pitched syntax is only focusing on "same-type" constraints while it isn't the only thing that is actually useful:
    Collection<any P> // .Element == any P
    // what if I want `Element` to actually conform to `P`?
    Collection<.Element: P>
    
  • The limitation of having a single primary associated type is artificial and just highlights how unsure this whole pitched syntax is.
  • I don't even want to touch the discussion around generic protocols, because it seems there is a huge fear or other strong feelings against that feature which often times kills the whole conversation.

Long story short, I'm strongly against the currently pitched form of this proposal. So -1 for me. The last part of the alternative consideration is something that I personally would rather support as it builds on the integrity of the language rather than trying very hard to write a few less characters with some sugar code!

26 Likes

I very much agree with points raised against this, it's -1 from me against the pitch in its current form. I like the intention to help with the use cases, but the solution doesn't feel right to me because:

  1. By reusing generics syntax, while not actually being a "generic protocol" it introduces more confusion.
  2. If this is pitched sugar for a more general feature, I feel it would be more appropriate to pitch this more general feature first before introducing any sugar for it.
  3. I don't think that the prior art of generic protocols (traits or type classes etc) has been sufficiently explored here. I see them being dismissed as "we don't need them", but that doesn't seem like a good justification. Rust and Haskell apparently did need them after all, and I would like to see how code using this feature in Rust and/or Haskell would translate to Swift without generic protocols. Or some other more detailed and more convincing explanation that addresses this point.
22 Likes

One more thing:

The alternative syntax can still drive this: some Collection<.Element == some P> or some Collection<.Element: P> which essentially means the same thing and on top of that, it actually provides more context on what is being constrained as often times protocols have a great set of hard discoverable associated types. IMO the currently pitched syntax adds more cognitive load to the protocol user. The user has to instinctively know the names of the associated types at the generic parameter type's position and their order.

protocol P<Element, Index> {}

// when reading the following:
// - I don't know the names of the associated types (if `P` is not my creation)
// - the ORDER of the associated types becomes important 
some P<String, Int> 

// even though this is a bit longer:
// - there is no need to remember the order of the associated types
// - IDEs with autocompletion can help discover the associated types after typing a leading dot
some P<.Index == Int, .Element == String> // same as `some P<.Element == String, .Index == Int>`

It also becomes impossible to skip associated types unless we want to mix this with placeholder type syntax:

some P<_, Int> 
some P<String, _> 
// vs.
some P<.Index == Int>
some P<.Element == String>

At least the shorter syntax adds cognitive load to my brain.

8 Likes

Also worth remembering:

In order for library users to take advantage of this using protocols they import from a dependency, changes are needed at the declaration site. That means, initially, some protocols will support this feature (probably Apple's ones), and no other protocols will work. That's likely to be very confusing for users, and it will require a transition period before users can just take that support for granted.

What's more, they're then going to report those failures as bugs to the library authors, who are going to be pressured to duplicate their protocols behind #if swift guards because these declarations are incompatible with all prior versions of Swift:

#if swift(>=5.7)

public protocol MyProtocol<Thing> {
//  ...
//  some long protocol, including all of its:
//  - other associated types
//  - requirements
//  - documentation for those associated types and requirements
//  ...
}

#else

public protocol MyProtocol {
  associatedtype Thing
//  ...
//  some long protocol, including all of its:
//  - other associated types
//  - requirements
//  - documentation for those associated types and requirements
//  ...
}

#endif

So it's going to look like that, multiplied by every protocol in your library. Also, because it needs to be specified at the definition site, this isn't really just sugar -- this is the new way you're going to have to write every protocol from now on, because users are going to expect this syntax to work.

On the other hand, something which is purely use-site sugar (like MyProtocol<.Thing: Numeric>) would work with all protocols on day one. No transition period. No code duplication for library authors. It has a number of other benefits as well (such as a much smoother migration once you reach the limits of the sugar and need full generics syntax), but I want to focus on the above points.

19 Likes

Big "agree" on everything @Karl, @DevAndArtist and @Max_Desiatov said so far.

This part:

and this part in particular:

Even the Swift Generics Manifesto addresses "generic protocols" in the very same dismissive tone and in the same hand-wavy manner based on the weird straw-man argument that "people misuse the term" and thus the real thing is of no use to Swift:

One of the most commonly requested features is the ability to parameterize protocols themselves. […] Implicit in this feature is the ability for a given type to conform to the protocol in two different ways. […] Most of the requests for this feature actually want a different feature.

Not only is the last part evidently wrong (i.e. none of the request to "please consider generic protocols as future addition to the language" in this or the previous pitch discussion were "actually asking for a different feature" afaict), but it also fails to provide any proper argument against generic protocols in Swift whatsoever.

How does the Swift team plan to fill the gaps that I outlined further up in the discussion if not by some kind of generic protocols (name them however you want)?

1 Like

I do like the parens syntax suggested above. While it is floated as some potential future syntax for "generic protocols", what if it's used for the syntax sugar proposed in the pitch?

protocol Sequence(Element) {
  // ...
}

func concatenate<S : Sequence(String)>(_ lhs: S, _ rhs: S) -> S {
  // ...
}

This also could resolve the issue with throwing associated type of AsyncSequence by reusing the familiar syntax for default arguments:

protocol AsyncSequence(Element, Failure = Never) {
    associatedtype Iterator : AsyncIteratorProtocol
      where Element == Iterator.Element
  // ...
}

This way some AsyncSequence(Int) is equivalent to some AsyncSequence(Int, Never), while still allowing a more explicit some AsyncSequence(Int, Error) form for throwing async sequences.

One other benefit is that this still leaves the possibility for some future "generic protocols" open without introducing any confusion with existing features related to generics.

2 Likes

A few thoughts:

I think that "normal" protocols with associated types are uncontroversially far more useful than generic protocols. Indeed, we have made it (nearly) to Swift 6 without significant outcry for true generic protocols—I don't believe they represent a dire expressivity gap in the language as would be felt without associatedtype. (Granted, perhaps this is a lack of imagination on my part due to the fact that we have PATs and we don't have generic protocols, but I don't think so.)

At the point of use of protocols with associated types, I think the angle bracket syntax matches what most Swift programmers (especially new Swift programmers) expect to "just work". An Array of Strings is spelled Array<String>, it seems natural to me that "some Collection which contains Strings" be spelled some Collection<String>. I view this alignment as highly desirable—the status quo of having to drop down to where clauses is a clear design gap, and the Collection<.Element == String> variants still introduce a significant departure from the fundamental 'thing' that people are trying to express. There really is, conceptually, a 'primary' associated type for Collection, with other associated types being secondary.

Is there a fundamental issue on the use side where same-type constraints differs in behavior from generic protocols? The best example I've seen in this thread is the confusion that may arise from attempting to conform to, e.g., Collection<String>, but I expect such situations will be vastly rarer than trying to use some Collection<String>, so I'm not convinced that it alone justifies disadvantaging the point-of-use syntax. It seems likely that we could offer a straightforward diagnostic to turn struct Lines : Collection<String> { ... } from the proposal into the likely-intended typealias version.

Particularly if there's another reasonable formulation of generic protocols (such as "multi-Self protocols" as mentioned by Joe) that covers all the use cases while still being markedly different than 'normal' generic types, I think it is justifiable to give same-type constraints the 'privileged' angle bracket syntax and leave this other feature for potential future consideration. Perhaps a loss for perfect purity of Swift's angle-brackets-as-generics syntax, but IMO it is a pragmatic choice that will make the language more usable in the vast majority of cases.

Of course, I'm making a lot of broad claims here based on gut instinct. It would be great if we had some empirical data that could help inform this proposal, but I'm not sure exactly what form that would take. We don't exactly incorporate scientific user studies into the evolution process. :sweat_smile:

2 Likes