[Pitch 2] Light-weight same-type requirement syntax

I'm not suggesting we take the sugar before opaque type constraints: I'm suggesting we need constraints on opaque types before we consider sugaring them. Which means the sugar coming later.

That's not the concern. The concern is that we have evaluations of the proposal like this:

"If it means we can have opaque return values with specified associated types, this is absolutely a much-needed feature."

If I may rephrase to how I interpret that, it's saying "if I need to accept the sugar to get the feature I really want, I'll take the sugar". Which is not as glowing an endorsement as it might seem on first reading. It also means the order in which these features are added to the language affects how they are perceived. By combining the sugar with the functional enhancement it sugars, we lose the ability to fairly consider each of them separately on their individual merits.

12 Likes

Do you think that it could be confusing to have 2 or more primary associated typeS? Maybe. Ideally, it would be nice if we could come up with a name in which we replace associated_ with something else, so an associatedtype would remain the same as it is right now, and the proposal would introduce some that's, conceptually, entirely new.

Here's some ideas:
primarytype
requiredtype
parameterizedtype
characterizingtype
constrainedtype
identifyingtype
definingtype

1 Like

I cannot believe how hard this syntax is being pushed. It makes me really sad actually. The evolution of Swift isn't a democracy, yet the whole process is being very well "selective" as authors of the proposals often times skip and ignore messages from the community which collide with their ideas outlined in the proposals.

I use Swift on nearly a daily basis and I care about the expressiveness of the language a lot, especially on the generics front. This syntax is not generic and should not allude to be generic. The pitched syntax is jumping over way too many hoops at once, but at what cost?

(A) The language already made a partial promise on the ability to create constraints using the where clause inside a typealais declaration, but it still hasn't delivered that feature to us.

(B) A fair while ago we already extensively discussed the problems in that area and pitched an alternative syntax which should provide some improvements while also maintaining the flexibility on the users side. That syntax was Collection<.Index == Whatever>.

Now this pitch (C):

  • It tries to make it look like either (A) or (B) were mistakes and we need further improvements, although I as a Swift user never seen the daylight of any of those. You cannot judge on issues on something that never was delivered.
  • It completely removes the flexibility which we could have with (A) and (B). What do you want me the developer to write when for instance Collection has only a single primary associated type being Element but I still want to constrain Index as well?
    // How is this
    Collection<Foo> where Index == Bar
    // better than this (other than it being a few character shorter)
    Collection<.Element == Foo, .Index == Bar>
    
  • The pitched syntax is only focusing on "same-type" constraints while it isn't the only thing that is actually useful:
    Collection<any P> // .Element == any P
    // what if I want `Element` to actually conform to `P`?
    Collection<.Element: P>
    
  • The limitation of having a single primary associated type is artificial and just highlights how unsure this whole pitched syntax is.
  • I don't even want to touch the discussion around generic protocols, because it seems there is a huge fear or other strong feelings against that feature which often times kills the whole conversation.

Long story short, I'm strongly against the currently pitched form of this proposal. So -1 for me. The last part of the alternative consideration is something that I personally would rather support as it builds on the integrity of the language rather than trying very hard to write a few less characters with some sugar code!

26 Likes

I very much agree with points raised against this, it's -1 from me against the pitch in its current form. I like the intention to help with the use cases, but the solution doesn't feel right to me because:

  1. By reusing generics syntax, while not actually being a "generic protocol" it introduces more confusion.
  2. If this is pitched sugar for a more general feature, I feel it would be more appropriate to pitch this more general feature first before introducing any sugar for it.
  3. I don't think that the prior art of generic protocols (traits or type classes etc) has been sufficiently explored here. I see them being dismissed as "we don't need them", but that doesn't seem like a good justification. Rust and Haskell apparently did need them after all, and I would like to see how code using this feature in Rust and/or Haskell would translate to Swift without generic protocols. Or some other more detailed and more convincing explanation that addresses this point.
22 Likes

One more thing:

The alternative syntax can still drive this: some Collection<.Element == some P> or some Collection<.Element: P> which essentially means the same thing and on top of that, it actually provides more context on what is being constrained as often times protocols have a great set of hard discoverable associated types. IMO the currently pitched syntax adds more cognitive load to the protocol user. The user has to instinctively know the names of the associated types at the generic parameter type's position and their order.

protocol P<Element, Index> {}

// when reading the following:
// - I don't know the names of the associated types (if `P` is not my creation)
// - the ORDER of the associated types becomes important 
some P<String, Int> 

// even though this is a bit longer:
// - there is no need to remember the order of the associated types
// - IDEs with autocompletion can help discover the associated types after typing a leading dot
some P<.Index == Int, .Element == String> // same as `some P<.Element == String, .Index == Int>`

It also becomes impossible to skip associated types unless we want to mix this with placeholder type syntax:

some P<_, Int> 
some P<String, _> 
// vs.
some P<.Index == Int>
some P<.Element == String>

At least the shorter syntax adds cognitive load to my brain.

8 Likes

Also worth remembering:

In order for library users to take advantage of this using protocols they import from a dependency, changes are needed at the declaration site. That means, initially, some protocols will support this feature (probably Apple's ones), and no other protocols will work. That's likely to be very confusing for users, and it will require a transition period before users can just take that support for granted.

What's more, they're then going to report those failures as bugs to the library authors, who are going to be pressured to duplicate their protocols behind #if swift guards because these declarations are incompatible with all prior versions of Swift:

#if swift(>=5.7)

public protocol MyProtocol<Thing> {
//  ...
//  some long protocol, including all of its:
//  - other associated types
//  - requirements
//  - documentation for those associated types and requirements
//  ...
}

#else

public protocol MyProtocol {
  associatedtype Thing
//  ...
//  some long protocol, including all of its:
//  - other associated types
//  - requirements
//  - documentation for those associated types and requirements
//  ...
}

#endif

So it's going to look like that, multiplied by every protocol in your library. Also, because it needs to be specified at the definition site, this isn't really just sugar -- this is the new way you're going to have to write every protocol from now on, because users are going to expect this syntax to work.

On the other hand, something which is purely use-site sugar (like MyProtocol<.Thing: Numeric>) would work with all protocols on day one. No transition period. No code duplication for library authors. It has a number of other benefits as well (such as a much smoother migration once you reach the limits of the sugar and need full generics syntax), but I want to focus on the above points.

19 Likes

Big "agree" on everything @Karl, @DevAndArtist and @Max_Desiatov said so far.

This part:

and this part in particular:

Even the Swift Generics Manifesto addresses "generic protocols" in the very same dismissive tone and in the same hand-wavy manner based on the weird straw-man argument that "people misuse the term" and thus the real thing is of no use to Swift:

One of the most commonly requested features is the ability to parameterize protocols themselves. […] Implicit in this feature is the ability for a given type to conform to the protocol in two different ways. […] Most of the requests for this feature actually want a different feature.

Not only is the last part evidently wrong (i.e. none of the request to "please consider generic protocols as future addition to the language" in this or the previous pitch discussion were "actually asking for a different feature" afaict), but it also fails to provide any proper argument against generic protocols in Swift whatsoever.

How does the Swift team plan to fill the gaps that I outlined further up in the discussion if not by some kind of generic protocols (name them however you want)?

1 Like

I do like the parens syntax suggested above. While it is floated as some potential future syntax for "generic protocols", what if it's used for the syntax sugar proposed in the pitch?

protocol Sequence(Element) {
  // ...
}

func concatenate<S : Sequence(String)>(_ lhs: S, _ rhs: S) -> S {
  // ...
}

This also could resolve the issue with throwing associated type of AsyncSequence by reusing the familiar syntax for default arguments:

protocol AsyncSequence(Element, Failure = Never) {
    associatedtype Iterator : AsyncIteratorProtocol
      where Element == Iterator.Element
  // ...
}

This way some AsyncSequence(Int) is equivalent to some AsyncSequence(Int, Never), while still allowing a more explicit some AsyncSequence(Int, Error) form for throwing async sequences.

One other benefit is that this still leaves the possibility for some future "generic protocols" open without introducing any confusion with existing features related to generics.

2 Likes

A few thoughts:

I think that "normal" protocols with associated types are uncontroversially far more useful than generic protocols. Indeed, we have made it (nearly) to Swift 6 without significant outcry for true generic protocols—I don't believe they represent a dire expressivity gap in the language as would be felt without associatedtype. (Granted, perhaps this is a lack of imagination on my part due to the fact that we have PATs and we don't have generic protocols, but I don't think so.)

At the point of use of protocols with associated types, I think the angle bracket syntax matches what most Swift programmers (especially new Swift programmers) expect to "just work". An Array of Strings is spelled Array<String>, it seems natural to me that "some Collection which contains Strings" be spelled some Collection<String>. I view this alignment as highly desirable—the status quo of having to drop down to where clauses is a clear design gap, and the Collection<.Element == String> variants still introduce a significant departure from the fundamental 'thing' that people are trying to express. There really is, conceptually, a 'primary' associated type for Collection, with other associated types being secondary.

Is there a fundamental issue on the use side where same-type constraints differs in behavior from generic protocols? The best example I've seen in this thread is the confusion that may arise from attempting to conform to, e.g., Collection<String>, but I expect such situations will be vastly rarer than trying to use some Collection<String>, so I'm not convinced that it alone justifies disadvantaging the point-of-use syntax. It seems likely that we could offer a straightforward diagnostic to turn struct Lines : Collection<String> { ... } from the proposal into the likely-intended typealias version.

Particularly if there's another reasonable formulation of generic protocols (such as "multi-Self protocols" as mentioned by Joe) that covers all the use cases while still being markedly different than 'normal' generic types, I think it is justifiable to give same-type constraints the 'privileged' angle bracket syntax and leave this other feature for potential future consideration. Perhaps a loss for perfect purity of Swift's angle-brackets-as-generics syntax, but IMO it is a pragmatic choice that will make the language more usable in the vast majority of cases.

Of course, I'm making a lot of broad claims here based on gut instinct. It would be great if we had some empirical data that could help inform this proposal, but I'm not sure exactly what form that would take. We don't exactly incorporate scientific user studies into the evolution process. :sweat_smile:

2 Likes

Or, better, we accept this shorthand spelling as-is also.

I like the shorthand as proposed, even though I know it is overloading the syntax with generics it's compelling to me. Collection<String> feels very natural if I just want to map a collection of strings, and I think there is sufficient leeway for evolution like Collection<String, .Index == Foo> or whatever down the road.

-1, from me, mostly because of prioritization. As @Karl wrote earlier, this proposal tries to address both expressiveness and light-weight syntax for common cases.

I think we should address expressiveness first, and return to discussion of light-weight syntax after gaining practical experience with constrained protocols in opaque types, existentials and typealiases in production.

If we can express constrained protocols in typealiases, then we may not need lightweight syntax at all:

/// SequenceOf is a constraint
/// Constraints are generalization of protocols
/// Constraints are reusable pieces of where clause
/// Constraints can be generic
typealias SequenceOf<T> = Sequence where Element == T
func concatenate<S : SequenceOf<String>>(_ lhs: S, _ rhs: S) -> S {
   ...
}

10 Likes

Is this not a concern with every new hypothetical language feature though?

1 Like

In this case it wouldn't be as bad if you could retroactively add the primary associated type to the protocol; then only that extension part would go between the #if.

Yes, users either need to wait to adopt features or add conditionals to adopt them while still supporting other version. For instance, I needed to add some swift(>=5.5) blocks to Alamofire to support the new static member capabilities. However, this case, as Karl pointed out, is especially egregious, as, due to Swift's limited #if syntax, it requires duplicating the entire protocol definition to just to support some small bit of new syntax. That's basically the worst case scenario. Community adoption of the feature will be very low until they reasonably require the right version of Swift, which usually occurs once Apple stops accepting App Store submission with older versions of Xcode, and sometimes not even then. So any feature which can't be easily adopted with swift blocks will necessarily lag in adoption, which is bad for the language.

Of course, there are a lot of things Apple could do to help this case that aren't related to the language it self, but...

5 Likes

This feature would need a detailed pitch outlining the precise semantics. Typealiases can appear in many places; if we're introducing a new kind of typealias we can to spell out where they're allowed and where they're forbidden, and what the behavior is in each case.

Another consideration is that a hypothetical typealias for where clauses would also not be as ergonomic as the concise syntax I'm promising in many instances because you'd have to name the alias.

The syntax is not meant to replace where clauses entirely. You can still write a where clause.

I think we need to articulate what we mean by generic protocols. A protocol type constructor instantiated on a set of concrete types doesn't seem like what you want; I think rather folks are asking for multi-parameter type classes. Today a protocol conceptually has a single 'Self' generic parameter; a multi-parameter type class would have multiple 'Self' types, none of them more privileged over another; where any conformance stated in a requirement or extension to such a protocol involves multiple 'Self' types. I'm not sure Self1 : Proto<Self2> syntax is the one that you want here.

2 Likes

This syntax is ambiguous because you just defined a protocol where the underlying type is the existential type any Sequence. The parameter T that you instantiate the type constructor with doesn't appear in the underlying type, so it doesn't play a role here.

However note that you can already express something like this via requirement inference in parameter position, just slightly awkwardly:

typealias SequenceOf<S, E> = S where S : Sequence, S.Element == E

func contains<S, E>(_ seq: SequenceOf<S, E>, _ elt: E) -> Bool {}

I'm hesitant to draw this kind of inference. If folks feel that constraints on opaque result types are desirable but the current syntax is bad, they can state that and suggest an alternate approach, such as named opaque result types that one can refer to from the where clause, like this:

func returnSequenceOfString() -> <S> S where S : Sequence, S.Element == Int

The underlying mechanism to define requirements on opaque result types is already there, just not officially. In fact there is an experimental flag to enable named opaque result types (but I wouldn't rely on it for anything real of course).

Sendable has the same issue, where we'd like it to act more like an effect that's carried through from the arguments to the opaque result type. I'd rather us tackle this issue with opaque result types directly, since it's independent of the sugar here.

Doug

2 Likes

I wrote that bit of the Generics Manifesto many years ago, and that last part has held constant for many years since then: the vast majority of requests we get for "parameterized protocols" or "generic protocols" specifically ask to be able to write Collection<String> to get a collection of strings, which aligns with this pitch. Let's call that interpretation #1.

There are other interpretations we could give to this syntax as well, so let's consider them.

Interpretation #2 is multi-Self protocols, as Joe points out. The canonical example is expressing conversions, e.g., ConvertibleTo<Double, Int>:

protocol ConvertibleTo<Other> {
  func asOther() -> Other   // "Self" is the type we convert from, "Other" is the type we're converting to
}

Except in Swift we use converting initializers, so maybe that should have been ConvertibleFrom<Int, Double>?

protocol ConvertibleFrom<Other> {
  init(_: Other) 
}

We have to decide what type to privilege as Self, because all of the members of the protocol are on that Self. What if you want both the initializer (on the "to" type) and the asOther operation (on the "from" type), how would you express that? Joe's straw man syntax

protocol Convertible(from: T, to: U) {
  // ... 
}

would make it clear that both the "from" nor the "to" type are on equal footing, with neither being functionally dependent on the other, which is the right semantics for this protocol.

Interpretation #3 is what Michael mentions in this post, where one wants to conform the same type to the same protocol in multiple different ways. There is still a primary Self type (so it's not the multi-Self case from interpretation #2), but it can bind the generic parameter in various different ways. For example, this might mean that String conforms to Collection (of Character) and Collect (of UnicodeScalar) and maybe others. Note that we could allow this with the same syntax we have today by lifting the restriction on overlapping conformances and allowing one to somehow specify that a particular member is only visible through the protocol conformance, e.g.,

extension String: Collection {
  typealias Collection.Element = Character // only visible through this conformance to Collection
  // ...
}

extension String: Collection {
  typealias Collection.Element = UnicodeScalar // only visible through this conformance to Collection
  // ...
}

There are lots of details here with ambiguity resolution and such, because when you have a primary Self type it still makes sense to say "String is a Collection " but that statement becomes ambiguous. Call sites will be able to sort through the ambiguity if you have other information, e.g.,

func find<C: Collection>(_ value: C.Element, in collection: C) -> C.Index? { ... }

let char: Character = a
find(char, in: "Hello, world")  // okay, only the String conformance where Element == Character matches

Indeed, you start to want to have a convenient way to say "String is a Collection where the element type is Character". The natural syntax for that is probably Collection<Character>, which is interpretation #1 that's being pitched.

Perhaps you have another interpretation of "generic protocols" in mind, or disagree with my analysis of interpretations #2 and #3, but I think both #2 and #3 are better expressed with a syntax that isn't of the form "protocol A<B>", whereas #1 (this pitch) is a highly-requested shorthand.

Doug

11 Likes