[Pitch 2] Light-weight same-type requirement syntax

Or, better, we accept this shorthand spelling as-is also.

I like the shorthand as proposed, even though I know it is overloading the syntax with generics it's compelling to me. Collection<String> feels very natural if I just want to map a collection of strings, and I think there is sufficient leeway for evolution like Collection<String, .Index == Foo> or whatever down the road.

-1, from me, mostly because of prioritization. As @Karl wrote earlier, this proposal tries to address both expressiveness and light-weight syntax for common cases.

I think we should address expressiveness first, and return to discussion of light-weight syntax after gaining practical experience with constrained protocols in opaque types, existentials and typealiases in production.

If we can express constrained protocols in typealiases, then we may not need lightweight syntax at all:

/// SequenceOf is a constraint
/// Constraints are generalization of protocols
/// Constraints are reusable pieces of where clause
/// Constraints can be generic
typealias SequenceOf<T> = Sequence where Element == T
func concatenate<S : SequenceOf<String>>(_ lhs: S, _ rhs: S) -> S {
   ...
}

10 Likes

Is this not a concern with every new hypothetical language feature though?

1 Like

In this case it wouldn't be as bad if you could retroactively add the primary associated type to the protocol; then only that extension part would go between the #if.

Yes, users either need to wait to adopt features or add conditionals to adopt them while still supporting other version. For instance, I needed to add some swift(>=5.5) blocks to Alamofire to support the new static member capabilities. However, this case, as Karl pointed out, is especially egregious, as, due to Swift's limited #if syntax, it requires duplicating the entire protocol definition to just to support some small bit of new syntax. That's basically the worst case scenario. Community adoption of the feature will be very low until they reasonably require the right version of Swift, which usually occurs once Apple stops accepting App Store submission with older versions of Xcode, and sometimes not even then. So any feature which can't be easily adopted with swift blocks will necessarily lag in adoption, which is bad for the language.

Of course, there are a lot of things Apple could do to help this case that aren't related to the language it self, but...

5 Likes

This feature would need a detailed pitch outlining the precise semantics. Typealiases can appear in many places; if we're introducing a new kind of typealias we can to spell out where they're allowed and where they're forbidden, and what the behavior is in each case.

Another consideration is that a hypothetical typealias for where clauses would also not be as ergonomic as the concise syntax I'm promising in many instances because you'd have to name the alias.

The syntax is not meant to replace where clauses entirely. You can still write a where clause.

I think we need to articulate what we mean by generic protocols. A protocol type constructor instantiated on a set of concrete types doesn't seem like what you want; I think rather folks are asking for multi-parameter type classes. Today a protocol conceptually has a single 'Self' generic parameter; a multi-parameter type class would have multiple 'Self' types, none of them more privileged over another; where any conformance stated in a requirement or extension to such a protocol involves multiple 'Self' types. I'm not sure Self1 : Proto<Self2> syntax is the one that you want here.

2 Likes

This syntax is ambiguous because you just defined a protocol where the underlying type is the existential type any Sequence. The parameter T that you instantiate the type constructor with doesn't appear in the underlying type, so it doesn't play a role here.

However note that you can already express something like this via requirement inference in parameter position, just slightly awkwardly:

typealias SequenceOf<S, E> = S where S : Sequence, S.Element == E

func contains<S, E>(_ seq: SequenceOf<S, E>, _ elt: E) -> Bool {}

I'm hesitant to draw this kind of inference. If folks feel that constraints on opaque result types are desirable but the current syntax is bad, they can state that and suggest an alternate approach, such as named opaque result types that one can refer to from the where clause, like this:

func returnSequenceOfString() -> <S> S where S : Sequence, S.Element == Int

The underlying mechanism to define requirements on opaque result types is already there, just not officially. In fact there is an experimental flag to enable named opaque result types (but I wouldn't rely on it for anything real of course).

Sendable has the same issue, where we'd like it to act more like an effect that's carried through from the arguments to the opaque result type. I'd rather us tackle this issue with opaque result types directly, since it's independent of the sugar here.

Doug

2 Likes

I wrote that bit of the Generics Manifesto many years ago, and that last part has held constant for many years since then: the vast majority of requests we get for "parameterized protocols" or "generic protocols" specifically ask to be able to write Collection<String> to get a collection of strings, which aligns with this pitch. Let's call that interpretation #1.

There are other interpretations we could give to this syntax as well, so let's consider them.

Interpretation #2 is multi-Self protocols, as Joe points out. The canonical example is expressing conversions, e.g., ConvertibleTo<Double, Int>:

protocol ConvertibleTo<Other> {
  func asOther() -> Other   // "Self" is the type we convert from, "Other" is the type we're converting to
}

Except in Swift we use converting initializers, so maybe that should have been ConvertibleFrom<Int, Double>?

protocol ConvertibleFrom<Other> {
  init(_: Other) 
}

We have to decide what type to privilege as Self, because all of the members of the protocol are on that Self. What if you want both the initializer (on the "to" type) and the asOther operation (on the "from" type), how would you express that? Joe's straw man syntax

protocol Convertible(from: T, to: U) {
  // ... 
}

would make it clear that both the "from" nor the "to" type are on equal footing, with neither being functionally dependent on the other, which is the right semantics for this protocol.

Interpretation #3 is what Michael mentions in this post, where one wants to conform the same type to the same protocol in multiple different ways. There is still a primary Self type (so it's not the multi-Self case from interpretation #2), but it can bind the generic parameter in various different ways. For example, this might mean that String conforms to Collection (of Character) and Collect (of UnicodeScalar) and maybe others. Note that we could allow this with the same syntax we have today by lifting the restriction on overlapping conformances and allowing one to somehow specify that a particular member is only visible through the protocol conformance, e.g.,

extension String: Collection {
  typealias Collection.Element = Character // only visible through this conformance to Collection
  // ...
}

extension String: Collection {
  typealias Collection.Element = UnicodeScalar // only visible through this conformance to Collection
  // ...
}

There are lots of details here with ambiguity resolution and such, because when you have a primary Self type it still makes sense to say "String is a Collection " but that statement becomes ambiguous. Call sites will be able to sort through the ambiguity if you have other information, e.g.,

func find<C: Collection>(_ value: C.Element, in collection: C) -> C.Index? { ... }

let char: Character = a
find(char, in: "Hello, world")  // okay, only the String conformance where Element == Character matches

Indeed, you start to want to have a convenient way to say "String is a Collection where the element type is Character". The natural syntax for that is probably Collection<Character>, which is interpretation #1 that's being pitched.

Perhaps you have another interpretation of "generic protocols" in mind, or disagree with my analysis of interpretations #2 and #3, but I think both #2 and #3 are better expressed with a syntax that isn't of the form "protocol A<B>", whereas #1 (this pitch) is a highly-requested shorthand.

Doug

11 Likes

Improving the UI of generics has huge benefits for new and experienced developers alike. So +1 — I really like the direction of this pitch and think the scope is about right.

I appreciate the comments regarding the spelling with < > but I prefer this to all alternatives I have seen thus far. For better or worse, there is precedence in the language where conflation of syntax occurs e.g. inheritance and conformance.

1 Like

With all my respect to you Slava and the others involved in the development of Swift. This just shows that the current pitch is a workaround + sugar code to provide the user the ability to write some Collection<Whatever> (probably even in return positions) while not actually allowing nor tackling (A), which is kinda the core of the improvements that are forcefully pushed on us. I remember the days where everyone was screaming a huge "no" to every possible sugar code proposal. Again, don't take any offense from my message, but this proposal does not truly add any new feature except a workaround one via short sugar code just to avoid (A), regardless on how cumbersome it would look like for opaque result types or at parameter level.

So how exactly am I supposed to use the where clause to write these?

This syntax is literally forced on me, as I don't see any other alternative that I can use over which this proposal is trying to improve. So can we have the feature for (A) before we continue talking about how to apply sugar over it?

Other points outlined in the proposal do not actually add anything new to the language except that they shorten some code we have to type. In other words, if you'd exclude the opaque result type thing from this proposal, it would really not add anything new except a bit of sugar and block certain design space of the language. :disappointed:

4 Likes

Syntax sugar has value if it makes code easier to read and write. An example of sugar is the inheritance clause of a generic parameter. There is really no reason for the language to allow you to write:

func reverse<S : Sequence>(_: S)

When a fully general where clause can express the requirement also:

func reverse<S>(_: S) where S : Sequence

Arguably though, the inheritance clause syntax is preferable for the restricted set of requirements it can express, as seen by its common usage in real-world Swift code. Similarly, the syntax proposed in this pitch also shorthand that makes code easier to read and write without really giving you anything new.

Similarly with associated types in protocols:

protocol Sequence {
  associatedtype Iterator : IteratorProtocol
}

is exactly equivalent to

protocol Sequence {
  associatedtype Iterator where Iterator : IteratorProtocol
}

Even where clauses on associated types are just sugar for where clauses on protocols; there is really no inherent reason to write

protocol Sequence {
  associatedtype Iterator : IteratorProtocol
  associatedtype Element where Element == Iterator.Element
}

when you can say

protocol Sequence where Iterator : IteratorProtocol, Element == Iterator.Element {
  associatedtype Iterator
  associatedtype Element
}

However attaching a where clause (or inheritance clause) to the associated type makes the protocol definition clearer; even more so in protocols with many associated types, such as Collection.

In the case of opaque result types, the sugar does give you something new, but even if we had the ability to name opaque result types and reference them from a general where clause, I think this pitch would still have value for the same reason -- it makes code easier to read and write by allowing you to avoid the where clause syntax except in more complex declarations where it is actually necessary.

I would suggest trying out the feature implemented in this proposal to see if it can clean up some of your own code, particularly when used with opaque parameter types.

Taken to the extreme, this philosophy would rule out most if not all of the changes introduced to the language since Swift 1.0. Everything is just sugar for lambda calculus once you have closures, function application and bindings.

1 Like

Don't get me wrong, writing less code is always a good thing, but this argument hides the main issue here. This proposal is forcing a syntax on us for a feature which is not available nor will be possible with any alternative syntax after this proposal ships. You cannot call this part of the proposal as sugar code, because it isn't. In reality it sneaks in a new feature, takes away a design space of the language for future development in a potentially different direction and claims to be some sugar code. Sorry, but sugar code over what? Take away the opaque result type from the proposal for a brief moment, that's the only sugar this proposal actually adds, because it's impossible to write named or anonymous opaque result types with a where clause even after the proposal ships.

I clearly can understand that there is a lot of appetite to not to write named opaque result types, especially in a framework like SwiftUI which extensively makes use of it. However this is a totally different and NEW feature. It's not sugar code anymore. Let me write a single named and anonymous opaque result type with a plain where clause first, then we can talk about how to actually apply sugar over it.

3 Likes

There's really no inherent reason for multi-parameter typeclasses to look like "generic protocols" with angle brackets after a protocol name. I still haven't seen anyone explain why instantiating protocol types with concrete arguments on the right hand side of a conformance requirement is a desirable feature, or what semantics it would have.

If you're looking for arbitrary where clauses on named opaque result types, please feel free to write up a pitch; the feature is 80% implemented behind an experimental flag on the main branch already.

4 Likes

I'm sorry I'm not a compiler developer, I wouldn't be able to finish even 1% of the remaining 20%. I find it unpleasant to be rolled over with a "if you want something else, do it / the rest yourself" counterargument. Have a great rest of your day.

6 Likes

This argument feels similarly structured to the argument that OOP-style methods shouldn’t exist because they awkwardly privilege one type in the defintion of equal(to:), and languages should use multimethods instead. I don’t find that form of argument to be very strong.

How did we get from a distinct interpretation of multiple-conformance back to a single conformance with an associated type constraint? Are you intending to imply that interpretation #3 is in fact equivalent to interpretation #1?

3 Likes

The "forcing" and the "sneaks" and the "takes away" come across as frantic and antagonistic. They are neither the tone we would like to set on these forums, nor are they helping your argument.

Fundamentally, your complaint is that this proposal is not syntactic sugar because, for opaque result types, it expresses something new: the ability to provide constraints on the associated types of the opaque result type.

As far as I can tell, you aren't disagreeing with that new feature, i.e., you agree that it is useful to be able to express a result type that is "some Sequence where the Element type is String". Your disagreement seems to come in two parts:

  1. You don't like taking the syntax Sequence<String> for this purpose, because it prevents us from using that syntax for something else in the future.
  2. You want the ability to express a complete where clause for the opaque type, rather than this restricted form.

Regarding (1), I don't actually think we want to use this syntax for the other things that "generic protocols" could mean. Here's my attempt at enumerating those things and why they should be spelled differently. It's not enough to say that generic protocols might need this syntax later; you actually need to make a strong case that this specific syntax is the best syntax for that future feature.

Regarding (2), the full "reverse generics" feature has an explicit type parameter list on the right-hand side of the ->. For example, let's write an "unzip" of sorts:

func unzip<C: Collection, T, U>(_ collection: C) -> <R1: Collection, R2: Collection> (R1, R2)
  where C.Element == (T, U), R1.Element == T, R2.Element == U { ... }

In other words, pass in a collection whose element type is (T, U) and get two collections back, one with the T's and one with the U's. With this proposal (and SE-0341), this can be expressed as:

func unzip<T, U>(_ collection: Collection<(T, U)>) -> (some Collection<T>, some Collection<U>)

That is so much clearer. The reverse-generics formulation isn't just more cluttered, it's forcing you to actively reason about both generic parameter lists and detangle the where clause to understand which bits affect the generic parameters left of the -> and which affect generic parameters to the right of the ->.

Reverse generics are a good conceptual underpinning for opaque result types that precisely matches the implementation model. Indeed, they are implemented in the compiler behind an experimental flag so we could test out all of the complicated combinations and internally desugar this pitch to that implementation. However, it is not at all clear to me that we ever want to surface the full reverse-generics models to users: you have to go very deep into the theory for the reverse-generics desugaring of this pitch to make more sense than other more-accessible ways of understanding opaque result types. This pitch covers the most common cases in a manner that we can teach.

If we did eventually get some other way to do more arbitrary where clauses, e.g., this suggestion:

then that would likely cover the expressivity gap. But I would say that this typealias solution by itself is not good enough to replace this pitch. Would we create Of-suffixed typealias versions of all of the collection protocols in the standard library? C++ did this with their type traits, from original class templates (is_same), to value forms (is_same_v) and finally concept forms (same_as), and the result is an awful mess, bloating the API with 3 names for each idea. We should not knowingly go down the same path.

If supporting an arbitrary where clause is important to be able to express in the language, then that feature needs supporting examples. And if the argument is that the need for an arbitrary where clause is so great that we should block progress on this particular pitch... then it needs to demonstrate that this pitch is going in the wrong direction, rather than just that this pitch isn't going far enough.

Doug

9 Likes

In Swift, we write equal(to:) as == in part because we don't want to privilege one particular type, so I'm not sure where your argument leads.

We didn't. If you allow multiple conformances, you're going to want a convenient way to talk about a specific one of those conformances, which is what this pitch does.

Doug