Improving the UI of generics

[DevAndArtist (Adrian Meister)]

@Joe_Groff one question about the reverse generics. Right now opaque types would require Swift 5.1 runtime for opaque types that would cross module boundary. Can you imagine what would the requirement could be for reverse generics (can they potentially deploy back to Swift 5 runtime)?

[Luiz (Luiz Fernando)]

@adamkemp

func concatenate<T>(a: some Collection<.Element == T>, b: some Collection<.Element == T>)
  -> some Collection<.Element == T>

Is that really better than this, which could mean the same thing?

func concatenate<T>(a: Collection<T>, b: Collection<T>)
  -> some Collection<T>

A special syntax for shorthanding parametrization of protocols could be discussed:

// Type names in angle brackets must match associatedtypes in protocol body
protocol Collection<Element> {
    associatedtype Element
}

Or maybe implement some sort of 'non-shadowing typealias' feature:

typealias Collection<T> = Collection<.Element == T>
// 'Collection' and 'Collection<T>' can both be used, with 'Collection<T>' just 
// serving as a convenient shortcut that you may choose to use when you see fit.

(these are all strawman syntaxes, mind you)

Syntax sugaring not much unlike the ones above could always be an option in order to reduce complexity if the readability burden, like you suggest, turns out to be too big of a problem to ignore, all while still keeping Collection and Sequence protocols with more than one generic type argument, like today.

[robnik]

@Joe_Groff

// also not type-safe, since the existential Any might not match the existential Collection's Element type 
var first = x.first! 
var indexOfFirst = x.index(of: first) // error

I'm a little slow and still trying to understand all of that initial (very helpful) post. Is this part above assuming that x is a var and might be changed in between these two lines? Otherwise I don't see how it could fail, so it seems like it could type check.

[cukr]

@Joe_Groff

func concatenate(a: some Collection, b: some Collection) -> some Collection

Am I understanding right that some is used here for two different purposes, both generics and reverse-generics? I think it should be noted that first two some are different than the third here.

What if someone wanted to return a generic type? Would you have to go back to using concatenate <T: Collection, U: Collection, V: Collection>(a: T, b: U) -> V syntax?

[anandabits (Matthew Johnson)]

@cukr

What if someone wanted to return a generic type? Would you have to go back to using concatenate <T: Collection, U: Collection, V: Collection>(a: T, b: U) -> V syntax?

Yes, if you want the caller to determine the return type you will need to do this. This is a good thing. Caller-specified return types that are unrelated to input parameters are a pretty advanced use case.

In your example there is no reasonable implementation. The caller is allowed to request any V: Collection. The implementation will not have any way to create that collection.

Ignoring that for a second, when you do want a caller-specified return type you don't need to abandon some for input parameters. Your example could be written as follows:

<V: Collection>(a: some Collection, b: some Collection) -> V

The good news is that under the improved UI, use of explicit type parameters will be much reduced, and often when they are necessary it will at least be possible to infer the necessary constraints based on usage in the signature.

[Joe_Groff]

Thanks everyone for the feedback! My apologies for getting back to this a little late.

@krilnon

Joe_Groff:

var x: any Collection = [1, 2, 3]

I really like the disambiguation and syntactic clarity of saying any Collection . It also clearly follows the orthogonality of generics and existentials.

A bit of an aside: Didn't we just preclude any from being a future keyword in SE-0251?

The Core Team also considered that this use of any and all will likely prevent their future use as keywords.

As I posted in that acceptance thread, I don't think it makes sense to warp concrete accepted use cases for identifiers in service of theoretical future language directions like what I'm proposing. It's a problem for any new keyword that it may collide with existing uses as an identifier. For instance, some is already prominently used as one of the case names for Optional. It seems to me that any could still be contextually parsed as part of type grammar, since identifier(identifier) or identifier identifier is not otherwise valid type syntax, and types are limited in where they can appear inside expressions. If that isn't palatable, Any is already a reserved keyword, and Any P seems fine too.

---

@adamkemp

I've been trying to understand why working with generics in Swift feels so cumbersome to me compared to my experience with C#, and I think this hints at one key reason:

Joe_Groff:

This nonetheless still doesn't give you full access to the underlying collection's API, since the Index types are still not known to be the same; the type system would still, without further language support, refuse to allow code working with an existential CollectionOf<T> to index itself. Having an existential type constrain Index to a specific implementation would make the existential nearly useless due to the tight coupling between collection implementations and their indexes.

Consider a similar (but simpler) protocol: Sequence has two associated types: the Element and the Iterator . Compare this to the closest C# equivalent: IEnumerable<T> , as you can see, has one generic argument: the element type. This interface has one required method, which returns an IEnumerator<T> .

These two interfaces (Swift's Sequence and C#'s IEnumerable<T> ) both seem to be solving the same problem, but the C# approach is far easier to use. You can easily and intuitively write a method in C# that consumes an IEnumerable<T> and produces another IEnumerable<T> , or you can write a method that consumes an IEnumerable<T> and an IEnumerable<U> and produce either of those types or a new type based on T.

I'm still trying to wrap my head around why it's so much easier in C#, or in other words why can't it work the same way in Swift? For instance, why does Swift need a separate associatedtype in Sequence for the Iterator instead of using something like (using your new notation, though I'm not sure it's as intended): func makeIterator() -> any Iterator<.Element = T> (or, as I'd prefer it, func makeIterator() -> Iterator<T> )?

There are definite advantages to C#'s approach. Because associated types for C# interfaces are represented as generic arguments, interface types always effectively bind all of those interface types, so they're only ever "existential" on Self. The .NET collections API is designed well around the tradeoffs this design provides. It's been personally annoying to myself that we haven't at least implemented Protocol<.AssocType == T> existentials in Swift yet, since as you laid out well, that would enable the more natural dynamic design for Sequence and Iterator you described.

Swift's design is aimed at enabling more a expressive type system to capture more interesting type-level relationships between values. The C# design would become more cumbersome if you tried to implement something like Swift's Collection hierarchy in it, since you'd need to define a type ICollection<Index, Element> and carry the index around with you everywhere. The type relationship between collections and indexes is what allows Swift's collections to approach "zero cost" in specialized code, since for instance, you know an Array is always indexed by Ints, and that a String is always indexed by valid code unit offsets represented by String.Index. Although you could express that relationship in C#, it would make ICollection not very useful as a dynamic interface type, since the Index generic argument is usually specific to a single collection family, so for instance ICollection<Int, T> would effectively be a type that can only hold Arrays. By using associated types, Swift allows you to express relationships between Collections using only the relevant associated types; you only need to refer to Index when indexing. With more flexible existential types, you'd also be able to refer to any Collection<.Element == T> to abstract over collections of a certain element type without confining yourself to a specific index. The goal of associated types is to allow for greater flexibility and expressivity, admittedly at the cost of some shorter-term awkwardness since we're missing so many key features still.

---

@ashin

One question I'm trying to wrap my head around still: with this proposal, would there be a practical difference between these function signatures?

func foo(a: some Collection, b: some Collection)
func foo(a: any Collection, b: any Collection)

For immutable arguments, these are in fact isomorphic. If the limitations on existentials were removed, then both functions have effectively the same input domain. The differences become interesting if one of the arguments is inout:

// Takes some collection and modifies it, but doesn't change its type
func foo(a: inout some Collection)

// Takes some collection and modifies it, and may change its type
func foo(a: inout any Collection)

Similarly, for return types, like in your second example:

@ashin

Edit: another question from the code in the original post. Could this concatenate method

func concatenate(a: some Collection, b: some Collection) -> some Collection
  where type(of: a).Element == type(of: b).Element,
        type(of: return).Element == type(of: b).Element

also be written with existentials instead of generics, since there are no constraints on the concrete types of the parameters?

func concatenate(a: any Collection, b: any Collection) -> any Collection
  where type(of: a).Element == type(of: b).Element,
        type(of: return).Element == type(of: b).Element

the two signatures suggest something different to the caller. If you return some Collection, you're saying you always return values of the same type. If you return any Collection, you could return different types between different calls.

---

@plorenzi

Can reverse generics be declared in protocol methods? because in that case there is no specific type returned, so maybe it is not possible to compile it as generics. For example in:

protocol P {
    func foo() -> some Collection
}

This would be akin to implicitly declaring that P has an associated type, since the some Collection would be chosen by the conforming type. That might be nice sugar for this:

protocol P {
  associatedtype ReturnTypeOfFoo
  func foo() -> ReturnTypeOfFoo
}

Relatedly, it might also be interesting to explore shorthands for generic wrapper types. Instead of writing something like:

// If we get the call constraints discussed in the "static callables" proposal...
struct MapCollection<C: Collection, Element, Transform: (C.Element) -> Element> {
  var underlying: C
  var transform: Transform
}

It'd be cool if something like this were possible:

struct MapCollection<T, U, ...> {
  var underlying: some Collection where Element == T
  var transform: some (T) -> U
}

(I realize this conflicts with the use of some to declare opaque types on the individual properties themselves; it's a sketch of a possible idea.)

---

@Andrew

The added syntax and complexity might be a necessary requirement of progress, but I’m wondering if this manifesto could also be an opportunity for simplification.

I don’t like being critical of things without a suggested alternative, so here are some ideas:

Use generic syntax for protocols definitions instead of associatedtype. It’s becoming less clear when protocol is syntactically distinct from generics, let’s see if we can reduce protocol specific syntax.

I think it'd be valuable to allow you to use generics syntax as sugar for associated types on protocols, so you could write:

protocol Collection<Element> {
  associatedtype Index
}

and that would be as if you'd written:

protocol Collection {
  associatedtype Element
  associatedtype Index
}
typealias Collection<Element> = Collection<.Element == Element>

I don't think we want generic arguments to completely replace associated types, for the reasons I alluded to above. For something like Collection, you frequently want to refer to its Element, but the Index is usually secondary. This is like the "embracing labels" idea you're proposing, in a sense.

---

@dwaite

static func ==(a: any Hashable, b: any Hashable) -> Bool { return AnyHashable(a) == AnyHashable(b) }

Please don't let people think this is a valid implementation of Equatable

Because of Foundation bridging, this is how we'd have to implement it, alas.

---

@Jon_Hull

Would it work to just add the name after another space when needed or would that conflict with some other syntax?

func foo (_ c: some Collection C)  

That's a reasonable idea!

---

@DeFrenZ

• Using type(of:) suggests the dynamic type of something, while in this case we're talking about static type. I actually use it quite a bit in init s and never understand the rules around when it can be used as a statically evaluated utility and when as a runtime one
• Especially type(of: return) looks like an hack

Yeah, I'm not a fan of type(of:) myself either, though I do like the idea of being able to refer to types relative to values in generic signatures. To me, the ideal would be to refer to associated types as members of the values themselves, like:

func foo(a: some Collection, b: some Collection) -> some Collection
  where a.Element == b.Element, b.Element == return.Element

(though that still needs a hack to refer to the return value, alas).

@DeFrenZ

• func foo <T> (_ t: T) translates to func foo(_ t: some Any) or func foo(_ t: some) ?
• Would this allow extension any Any ?
• Should then current mixins like extension Collection { ... } be extension some Collection { ... } instead?

Yeah. some Any and any Any aren't great, I agree. If we could do it all over again, maybe we should've had Any be called Value, and AnyObject just Object. I like the idea of using some and any also in extensions to help disambiguate the "extend conforming types" case from the "extend the existential type" case. That would also fit well with the future addition of generic extensions, since:

extension some P

would naturally expand to:

extension <T: P> T

• Can properties be some without making the type itself generic? Sounds like it would be some kind of implicit genericness...

Since the property's type is a "reverse generic", it's derived from the container's type, but doesn't require the container to itself be generic. The property's definition is what binds the property type, as in:

extension Int {
  var foo: some Any { return "foo" }
}

foo's opaque return type is bound to String.

---

I'm still working through the thread; I'll reply to more posts in a bit. Thanks again for all of your feedback!

[krilnon (Kyle Murray)]

@Joe_Groff

As I posted in that acceptance thread, I don't think it makes sense to warp concrete accepted use cases for identifiers in service of theoretical future language directions like what I'm proposing. It's a problem for any new keyword that it may collide with existing uses as an identifier.

Yeah, I think it's totally fine. I was reacting to the coincident timing: Within a few days, any was out as a keyword, then pitched again.

[Joe_Groff]

@DevAndArtist

@Joe_Groff one question about the reverse generics. Right now opaque types would require Swift 5.1 runtime for opaque types that would cross module boundary. Can you imagine what would the requirement could be for reverse generics (can they potentially deploy back to Swift 5 runtime)?

The runtime support I'm implementing is being done with an eye toward supporting generalized opaque types in the future, so if all goes well, there won't be further runtime deployment constraints on generalizing the feature (except maybe for debugger and tooling support).

---

@robnik

// also not type-safe, since the existential Any might not match the existential Collection's Element type 

var first = x.first!
var indexOfFirst = x.index(of: first) // error

I'm a little slow and still trying to understand all of that initial (very helpful) post. Is this part above assuming that x is a var and might be changed in between these two lines? Otherwise I don't see how it could fail, so it seems like it could type check.

You are correct. If the values involved are immutable, then it wouldn't formally be possible for the dynamic type of first to change from the element type of x. This ties in with the discussion of "opening" existential types a little further down. Right now, Swift's type system doesn't have any way to tie back into the dynamic types of values, so if x is any Collection then the best type we could use to describe the type of x.first! would be Any. For immutable values, we could implicitly open existential types, letting us say instead that first has the type type(of: x).Element or something like that. That would then let you use the value as an Element of x. This would however fundamentally only work with immutable values.

[Alejandro_Martinez]

First of all thanks @Joe_Groff for writing this up! It definitely clarifies some concepts around Swift type system that we keep trowing around on these forums but that are not easy to grasp. I'm pretty sure I'm still a little confused so don't mind these questions.

• Would it make sense to keep these together with the manifesto in the repo? This information is quite useful and it would be nice to have it in a more accessible place (maybe even in official documentation or the book?)

Do these changes mean that any normal use of protocols would require the extra any keyword? I'm imagining all the DI uses of protocols and I'm not sure it's good marketing for Swift.

protocol NetworkManager {...}
class VM {
 init(networkManager: any NetworkManager) {...}
}

If that's the case, even if I like the direction of this, I'm a little hesitant. If is not the case it mens I need to read it again ^^'

@Joe_Groff

extension any Hashable: Hashable {

I think on this line I agree with other comments in that it seems to extend all implementors of Hashable. Maybe is just because I don't fully get it or because I"m not used to it.

@koher

It also helps to notify people who are familiar with languages like Java that protocols in Swift are different from interfaces in those languages when they learn Swift.

This is something that has been commented a bunch but I'm pretty sure that the majority of Swift devs wouldn't understand that easily. For my own sake, can somebody explain why that is the case? And I think it would be worth adding that to the original text.

Even with these concerns and not being an expert on the topic I'm really excited to see the future direction of the type system in Swift. I'm a huge fan of having this bigger overarching manifestos.

Thanks!

[Joe_Groff]

@Alejandro_Martinez

Do these changes mean that any normal use of protocols would require the extra any keyword? I'm imagining all the DI uses of protocols and I'm not sure it's good marketing for Swift.

protocol NetworkManager {...}
class VM {
 init(networkManager: any NetworkManager) {...}
}

If that's the case, even if I like the direction of this, I'm a little hesitant. If is not the case it mens I need to read it again ^^'

By a "normal" use of protocols, do you mean use of protocols as types? That's what I'm suggesting, yeah. If we look back at ObjC, this would be similar to how using a protocol as a type required writing id<Protocol>.

[Alejandro_Martinez]

@Joe_Groff

By a "normal" use of protocols, do you mean use of protocols as types? That's what I'm suggesting, yeah. If we look back at ObjC, this would be similar to how using a protocol as a type required writing id<Protocol> .

Yes that's what I meant. Thanks for the quick reply.

Looking back at Objective-C it doesn't look that bad ^^' I still think it will have a little of bad reception, specially from people coming from the mentioned languages with "interfaces", but if I prefer to think in the future

Cheers

[anandabits (Matthew Johnson)]

@Joe_Groff

I think it'd be valuable to allow you to use generics syntax as sugar for associated types on protocols, so you could write:

protocol Collection<Element> {
  associatedtype Index
}

How would this relate to multi-parameter protocols? If we use “generic protocol” syntax in this way is there some other syntax that we might use for multi-parameter protocols? What might that look like? I would hate to see multi-parameter protocols be precluded just because people have a hard time with associated types mostly due to current limitations in the type system that should be lifted.

@Joe_Groff

Would it work to just add the name after another space when needed or would that conflict with some other syntax?

func foo (_ c: some Collection C)  

That's a reasonable idea!

This is a fantastic idea! I missed it earlier. It would further reduce the cases where angle brackets are necessary.

@Joe_Groff

Yeah, I'm not a fan of type(of:) myself either, though I do like the idea of being able to refer to types relative to values in generic signatures. To me, the ideal would be to refer to associated types as members of the values themselves, like:

func foo(a: some Collection, b: some Collection) -> some Collection
  where a.Element == b.Element, b.Element == return.Element

(though that still needs a hack to refer to the return value, alas).

Doesn’t the syntax of the previous quote make this unnecessary? With that sugar we can write it like this:

func foo(a: some Collection A, b: some Collection B) -> some Collection Result
  where A.Element == B.Element, B.Element == Result.Element

We can also require the input collections have the same type without having to use angle brackets:

func foo(a: some Collection C, b: C) -> some Collection Result
  where C.Element == Result.Element

@Joe_Groff

I like the idea of using some and any also in extensions to help disambiguate the "extend conforming types" case from the "extend the existential type" case. That would also fit well with the future addition of generic extensions, since:

extension some P

would naturally expand to:

extension <T: P> T

This is nice! Would we still support extending the protocol directly for cases when you wanted to extend both the existential and the conforming types?

[Joe_Groff]

@anandabits

I think it'd be valuable to allow you to use generics syntax as sugar for associated types on protocols, so you could write:

protocol Collection<Element> {

associatedtype Index
}

How would this relate to multi-parameter protocols? If we use “generic protocol” syntax in this way is there some other syntax that we might use for multi-parameter protocols? What might that look like? I would hate to see multi-parameter protocols be precluded just because people have a hard time with associated types mostly due to current limitations in the type system that should be lifted.

I personally don't think that using the "generic protocols" for multi-parameter protocols is a good idea, since IMO the generic notation is the "good syntax", but multi-parameter protocols without a functional dependency are not the common case. Furthermore, the generics syntax implies an asymmetric relationship between Self and the arguments, which is appropriate for associated types, but completely artificial for multi-parameter protocols. I like the idea of a tuple-ish syntax like early versions of Rust had:

protocol (A, B): Protocol { }

func foo<A, B>() where (A, B): Protocol

or an application-like syntax similar to what's been proposed for Go 2:

protocol Protocol(A, B) { }

func foo<A, B>() where Protocol(A, B)

Since the set of protocols that need to be modeled this way is the minority, I think it's OK for it to have a somewhat more obscure syntax, if we ever support it at all.

[anandabits (Matthew Johnson)]

@Joe_Groff

Since the set of protocols that need to be modeled this way is the minority, I think it's OK for it to have a somewhat more obscure syntax, if we ever support it at all.

This makes sense and the syntax above looks reasonable to me. I do hope we support this eventually. It would be nice to be able to model things like conversions, etc. Regardless, with other reasonable options for multi-parameter protocols available, using the generic syntax for associated types such as Element that would commonly be bound in existentials makes sense to me. This makes even more sense if it allows us to say any Collection<Int> (as so many people have requested) instead of any Collection<.Element == Int>.

[tanner0101 (Tanner)]

@Joe_Groff

func concatenate<A: Collection, B: Collection>(a: A, b: B) -> <C: Collection> C
  where A.Element == B.Element, B.Element == C.Element
{
  return ConcatenatedCollection(a, b)
}

At this point, the generalized declaration is no longer clearly visually related to its original concrete form.

This is very good point. Could using <> after the type name be combined with existing generic parameter syntax (<> after the function name)?

func concatenate<T>(
    a: some Collection<Element == T>, 
    b: some Collection<Element == T>
) -> some Collection<Element == T> { }

Granted this example kind of breaks down if you need the two inputs to be the same collection type.

I also like this idea:

@Joe_Groff

func foo(a: some Collection, b: some Collection) -> some Collection
  where a.Element == b.Element, b.Element == return.Element

Could this be combined with the trailing <> syntax?

func concatenate(
    a: some Collection, 
    b: some Collection<Element == a.Element>
) -> some Collection<Element == a.Element> { }

Or perhaps where would be more readable.

func concatenate(
    a: some Collection, 
    b: some Collection where Element == a.Element
) -> some Collection where Element == a.Element { }

Perhaps this could be made more concise with an alias:

typealias CollectionOf<T> = Collection where Element == T
func concatenate(
    a: some Collection, 
    b: some CollectionOf<a.Element>
) -> some CollectionOf<a.Element> { }

[DevAndArtist (Adrian Meister)]

I would be really disappointed if we did that. Everyone is familiar with generic types and if we would introduce full support for type nesting it will be unavoidable to also introduce generic protocols. I don‘t buy the argument Joe said that generic protocols are the minority as they are not possible today there is nothing you can compare to. There are plenty of use cases I had where generic protocols would be the right choice and the alternative solution was not using PATs but a complete redesign to workaround these shortcomings. When Joe first pitched this Protocol<.Assoc == T> syntax I explicitly asked if it will interfere with the generic protocols and got as a reply that the leading dot is meant exactly for the use-case to refer to an associated type, which means if you had a generic protocol with associated type you could still write Protocol<Param, .Assoc == T> where Param is the generic type parameter of the generic protocol. That said I don‘t understand the resistance here nor do I appreciate the desire of trying to come up with some syntax for an alternate feature which eventually would block GPs.

---

I think as both Rust and Swift developer @regexident knows exactly the best practices on how to use PATs and GPs or even a mix of them that allows great expressiveness in the code base.

[DevAndArtist (Adrian Meister)]

One more thing against any Collection<T> and in favor of any Collection<.Element == T> is that with the latter syntax we can express a super-type constraint as well any Collection<.Element: T> while the former syntax does not allow that.

[DeFrenZ (Davide De Franceschi)]

I am also against an eventual generic-like syntax for primary associatedtypes: even though Swift doesn't have generic protocols and likely won't have them for a good while (if ever), the "concept" of them exists and even if not well known or understood it is fitting with the rest. Overriding that to make PATs nicer for the majority of use-cases but against more complex one I feel would be a bad move.

[DeFrenZ (Davide De Franceschi)]

@Jon_Hull

Would it work to just add the name after another space when needed or would that conflict with some other syntax?

func foo (_ c: some Collection C)  

I like the idea and how it feels, but at the same time I have a reservation about it: having the type name after the constraint makes it feel like the name is attached to that, so in a way if this works I would feel that any Collection C should work as well, but it doesn't as C is really about some.
Maybe some<C> Collection? Though this basically goes down the road of having some<C: Collection> and any<Collection>, which works but I'm not sure that I like

[Joe_Groff]

@DevAndArtist

I would be really disappointed if we did that. Everyone is familiar with generic types and if we would introduce full support for type nesting it will be unavoidable to also introduce generic protocols. I don‘t buy the argument Joe said that generic protocols are the minority as they are not possible today there is nothing you can compare to. There are plenty of use cases I had where generic protocols would be the right choice and the alternative solution was not using PATs but a complete redesign to workaround these shortcomings. When Joe first pitched this Protocol<.Assoc == T> syntax I explicitly asked if it will interfere with the generic protocols and got as a reply that the leading dot is meant exactly for the use-case to refer to an associated type, which means if you had a generic protocol with associated type you could still write Protocol<Param, .Assoc == T> where Param is the generic type parameter of the generic protocol. That said I don‘t understand the resistance here nor do I appreciate the desire of trying to come up with some syntax for an alternate feature which eventually would block GPs.

What sorts of examples do you have? The most common examples I see from Rust are conversion-style protocols, a design choice Swift generally avoids in favor of promoting common currency types, and where the implied asymmetry of the generic syntax feels counterproductive to me. It isn't clear except by convention which end of the conversion should be Self versus being a generic parameter. Using a more application-style syntax could let us deploy argument labels to make the relationship between peer types in a conformance clearer.

@DeFrenZ

I am also against an eventual generic-like syntax for primary associatedtype s: even though Swift doesn't have generic protocols and likely won't have them for a good while (if ever), the "concept" of them exists and even if not well known or understood it is fitting with the rest. Overriding that to make PATs nicer for the majority of use-cases but against more complex one I feel would be a bad move.

The concept of multi-parameter type classes exists, yes, and Rust and C# choose to spell them as generic protocols, but that's not the only way to express them. In JVM languages, generic interfaces behave more like Swift associated types, so there's precedent for both behaviors.