Improving the UI of generics

@Joe_Groff one question about the reverse generics. Right now opaque types would require Swift 5.1 runtime for opaque types that would cross module boundary. Can you imagine what would the requirement could be for reverse generics (can they potentially deploy back to Swift 5 runtime)?

A special syntax for shorthanding parametrization of protocols could be discussed:

// Type names in angle brackets must match associatedtypes in protocol body
protocol Collection<Element> {
    associatedtype Element
}

Or maybe implement some sort of 'non-shadowing typealias' feature:

typealias Collection<T> = Collection<.Element == T>
// 'Collection' and 'Collection<T>' can both be used, with 'Collection<T>' just 
// serving as a convenient shortcut that you may choose to use when you see fit.

(these are all strawman syntaxes, mind you)

Syntax sugaring not much unlike the ones above could always be an option in order to reduce complexity if the readability burden, like you suggest, turns out to be too big of a problem to ignore, all while still keeping Collection and Sequence protocols with more than one generic type argument, like today.

I'm a little slow and still trying to understand all of that initial (very helpful) post. Is this part above assuming that x is a var and might be changed in between these two lines? Otherwise I don't see how it could fail, so it seems like it could type check.

Am I understanding right that some is used here for two different purposes, both generics and reverse-generics? I think it should be noted that first two some are different than the third here.

What if someone wanted to return a generic type? Would you have to go back to using concatenate <T: Collection, U: Collection, V: Collection>(a: T, b: U) -> V syntax?

1 Like

Yes, if you want the caller to determine the return type you will need to do this. This is a good thing. Caller-specified return types that are unrelated to input parameters are a pretty advanced use case.

In your example there is no reasonable implementation. The caller is allowed to request any V: Collection. The implementation will not have any way to create that collection.

Ignoring that for a second, when you do want a caller-specified return type you don't need to abandon some for input parameters. Your example could be written as follows:

<V: Collection>(a: some Collection, b: some Collection) -> V

The good news is that under the improved UI, use of explicit type parameters will be much reduced, and often when they are necessary it will at least be possible to infer the necessary constraints based on usage in the signature.

3 Likes

Thanks everyone for the feedback! My apologies for getting back to this a little late.

As I posted in that acceptance thread, I don't think it makes sense to warp concrete accepted use cases for identifiers in service of theoretical future language directions like what I'm proposing. It's a problem for any new keyword that it may collide with existing uses as an identifier. For instance, some is already prominently used as one of the case names for Optional. It seems to me that any could still be contextually parsed as part of type grammar, since identifier(identifier) or identifier identifier is not otherwise valid type syntax, and types are limited in where they can appear inside expressions. If that isn't palatable, Any is already a reserved keyword, and Any P seems fine too.


There are definite advantages to C#'s approach. Because associated types for C# interfaces are represented as generic arguments, interface types always effectively bind all of those interface types, so they're only ever "existential" on Self. The .NET collections API is designed well around the tradeoffs this design provides. It's been personally annoying to myself that we haven't at least implemented Protocol<.AssocType == T> existentials in Swift yet, since as you laid out well, that would enable the more natural dynamic design for Sequence and Iterator you described.

Swift's design is aimed at enabling more a expressive type system to capture more interesting type-level relationships between values. The C# design would become more cumbersome if you tried to implement something like Swift's Collection hierarchy in it, since you'd need to define a type ICollection<Index, Element> and carry the index around with you everywhere. The type relationship between collections and indexes is what allows Swift's collections to approach "zero cost" in specialized code, since for instance, you know an Array is always indexed by Ints, and that a String is always indexed by valid code unit offsets represented by String.Index. Although you could express that relationship in C#, it would make ICollection not very useful as a dynamic interface type, since the Index generic argument is usually specific to a single collection family, so for instance ICollection<Int, T> would effectively be a type that can only hold Arrays. By using associated types, Swift allows you to express relationships between Collections using only the relevant associated types; you only need to refer to Index when indexing. With more flexible existential types, you'd also be able to refer to any Collection<.Element == T> to abstract over collections of a certain element type without confining yourself to a specific index. The goal of associated types is to allow for greater flexibility and expressivity, admittedly at the cost of some shorter-term awkwardness since we're missing so many key features still.


For immutable arguments, these are in fact isomorphic. If the limitations on existentials were removed, then both functions have effectively the same input domain. The differences become interesting if one of the arguments is inout:

// Takes some collection and modifies it, but doesn't change its type
func foo(a: inout some Collection)

// Takes some collection and modifies it, and may change its type
func foo(a: inout any Collection)

Similarly, for return types, like in your second example:

the two signatures suggest something different to the caller. If you return some Collection, you're saying you always return values of the same type. If you return any Collection, you could return different types between different calls.


This would be akin to implicitly declaring that P has an associated type, since the some Collection would be chosen by the conforming type. That might be nice sugar for this:

protocol P {
  associatedtype ReturnTypeOfFoo
  func foo() -> ReturnTypeOfFoo
}

Relatedly, it might also be interesting to explore shorthands for generic wrapper types. Instead of writing something like:

// If we get the call constraints discussed in the "static callables" proposal...
struct MapCollection<C: Collection, Element, Transform: (C.Element) -> Element> {
  var underlying: C
  var transform: Transform
}

It'd be cool if something like this were possible:

struct MapCollection<T, U, ...> {
  var underlying: some Collection where Element == T
  var transform: some (T) -> U
}

(I realize this conflicts with the use of some to declare opaque types on the individual properties themselves; it's a sketch of a possible idea.)


I think it'd be valuable to allow you to use generics syntax as sugar for associated types on protocols, so you could write:

protocol Collection<Element> {
  associatedtype Index
}

and that would be as if you'd written:

protocol Collection {
  associatedtype Element
  associatedtype Index
}
typealias Collection<Element> = Collection<.Element == Element>

I don't think we want generic arguments to completely replace associated types, for the reasons I alluded to above. For something like Collection, you frequently want to refer to its Element, but the Index is usually secondary. This is like the "embracing labels" idea you're proposing, in a sense.


Because of Foundation bridging, this is how we'd have to implement it, alas.


That's a reasonable idea!


Yeah, I'm not a fan of type(of:) myself either, though I do like the idea of being able to refer to types relative to values in generic signatures. To me, the ideal would be to refer to associated types as members of the values themselves, like:

func foo(a: some Collection, b: some Collection) -> some Collection
  where a.Element == b.Element, b.Element == return.Element

(though that still needs a hack to refer to the return value, alas).

Yeah. some Any and any Any aren't great, I agree. If we could do it all over again, maybe we should've had Any be called Value, and AnyObject just Object. I like the idea of using some and any also in extensions to help disambiguate the "extend conforming types" case from the "extend the existential type" case. That would also fit well with the future addition of generic extensions, since:

extension some P

would naturally expand to:

extension <T: P> T

Since the property's type is a "reverse generic", it's derived from the container's type, but doesn't require the container to itself be generic. The property's definition is what binds the property type, as in:

extension Int {
  var foo: some Any { return "foo" }
}

foo's opaque return type is bound to String.


I'm still working through the thread; I'll reply to more posts in a bit. Thanks again for all of your feedback!

15 Likes

Yeah, I think it's totally fine. I was reacting to the coincident timing: Within a few days, any was out as a keyword, then pitched again.

The runtime support I'm implementing is being done with an eye toward supporting generalized opaque types in the future, so if all goes well, there won't be further runtime deployment constraints on generalizing the feature (except maybe for debugger and tooling support).


You are correct. If the values involved are immutable, then it wouldn't formally be possible for the dynamic type of first to change from the element type of x. This ties in with the discussion of "opening" existential types a little further down. Right now, Swift's type system doesn't have any way to tie back into the dynamic types of values, so if x is any Collection then the best type we could use to describe the type of x.first! would be Any. For immutable values, we could implicitly open existential types, letting us say instead that first has the type type(of: x).Element or something like that. That would then let you use the value as an Element of x. This would however fundamentally only work with immutable values.

1 Like

First of all thanks @Joe_Groff for writing this up! It definitely clarifies some concepts around Swift type system that we keep trowing around on these forums but that are not easy to grasp. I'm pretty sure I'm still a little confused so don't mind these questions.

  • Would it make sense to keep these together with the manifesto in the repo? This information is quite useful and it would be nice to have it in a more accessible place (maybe even in official documentation or the book?)

Do these changes mean that any normal use of protocols would require the extra any keyword? I'm imagining all the DI uses of protocols and I'm not sure it's good marketing for Swift.

protocol NetworkManager {...}
class VM {
 init(networkManager: any NetworkManager) {...}
}

If that's the case, even if I like the direction of this, I'm a little hesitant. If is not the case it mens I need to read it again ^^'

I think on this line I agree with other comments in that it seems to extend all implementors of Hashable. Maybe is just because I don't fully get it or because I"m not used to it.

This is something that has been commented a bunch but I'm pretty sure that the majority of Swift devs wouldn't understand that easily. For my own sake, can somebody explain why that is the case? And I think it would be worth adding that to the original text.

Even with these concerns and not being an expert on the topic I'm really excited to see the future direction of the type system in Swift. I'm a huge fan of having this bigger overarching manifestos.

Thanks!

By a "normal" use of protocols, do you mean use of protocols as types? That's what I'm suggesting, yeah. If we look back at ObjC, this would be similar to how using a protocol as a type required writing id<Protocol>.

4 Likes

Yes that's what I meant. Thanks for the quick reply.

Looking back at Objective-C it doesn't look that bad ^^' I still think it will have a little of bad reception, specially from people coming from the mentioned languages with "interfaces", but if I prefer to think in the future :smiley:

Cheers

How would this relate to multi-parameter protocols? If we use “generic protocol” syntax in this way is there some other syntax that we might use for multi-parameter protocols? What might that look like? I would hate to see multi-parameter protocols be precluded just because people have a hard time with associated types mostly due to current limitations in the type system that should be lifted.

This is a fantastic idea! I missed it earlier. It would further reduce the cases where angle brackets are necessary.

Doesn’t the syntax of the previous quote make this unnecessary? With that sugar we can write it like this:

func foo(a: some Collection A, b: some Collection B) -> some Collection Result
  where A.Element == B.Element, B.Element == Result.Element

We can also require the input collections have the same type without having to use angle brackets:

func foo(a: some Collection C, b: C) -> some Collection Result
  where C.Element == Result.Element

This is nice! Would we still support extending the protocol directly for cases when you wanted to extend both the existential and the conforming types?

1 Like

I personally don't think that using the "generic protocols" for multi-parameter protocols is a good idea, since IMO the generic notation is the "good syntax", but multi-parameter protocols without a functional dependency are not the common case. Furthermore, the generics syntax implies an asymmetric relationship between Self and the arguments, which is appropriate for associated types, but completely artificial for multi-parameter protocols. I like the idea of a tuple-ish syntax like early versions of Rust had:

protocol (A, B): Protocol { }

func foo<A, B>() where (A, B): Protocol

or an application-like syntax similar to what's been proposed for Go 2:

protocol Protocol(A, B) { }

func foo<A, B>() where Protocol(A, B)

Since the set of protocols that need to be modeled this way is the minority, I think it's OK for it to have a somewhat more obscure syntax, if we ever support it at all.

4 Likes

This makes sense and the syntax above looks reasonable to me. I do hope we support this eventually. It would be nice to be able to model things like conversions, etc. Regardless, with other reasonable options for multi-parameter protocols available, using the generic syntax for associated types such as Element that would commonly be bound in existentials makes sense to me. This makes even more sense if it allows us to say any Collection<Int> (as so many people have requested) instead of any Collection<.Element == Int>.

1 Like

This is very good point. Could using <> after the type name be combined with existing generic parameter syntax (<> after the function name)?

func concatenate<T>(
    a: some Collection<Element == T>, 
    b: some Collection<Element == T>
) -> some Collection<Element == T> { }

Granted this example kind of breaks down if you need the two inputs to be the same collection type.

I also like this idea:

Could this be combined with the trailing <> syntax?

func concatenate(
    a: some Collection, 
    b: some Collection<Element == a.Element>
) -> some Collection<Element == a.Element> { }

Or perhaps where would be more readable.

func concatenate(
    a: some Collection, 
    b: some Collection where Element == a.Element
) -> some Collection where Element == a.Element { }

Perhaps this could be made more concise with an alias:

typealias CollectionOf<T> = Collection where Element == T
func concatenate(
    a: some Collection, 
    b: some CollectionOf<a.Element>
) -> some CollectionOf<a.Element> { }
1 Like

I would be really disappointed if we did that. Everyone is familiar with generic types and if we would introduce full support for type nesting it will be unavoidable to also introduce generic protocols. I don‘t buy the argument Joe said that generic protocols are the minority as they are not possible today there is nothing you can compare to. There are plenty of use cases I had where generic protocols would be the right choice and the alternative solution was not using PATs but a complete redesign to workaround these shortcomings. When Joe first pitched this Protocol<.Assoc == T> syntax I explicitly asked if it will interfere with the generic protocols and got as a reply that the leading dot is meant exactly for the use-case to refer to an associated type, which means if you had a generic protocol with associated type you could still write Protocol<Param, .Assoc == T> where Param is the generic type parameter of the generic protocol. That said I don‘t understand the resistance here nor do I appreciate the desire of trying to come up with some syntax for an alternate feature which eventually would block GPs.


I think as both Rust and Swift developer @regexident knows exactly the best practices on how to use PATs and GPs or even a mix of them that allows great expressiveness in the code base.

5 Likes

One more thing against any Collection<T> and in favor of any Collection<.Element == T> is that with the latter syntax we can express a super-type constraint as well any Collection<.Element: T> while the former syntax does not allow that.

4 Likes

I am also against an eventual generic-like syntax for primary associatedtypes: even though Swift doesn't have generic protocols and likely won't have them for a good while (if ever), the "concept" of them exists and even if not well known or understood it is fitting with the rest. Overriding that to make PATs nicer for the majority of use-cases but against more complex one I feel would be a bad move.

4 Likes

I like the idea and how it feels, but at the same time I have a reservation about it: having the type name after the constraint makes it feel like the name is attached to that, so in a way if this works I would feel that any Collection C should work as well, but it doesn't as C is really about some.
Maybe some<C> Collection? Though this basically goes down the road of having some<C: Collection> and any<Collection>, which works but I'm not sure that I like

What sorts of examples do you have? The most common examples I see from Rust are conversion-style protocols, a design choice Swift generally avoids in favor of promoting common currency types, and where the implied asymmetry of the generic syntax feels counterproductive to me. It isn't clear except by convention which end of the conversion should be Self versus being a generic parameter. Using a more application-style syntax could let us deploy argument labels to make the relationship between peer types in a conformance clearer.

The concept of multi-parameter type classes exists, yes, and Rust and C# choose to spell them as generic protocols, but that's not the only way to express them. In JVM languages, generic interfaces behave more like Swift associated types, so there's precedent for both behaviors.

2 Likes