[Discussion] Easing the learning curve for introducing generic parameters

An interesting point re contextual static requirements. Note that the counterpart is a return value of type specified by the caller, which it is already possible to spell:

func f<T: Collection>() -> T { … }

(Typically we don’t make callers write as Foo and instead have them pass in the desired return type as an argument, but it is supported.)

The use case presented above doesn’t argue against the overall point that the position of the function arrow is sufficient to denote who controls the choosing of the type. As noted earlier, generalizing the other way would allow us to write opaque return types such as:

func f() -> <T: Collection> where T.Element: Codable { … }

Both callee-chosen argument types and caller-chosen return types have much more narrow use cases than the other way around (that is, plain old generics and opaque types). I think they would be adequately served by this long-form notation, which we already have for caller-chosen return types and can be extended in the manner shown above to callee-chosen argument types if sufficient use cases arise.

With the intuitive reading of some that I’ve outlined in an earlier post, I’d still expect that to mean a generic constraint on the left side of a function arrow and an opaque type on the right side. To me, it’s no different than how covariance and contravariance work with function argument and return types.

1 Like

…which, as I’ve argued, is amply denoted by the function arrow, in the same way that covariance versus contravariance are, not to mention who chooses the value.

1 Like

—Not to beat a dead horse, but I wonder if I’m wrong in this:

If the pitched syntax generalizing some to generic constraints is adopted, then who chooses the conforming type here:

protocol P { }
func f(_: some P) -> some P { }

…exactly parallels who chooses the subtype here:

class C { }
func f(_: C) -> C { }
6 Likes

+1 for me on type parameter inference via some – it feels like a natural evolution to the generics system, and a path towards what I see as the holy grail of Swift generics UI:

-> some AsyncSequence<Int>

This would mean getting rid of many of the the Any... type erasers such as AnyPublisher or swift-parsing's AnyParser which were created to preserve abstraction across API / module boundaries and for general developer experience.

For me, unlike visibly for many others, having a some P as a parameter and a some P as a return value being two completely different types feels actually very normal and instinctual for me – I've always seen functions as taking control of my "type context" and doing whatever they want with it. @xwu 's code snippet above is very salient in this regard.

I feel less sure however about some generics for stored properties of a type. This may be misguided, but a type's generics have for me always intrinsically part of their identity, much less than for a function's generics. That may be because a (algebraic) data type's type is a product of its properties' types, which give them for me a much more important place in the type's definition.

Some details for those who are interested Let's say we have a struct called Pair:
struct Pair {
    let first: A
    let second: B
}
In type theory, the type of a struct is the product of the types of its properties, aka:
 Pair<A, B> = A * B 
That's why to me a type's generics feel inherent to the type itself, and deserve a place in the types head declaration next to its name.

Warning - even more type theory...

Technically, a function's input and output types are as intrinsic to the function as generics are to a struct (which could bring down my whole argument about some generics in the input and output being different types).

Indeed, the "type" of a function in type theory is the output type to the power of its input type:

 (A) -> B = B^A 

Practically though, I mentally don't tie a function to its generics as much as I do for a struct.

Meanwhile, a (pure) function feels more like a computation on an input, spitting back out some output. Its generics matter mentally less to me. That's why some generics on parameter declarations feel very natural to me, but some generics on stored property declarations much less so.

So glad you are working on this area! My own wish would be to manifest the information flow and the workings of the type-checker with physical metaphors. That's more a question than an answer, though. If the program is a landscape, type-checking takes us on a journey through it. There can be a fork in the road, and we'll need bird's-eye views too, as well as chunking mechanisms, and ways of seeing the same thing from multiple perspectives at once.

(Much of this is not new to anyone, so apologies for stating the obvious.) The norm is code that does not type check, so the type system must encompass that, somehow, maybe even with probabalistic guesses.

Anyway, I would try to draw the picture in your head when you have to deal with some code that doesn't check out. Then externalize that picture onto the screen so that you can dive deeper into any part, and manipulate any thing you can see. Maybe even change focus so that you can change what is most salient at any given time.

One thing that would be great is if there wasn't such a brick wall between the existential and generic worlds. Of course we'd rather people just write the correct thing, but we could do a lot more to make common situations easier.

  1. A function like

    func doSomething(items: Collection)
    

    Should just be a generic function. There is literally no reason for value-level abstractions for a function's input arguments (their underlying types are fixed the moment they are passed in to a function call).

    That means it should be possible to access the associated types from items, including getting indexes and storing them in local variables, etc - just like I would in a generic function.

    It should also be possible to call generic functions constrained to <T: Collection> from within doSomething, passing our existential items as a parameter.

    func genericFn<T: Collection>(_: T) { ... }
    
    func doSomething(items: Collection) {
      genericFn(items) // Currently an error, but doesn't need to be.
    }
    
  2. Allow opening existentials

    When you do hit the bridge between value and type-level abstractions, it's very bumpy. Whilst we have a way to erase concrete/generic types as existentials (the compiler will even do it implicitly), going the other way isn't even possible.

    We've talked about this for years - the idea of opening an existential as a local generic parameter, so there will actually be something you can do when faced with a Collection does not conform to Collection error.

    If the compiler did it implicitly, as it does for erasing, point 1 would fall out naturally:

    func doSomething(items: Collection) {
      openExistential(items) { <T: Collection>(bound_items: T) in
        genericFn(bound_items)
      }
    }
    

inout arguments are more delicate, because an argument which is an inout Collection may indeed be reassigned with an underlying value of a different type. It's easy enough to spot - usage requires creating a variable explicitly typed to be an existential:

func doSomething(items: inout Collection) { ... }

var someCollection = "hello"
doSomething(items: &someCollection) // Error

var someCollection: Collection = "hello"
doSomething(items: &someCollection) // Now it's okay.

It might be conceivable to change this in a new language mode, so inout Collection also works like an anonymous generic type. With implicit existential opening, both examples above would work, but doSomething would fail to compile in that new mode if it actually changes the underlying type of items. In that case, it would have to write its parameter inout any Collection (or something), which I think is fair. It's quite rare to see code which uses inout existentials at all.

This would basically remove value-level abstractions for function parameters, save for inout parameters where it is actually meaningful. The difference between value- and type- level abstraction isn't actually relevant in these contexts, so why not just remove that difference? :grinning_face_with_smiling_eyes:

It may not be as intellectually pleasing as keeping a harsh wall between existentials and generics, but it could improve usability.

5 Likes

Being able to write AsyncSequence<String> (and other such things) will definitely be more than just useful, I have a feeling it will be critical as Swift grows. How does this interact with ABI?

e.g.

Can a developer wishing ABI stability for an API in a framework change the type they are returning later on?

Does that mean there is a cost of an existential heap allocation for that container?

If those two line items are resolved as "they can change without breaking ABI" and "it is not a heap allocation" then this is a game changer. I feel that it will make a good swath of new API written leverage this, because it gives the flexibility of classes with the speed and safety of structures!

I love the motivation here. I'm sure you'll come up with something that will help people. :sparkling_heart:

There's an opportunity to ease a lot of the friction that stems from having both associated types and generic placeholders. And that's applicable to everyone, not just the inexperienced.


For example, I don't believe this proposed syntax can work…

func maxValue(in collection: some Collection<Int>) -> Int

… because, which one of Collection's various associated types is that?

And look at how these functions have to be written so differently, based on level of genericism:

func count<Element>(array: Array<Element>) -> Int {
  array.count
}

func count<Collection: Swift.Collection>(collection: Collection) -> Int {
  collection.count
}

Also, while you can parameterize an associated type, or a function argument…

protocol Protocol {
  associatedtype IntCollection: Collection
  where IntCollection.Element == Int
}

func ƒ<IntCollection: Collection>(_: IntCollection)
where IntCollection.Element == Int { }

…you can't parameterize a typealias.


I propose a combination of

  1. support of labeling of generic placeholders and associated types
  2. associatedtype-esque disregarding of irrelevant generic placeholders
func maxValue(in collection: Collection<Element: Int>) -> Int
func count(array: Array) -> Int {
  array.count
}

func count(collection: Collection) -> Int {
  collection.count
}
typealias IntCollection = Collection<Element: Int>

I don't think the some keyword would be helpful to the programmer for this. It might be useful for decorating what would otherwise be existential parameters, for performance reasons?

1 Like

If you're using something like AsyncSequence with opaque types, here's how I think it would interact with API resilience:

  • Given an existing protocol AsyncSequence with an existing associated type Element, the protocol can opt Element into being a "primary" associated type without breaking ABI, allowing the some AsyncSequence<String> spelling.
  • Changing the requirements on the opaque type is not a resilient change. For example, you cannot change some AsyncSequence<String> to some AsyncSequence<Int>.
  • Given an opaque type some AsyncSequence<String>, changing the underlying return type is a resilient change.

I think it depends on what AsyncSequence<String> means. We could decide that this is an opaque type, or that you have to qualify it with some or any to distinguish between opaque and existential. One of the goals here is to push opaque types further, and those don't need an existential container since the underlying type can't change dynamically. We could also do what @Karl suggests and actually make the distinction not matter in all cases where the underlying type can't change dynamically, such as immutable function parameters.

Like you mention, part of the motivation for improving the ergonomics of generics is so that programmers can more easily maintain the wins of static type safety and its performance benefits :slightly_smiling_face:

5 Likes

So I want to make it clear, if you can pull this off with some slick syntax like some AsyncSequence<Int> etc - this is going to be perhaps one of the biggest improvements for the quality of APIs that folks that are bound by the constraints of ABI stability.

Insert Flip J Fry take my money meme here...

4 Likes

I should add for the point about treating existential function parameters like anonymous generics - the optimiser already does this in some cases, so we'd just be lifting this in to the language model. Depending on how we decide which functions qualify, this could be a source-compatible change.

It would help a lot of generic code. Simple example, using the clock protocol from the recent pitch thread:

func measure<ClockType>(
  _ body: () -> Void, using clock: ClockType
) -> Duration where ClockType: Clock {
  let start = clock.now()
  body()
  return start.duration(to: clock.now())
}

Currently, this requires a named generic type for the clock; you won't be able to access its Instant type or the now() method which returns an instance of it otherwise. By promoting existential function parameters to generics at the language level, we can lose the angle brackets and make these functions a lot simpler:

func measure(_ body: () -> Void, using clock: Clock) -> Duration {
  let start = clock.now()
  body()
  return start.duration(to: clock.now())
}

Allowing generic code to use this syntax, and opening up the full power of generics to code which happens to be written "the wrong way", and having it be source-compatible, just seems like a really great opportunity. Even if we don't do anything to inouts.

You could imagine this composing well with further shorthands, such as inline constraints. We could also consider more radical ideas to reduce angle bracket blindness, like allowing protocols which are only used once to stay anonymous and be referred to by their protocol name (e.g. for this function's return type, it's obvious which collection it's talking about):

func average(items: Collection where .Element: Numeric) -> Collection.Element? {
  ...
}

And I think this would be a considerable improvement over how you'd express the same thing today:

func average<CollectionType>(
  items: CollectionType
) -> CollectionType.Element?
 where CollectionType: Collection, CollectionType.Element: Numeric {
  ...
}

The any MyProtocol syntax may not be possible, due to the global any(_:) function:

Any & MyProtocol is already permitted, so could this be a backwards-compatible syntax for existential types? The compiler might generate a warning and fix-it in Swift 5.x, followed by an error in Swift 6 or later?

However, MemoryLayout<Any & MyProtocol>.size doesn't compile in Swift 5.5 — a workaround is to use a typealias of the protocol composition.

I'm not understanding, could you motivate? You can declare a global some(_:) function and it wouldn't collide with the some contextual keyword: the former can be used at the value level, the latter at the type level.

3 Likes

There is one use you seem to have missed: generic functions can’t be used as values, while functions over existentials can. (As I understand it this is a design decision, not a fundamental impossibility.)

any Protocol syntax for existentials, please. It’s a breaking change, but worth it to make Swift much more easier to understand.

2 Likes

Right, but it doesn't matter to the language model whether the existential spelling is literally transformed in to a generic. If we can't/won't support fully generic closures directly, an existential closure could immediately open all of its existential arguments and thunk over to a generic. Really, it's just extending the same automatic type -> value level boxing we already have, but in the other direction.

I think we need to get back to the idea of the syntax trying to express what you mean in the most natural way, and the compiler figuring out how to make it fast. It really shouldn't matter whether your closure/function value/function is written as:

let someFunc: (Collection) -> Void
// or
let someFunc: <C: Collection>(C) -> Void // hypothetical generic closure

Value-level abstraction and type-level abstraction are semantically interchangeable, except across rare points where a value-abstraction's (existential's) type changes. There's an analogy with how it's difficult to reason across suspension points from an await, and easier to reason about the synchronous bits between them. If you will never observe a change in type, there's no reason for your code to care about it, so everything should be possible in those contexts, IMO:

// Returns an existential - maybe a Range<Int>, or Array<Bool>, String, etc.
func giveMeACollection(size: Int) -> /* any */ Collection

// 'result' doesn't need to be bound by the limitations of existentials here.
// Its type will not change.
let result = giveMeACollection(size: 2000) 

result.startIndex   // Error. Doesn't need to be.
print(result.first) // Error. Doesn't need to be.

Parametric Polymorphism is hard very.. Sometimes the compiler it's self is misleading with some error messages. I solely support a source code break if possible.

The high curve of "associated types" on the Swift Language cannot be overemphasized for this same reason I wrote an article for me to understand how it works and how I can use it. Here is the article on Swift Associated Type Design Patterns
The collection aspect of it is what really gets me angry that I have to do a type-erasure else you cannot unify the collection into a single kind of T. Meaning anything that conforms to the protocol and provides a T should be able to stay in the collection. I don't think I am asking too much.
I would love to have a possibility to NOT implement the the Type-Erasure aspect if it. The compiler should be more intelligent and just understand me. But apparently the compiler is NOT intelligent enough to understand my intents and keeps requiring lots of information.

No matter the quantity of sugar, sooner or later they’ll get to some bitterness, for example having to write a type erasure.
At that point they’ll discover they maybe have been using generics and protocol oriented programming wrongly all along.

I've thought about this a lot over the last few weeks, and I think the transition might be feasible. I wrote up my thoughts in a pitch that we can discuss over here: [Pitch] Introduce existential `any`

6 Likes

If you're talking about encouraging the adoption of generics, in my experience compiler performance is also an issue.

A few times, I tried to adopt the "right way" and use more generics and protocols instead of a less ideal class-based inheritance design... but I hit a significant headwind with compile times.

I'm usually doing something with SwiftUI Views, so maybe there is something especially problematic with the mix of generics and result builders (like view bodies). I think that's where I've seen the more pathological stuff, like 20+ seconds type checking a ≈20 line function, that didn't look that complicated to the human eye.

Terms of Service

Privacy Policy

Cookie Policy