SE-0341: Opaque Parameter Declarations

Users don't need to know how the signature of append work to use the API. Their natural understanding of append is enough to understand what the method is supposed to do and what the result will be. Usually the only part they're thrown off by is the different between append and `appending. Beginners, and probably most users in general, will never look at the generic signature of a function. They just expect it to work. So I don't think your concern here is based on how users actually behave and rather overstated.

1 Like

Sure, I think it's fair to say that append is intuitive enough and maybe folks won't need to read the function signature. And to Doug's point, if the light weight syntax is also adopted and if the entire standard lib API is migrated, that wouldn't be an issue. If.

However, let's also think about every generic function in every Swift codebase to date that currently use angle bracket syntax. Many codebases contain functions like your example:

func save<Value: Encodable>(_ value: Value) throws

Refactoring these functions to use some will be a gradual process. Some function signatures may never be migrated, primarily because the function already works and there's not some immediate and overwhelming benefit to changing it. Even if the standard lib is migrated to some params on day 1, Swift users will be encountering type variables in angle brackets for years to come in both proprietary and open source codebases, and so they'll still need to learn both. Everyone updating all of their generic functions creates churn, and the mixed state that will exist in the interim (or forever) will be confusing and require more learning for these beginners that we claim to be helping.

Let's also keep in mind that not every programmer new to Swift is a beginner programmer. Many are coming from languages that make heavy use of type variables in angle brackets, Java, C++, C#, Kotlin, etc. some will make Swift generics less familiar and intuitive to this crowd, without any benefit to expressiveness or performance. It doesn't make Swift generics more powerful, just less approachable.

The point also remains that some now means different things in different contexts. My overarching point here is that it's not obvious, to me at least, that the learning curve is being improved at all. It's break even at best.

4 Likes

The protocol requirement and default implementation are written as:

  mutating func append<S: Sequence>(contentsOf newElements: __owned S)
    where S.Element == Element

The generated interface and documentation are translated into:

  mutating func append<S>(contentsOf newElements: S)
    where S : Sequence, Self.Element == S.Element

The constraint written in angle brackets has moved to the where clause.

Could the tooling automatically generate the simplified interface you posted?

(We would still want to gradually refactor code to use the improved syntax.)

4 Likes

Recall that Xcode already fibs in autocomplete lists, showing methods like this (misleadingly IMO) as append(contentsOf: Sequence). Having an actual abbreviated syntax would allow this to be spelled correctly and succinctly.

8 Likes

I'd like to drill into this point with a specific example of how this will be confusing. With this proposal, some P takes on a new meaning that does not overlap with opaque return types and is now context dependent. Imagine the following two function declarations:

// opaque return type, known only to the callee
func makeSomething() -> some P
// sugar for a type variable that you'd normally find in angle brackets
func takeSomething(value: some P) 

Depending on which of these functions a user encounters first, they'll decide on what they think some P means, but later they'll bump their head and realize that it actually depends on the context.

If they encounter makeSomething first, they'll decide that some P means a single, concrete, callee-specified type. When they later encounter takeSomething, they'll realize this can't be true as the callee doesn't construct argument values, so can't possibly be the one to specify their types. some P must mean something different here, more learning is required.

If they encounter takeSomething first, they'll decide that some P de-sugars to a type variable in the function declaration, as if it were written func takeSomething<T: P>(value: T). Later when they encounter makeSomething, they might assume that they could write something like:

let value: MyType = makeSomething()

Which won't compile because some P in the return position does not de-sugar to a type variable at all. It's not the same as writing func makeSomething<T: P>() -> T. Head == bonked and queue more learning.

So the idea that users "will need to learn the some syntax anyways" is false. They'll need to learn two different features which, very unfortunately, share the same keyword.

Putting myself in the shoes of someone who is new to Swift, I think my head would be swimming at this point. Then I inevitably encounter angle bracket type variables (see previous post, they're not going away nor are they "advanced") and I've got to learn that too?? I don't think we're doing them any favors with this proposal.

Side note about the name of the proposal

I think that "Opaque Parameter Declarations" is a misnomer. Generic arguments types are already opaque to the callee, sugaring from <T: P>(value: T) to value: some P does nothing to make the type of value more opaque than it already was. A more appropriate name might be Anonymous Argument Type Variables, because all we're doing here is obviating the need to name a generic type before you can use it (but only for arguments, not for a return type, because that would clash with opaque return types and will never be possible using the word some).

5 Likes

+1. I don't have anything to add to the discussion except for my comment on "Light-weight same-type requirement syntax" pitch.

Regarding this comment from your other post:

“I believe Swift will become the language that introduces the majority of new programmers to generic programming in a few years.”

I’m genuinely curious, what’s the justification for this belief?

There's something in this reasoning. Should we use different names for different concepts? For example this or a better alternative:

func makeSomething() -> some P
func takeSomething(value: generic P) 
3 Likes

I don't feel like these are two separate concepts at all. In both cases, some P means some specific kind of P; you just don't know what it is. That's true in the parameter position, and it's true in the return position.

You can describe some P in return position as "reverse generics" if it helps you understand the concept, but that is not the official description of the feature. It's an "opaque type"; some specific type that conforms to P, you just don't know exactly what it is.

You can describe some P in parameter position as "generic" if it helps you understand the concept, but you don't have to understand generics to use it. It's exactly the same concept as in return position; when you receive it, it's some specific type of P; you just don't know exactly what it is.

I don't see why the "callee-constructed"-ness of the type would be understood as essential to some here. The name some doesn't suggest "callee-constructed". I would think that the callee-constructed-ness of it would be inferred not by the keyword some, but rather by the fact that the value is presented as a return type, and the return type comes from the callee, so the callee must have constructed it.

They'll only decide this if they're already familiar with generics. I view this feature as a nice step in Swift's philosophy of progressive disclosure. Most users a few years from now will likely encounter func f1(p: some P) long before they encounter func f2<T: P>(t: T). If that's the case, then users wouldn't immediately start by seeing some P as a sugar for generics—rather, generics would be seen as a more advanced version of opaque parameter types, to be learned later.

In short, I like this proposal. +1.

10 Likes

I replied to the wrong address. This is a reply to @bjhomer.

I am one of people who feel "callee-constructed"-ness is essential to some. In my case, this feeling comes from the strangeness to have two different equal-ness in the same language.
As we all know, generic parameters and generic result types are similar features. Then, generic parameters and 'reverse-generic' result types will seem to be similar features, because they share the same some syntax.
As a result, we will have two different identity relating to generics in Swift, if we introduce proposed feature. One is whether it is generic or reverse-generic, and the other is whether it is some or not. We don't usually need two different viewpoints, but there are two.
If I consider "callee-constructed"-ness is essential to some, then there is only one sameness in Swift. This is much more natural for me.

1 Like

When users learn that the advanced angle bracket syntax can be used for some parameter types they would try to apply that advanced angle bracket syntax for some result type and would be confused why they can't.

1 Like

I somehow got reminded on something that has been posted long ago:

Staying in this analogy, I lately got a strong feeling that there are some bricks which Core does not like anymore, but instead of searching or creating new stones, they try to reshape everything with lots of mortar…

I'm not opposing this particular proposal and don't think it will change Swift significantly, but small things add up, and it looks unavoidable that the language becomes more and more complex and less elegant :-(.

2 Likes

My justification will need a full sized article and in the end, it will be just my opinion. Tbh, I don't have time to get into this discussion. Let's assume it is my gut feeling without justification and leave it at that.

1 Like

I see this proposal, SE-0309 (unlocking existentials), SE-0335 (existential any), and the lightweight syntax as all pieces of the same puzzle: both existential types and opaque types are getting more general and are growing the same syntax for representing constraints. We don't want one monolithic proposal, and I don't think it makes sense to try to build up a tower of conditional-accepts either. Rather, I think we treat this as one piece in a larger story: by itself it generalizes some in a natural way and makes a few things cleaner, and with more of the puzzle pieces in place it becomes more valuable.

I suppose it could, although it would be a lot easier for us to go refactor the code to make it nicer.

I had completely forgotten this! And yes, it would be much nicer to write out the some Sequence<Element> here in autocomplete.

If you haven't done so before, I recommend reading the Rust RFC for a similar feature, especially the section on learnability. I think this use of some lines up with programmer's intuition about arguments vs. return values, and who gets to choose those values, which extends naturally to types.

Doug

7 Likes

Ok. Let’s assume you’re right and everyone learns generics via Swift, all the more reason to get them right! Maybe you've got some feedback on the critiques above that don't involve writing a full article? Your input is valuable and would be much appreciated.

Yes, I think we should use different names for different concepts and I would be in favor of this syntax. Great suggestion.

I think "reverse generics" is a total misnomer and creates significant cognitive dissonance. See my points above about how opaque return types cannot, in any way, be rewritten as a generic type parameter. The return type is strictly not generic, it's just hidden. Once you've selected the return type (as the function author), it will only ever be that one type unless you choose to refactor it's internals. When you write a generic function, the type variables can take on any type that fit the constraints depending on how they are applied by the caller. That is what makes a function generic.

Also, the types of generic arguments are always unknown to the function author, even with the existing syntax. This proposal does nothing to make them more opaque than they already are. I think a better name for this proposal would be Anonymous Type Variables. It would allow you to create a generic type inline without having to declare it beforehand in the angle brackets, which is pretty cool, but I think not similar at all to the existing idea of some.

Can you define what in your view "generic" means, such that you can state opaque return types are strictly not generic?

1 Like

Happy to do so, and please correct me if my understanding is incorrect. I'm hoping it is incorrect because I'm really confused by the notion that opaque return types are considered generic.

My understanding of generics, thanks for reading

Generic functions (or structs, etc) contain type variables that act as a placeholder that can be filled in with a real, concrete type that fits the constraints after the fact and which are unknown to the function author. Generics are sort of like a template. Depending on what types are supplied to the type variables (either within the author's module or by users of the module), the compiler can and will generate copies of the function (or struct, etc), with those selected types filled in. This is called monomorphization, and it's great because it allows polymorphism without dynamic dispatch (but at the cost of code size).

Opaque return types (in my understanding) do not create type variables that can participate in monomorphization. They are compiled to have a single, unchanging return type which is known at compile time of the module, even if the function is being exported in a library where it might be called by others.

Generic functions, on the other hand, must maintain their type variables after compilation (at least when publicly exported). This is so that they can be further monomorphized by users compiling their code against the library with the generic function. Otherwise we'd be limited to exporting functions that only use existentials which have performance drawbacks (eg dynamic dispatch).

To give an example, here's a function with an opaque return type:

func makeString() -> some StringProtocol {
    return "hello"
}

This can (and in my understanding, will) be compiled down to:

func makeString() -> String {
    return "hello"
}

with the caveat that some StringProtocol is preserved in the module's interface, as to not expose the actual static type, avoiding dynamic dispatch and API fragility. But under the hood, (namely at the SIL layer, I think) it really is just a function that returns a String (doesn't sound very generic to me, but who knows?).

Compare that (to what I think of) a generic function that has an explicit type variable:

func printDescription<T: CustomStringConvertible>(value: T) {
    print(value.description)
}

Perhaps if this function is only used internal to the module it's defined in, it can be monomorphized to all of its use cases, the type variables are eliminated and the original function disappears. But, if the function is export in the module's public interface, it can't be compiled away, because users of the library may want to supply their own custom types to the type variable T and generate additional monomorphizations.

That distinction is what, in my mind, makes opaque return types distinct from generics. Please do provide any insight if I'm off base, thanks in advance!

2 Likes

Thanks Doug, the Rust RFC was helpful, specifically this line regarding impl Trait

If you pick the value, you also pick the type

That gives me an intuition of what some means, both in parameter and return positions. However! This is a departure from true (or at least classical / historical) generics, where the type variables are always filled in by the caller (both in Swift and many other languages) and can vary depending on the context. So the fact that we're moving away from that notion is what makes me feel like some (at least in the return position) is tangential but not equivalent to generics.

Maybe someone who is new to programming will not be surprised by this and simply adopt the intuition found in the quote above. But to me, it's very surprising that we consider some in the return position generic, where the type is supplied by the callee, and limited to a single known (to the compiler) type that does not and cannot vary after the fact.

1 Like

That. Plus from a pure lexical standpoint, if these two are considered equivalent:

func foo<T: P>(param: T) { ... }
func foo(param: some P) { ... }

users will rightfully assume these should be also equivalent:

func foo() -> some P {}
func foo<T: P>() -> T {}

and be very much surprised to learn they are not!


A different keyword can work here and make it mentally (and otherwise) unambiguous:

func foo<T: P>(param: T) { ... }
<--->
func foo(param: generic P) { ... }

func foo<T: P>() -> T {}
<--->
func foo() -> generic P { ... }

and another thing of its own kind with no angle bracket innuendo:
func foo() -> some P {}

Alternatively...

Alternatively something like this (with no extra keyword):

func foo<Apple: Comparable>(param: Apple) { ... }
<--->
func foo(param: Comparable Apple) { ... }

func foo<Apple: Comparable>() -> Apple {}
<--->
func foo() -> Comparable Apple { ... }

This latter form would allow repeating Apple more than once, allowing more advanced use cases.

func foo(apple: Comparable Apple, apple2: Apple) { ... }
    <--->
func foo<Apple: Comparable>(apple: Apple, apple2: Apple) { ... }
2 Likes

In the general case, the underlying return type is unknown to the compiler of the caller and it can vary.

To develop the intuition more fully, consider:

Given func f() -> some P and func g(_: some P)—aka func g<T: P>(_: T)—you can call g(f()), supplying the outer function with the result of the inner function without knowing the type.