SE-0341: Opaque Parameter Declarations

[xwu (Xiaodi Wu)]

@Jumhyn

Importantly, in both cases we are on the side of the generics where we could ostensibly be dealing with any conforming type, but we still know more about the behavior of g 's return value than we know about f 's parameter.

As I've said before, this would be no different than is the case for a non-final class C, today:

func f(_ t: C) {
  usedTypesF.insert(ObjectIdentifier(type(of: t)))
}
func g() -> C {
  return C()
}
usedTypesG.insert(ObjectIdentifier(type(of: g())))

[Jumhyn (Frederick Kellison-Linn)]

Sure—I don't think this is that big of an 'issue' compared to the advantages of this syntax, but I think it's important to give the argument a fair shake that return and parameter position for some types are not always just the same but with a flip of perspective.

When the issue is restricted just to classes which share a common base you have some minimal guarantees about things such as (I believe) mutual layout compatibility. I don't think there's much you could usefully do with the existing invariant of some P in return position, but I understand the instinct that there is a way in which opaque return types are more 'singular' than opaque parameters would be.

[allevato (Tony Allevato)]

@AlexanderM

Would it be possible to reference the type variables of sequence1 and sequence2 in a where clause? E.g. something like:

func eagerConcatenate(
    _ sequence1: some Sequence, _ sequence2: some Sequence
) -> [T] where type(of: sequence1).Element == type(of: sequence2).Element

It was mentioned in the pitch thread that this wouldn't be possible.

I agree with the rationale there; if there's a critical need to refer to the member types of the generic parameter type, then the generic parameter type should just be given an explicit name. This proposal aims to reduce boilerplate for a certain class of common uses of generics, but I think it would be bad to start adding other ways to extract things like member types from an unnamed generic parameter. Something like the syntax above would effectively create two parallel generics syntaxes in the language, instead of one just being sugar for the more complex one.

[Douglas_Gregor]

@GreatApe

You can of course argue, that the meaning of some is not quite this, but rather something like "unnamed and chosen by the relevant party". The relevant party in a parameter being the caller, and in the return type it's the callee.

This formulation is effectively what I'm saying, and as @Michael_Ilseman said elsewhere. The caller chooses the type for the parameters, and the callee chooses the type for the result. It's the same thing with values: the caller chooses the values passes into the parameters, and the callee chooses the value that's returned.

When this has come up before, folks have pointed at the motivation for the similar impl Trait feature in Rust. This phrase really stood out for me:

But, on the other hand, programmers have a very deep intuition around the difference between arguments and return values, and "who" provides which (amongst caller and callee).

I think that is absolutely true here. As to this...

@GreatApe

But if this is the argument, then I would say that this is confusing, since in Swift we do allow a situation where the caller determines the return type, in normal generics.

We do allow inference when the generic parameter is stated in the return type

func f() -> T

but it seems more and more like designing functions like that is considered poor form, because you always need type context to disambiguate. I tend to see more generic functions following the approach taken by unsafeBitCast(_:to:), where you ensure that you can deduce all generic arguments from the argument types:

func unsafeBitCast<T, U>(_ x: T, to type: U.Type) -> U

@Jon_Shier

One bit of awkwardness I noticed, unless I'm missing something, is that to start using a generic, you have to start with a placeholder, as the opaque syntax doesn't sugar unqualified types. func do<T>(with value: T) doesn't seem to be helped by this proposal

Right, it could be written as func do(with value: some Any) but it's not that much better.

@ensan-hcl

In the future, full-power reverse generics syntax (often reffered as func foo() -> <T> T or func<^T> foo() -> T ) can be introduced. Swift then has three different features at that point; 1: generics, 2: some , 3: reverse generics. Among these three features, some only seems a chimera of generics and reverse generics.

Let's say I write a function like this:

func g(value: SomeClass) -> SomeClass

As a caller, I can provide a SomeClass or some subclass of it. Only I know the specific type. The value I get back from calling g will be a SomeClass or some subclass of it. Only g knows the specific type. This is not a contradiction, it's how arguments and results work, consistently throughout Swift and many other languages.

@ensan-hcl

However, considering the next code, it seems quite strange because function f and closure f both have type (some Numeric) -> () but behave totally different.

// proposed syntax
func foo(f: (some Numeric) -> ()) {
    f(42) // f is not a generic function
    f(42 as Int) // error
    f(42 as Float) // error
}
// proposed syntax
func f(_ value: some Numeric) {}
f(42)     // ok, f is a generic function here!
f(42 as Int) // ok
f(42 as Float) // ok

... this is contravariance of parameters at work. foo takes a function which accepts a closure taking a specific Numeric type and returning nothing. Unless you have a value of that specific Numeric type, you won't be able to call it. f takes a value of some specific caller-chosen type that is Numeric.

@ensan-hcl

SE-0328 rejected adding some P in 'consuming' position of closure, because at that point we were afraid that adding such a closure contradicts with this SE-0341 == 'generalized some syntax'. But this SE-0341 introduces the some P in 'consuming' position of closure in the parameter position of function. At least, please explain, why?

The simple "why" is that it falls out of the direct translation of some in parameter position into generics. The SE-0328 decision had two bits of reasoning: the potential conflict with this proposal (i.e., we could choose for those cases SE-0328 banned to be treated as callee-chosen generic arguments as in those proposal) and because the semantics you'd get from SE-0328 are really not that useful. I suppose you could say that the second bit of reasoning could apply to this proposal as well: because the caller is choosing the parameter type of the closure parameter in your foo example, and you don't have another way to name that type, you can't really call that closure. We could say that that's enough reason to also ban the use of some P in covariant positions within parameters.

Doug

[ensan-hcl (ensan)]

@Douglas_Gregor

this is contravariance of parameters at work

But I don't know any other example in Swift in which closure and function behave differently. Can you explain more in detail?

class Animal {}
class Dog: Animal {}
class Cat: Animal {}
func foo(f: (Animal) -> ()) { 
    f(Dog())  // ok
    f(Cat())  // ok
}
func f(_ value: Animal) {}
f(Dog())  // ok
f(Cat())  // ok

@Douglas_Gregor

I suppose you could say that the second bit of reasoning could apply to this proposal as well: because the caller is choosing the parameter type of the closure parameter in your foo example, and you don't have another way to name that type, you can't really call that closure. We could say that that's enough reason to also ban the use of some P in covariant positions within parameters.

Then I suggest banning it, because the first bit of reasoning (the conflict) is a larger problem, if we must introduce some. But I suppose you can just avoid some so that the problem does not occur.

@Douglas_Gregor

As a caller, I can provide a SomeClass or some subclass of it. Only I know the specific type. The value I get back from calling g will be a SomeClass or some subclass of it. Only g knows the specific type. This is not a contradiction, it's how arguments and results work, consistently throughout Swift and many other languages.

I'm always puzzled why people use subtyping as an example when we are talking about specification of type parameters. They are distinct concepts. I think the proposed feature is extracting parts from generics and 'reverse generics' that behave like subtyping. It's just making different things look similar.

[tem (GalaxySwift)]

@Jumhyn

Viewed from the opaque side of the type, there is an invariant enforced for callee-side generics that is not enforced for caller-side generics—namely, that the identity of the unknown type remains the same across different invocations.

@GreatApe

func test(viewParam: some View) -> some View {
    let y = viewParam
    // ...
    return a
}

Yes, excellent points! But it's not a hard rule that opaque return types are always the same.

Here is a silly example of that, but it works:

func test<V: View>(viewParam: V) -> some View {
    return viewParam // OK, non-fixed
}

So it could be said that the invariance is only due to opaque return types normally not being constrained to the parameters (because there hasn't been a way to spell that), right?

If that is the case, then isn't it also fair to say that opaque types are already just a shorthand for generics, and that the spelling some is just sugar for that not reserved specifically for reverse-generics? I.e. this proposal could have been part of the original proposal for opaque types and if it had, this association would have never been made?

@allevato

This proposal aims to reduce boilerplate for a certain class of common uses of generics, but I think it would be bad to start adding other ways to extract things like member types from an unnamed generic parameter. Something like the syntax above would effectively create two parallel generics syntaxes in the language, instead of one just being sugar for the more complex one.

Yes, great point! That doesn't seem ideal.

[Douglas_Gregor]

@ensan-hcl

But I don't know any other example in Swift in which closure and function behave differently. Can you explain more in detail?

class Animal {}
class Dog: Animal {}
class Cat: Animal {}
func foo(f: (Animal) -> ()) { 
    f(Dog())  // ok
    f(Cat())  // ok
}
func f(_ value: Animal) {}
f(Dog())  // ok
f(Cat())  // ok

It's not about closure-vs-function; it's about value of specific type vs. value of function type. Extend your example like this:

func g(_: Dog) -> Void {}
foo(f: g) // error: cannot convert value of type '(Dog) -> Void' to expected argument type '(Animal) -> ()'

Why is it that I can convert Dog to Animal but not (Dog) -> Void to (Animal) -> Void? Because function types are contravariant in their parameters.

@ensan-hcl

I'm always puzzled why people use subtyping as an example when we are talking about specification of type parameters. They are distinct concepts.

When you have caller-chooses generics and callee-chooses reverse generics (opaque result types, potentially made more explicit by named opaque result types), the fact that result types are covariant and parameter types are contravariant matters has an impact on usability. We banned this in SE-0328...

func f() -> ((some P) -> Void) {
  // ...
}

because it's hard to use that result type for anything. How do you call the function you get back, when you can't express the type of the parameter?

Doug

[ensan-hcl (ensan)]

@Douglas_Gregor

Why is it that I can convert Dog to Animal but not (Dog) -> Void to (Animal) -> Void ? Because function types are contravariant in their parameters.

I know the behavior. But I never talked about passing value of function type, I'm talking about using the closure only after it is passed. I'm asking why the closure and function which have the same signature in appearance behave such differently. It's not a normal behavior in Swift.

---

@Douglas_Gregor

When you have caller-chooses generics and callee-chooses reverse generics (opaque result types, potentially made more explicit by named opaque result types), the fact that result types are covariant and parameter types are contravariant matters has an impact on usability. We banned this in SE-0328...

func f() -> ((some P) -> Void) {
  // ...
}

because it's hard to use that result type for anything. How do you call the function you get back, when you can't express the type of the parameter?

FWI, there are a lot of way to call such functions with protocols like Numeric. But to be clear, the existence of usage is not a big problem here, because the spelling of (some P) -> () is a natural application of structural opaque types. Like it's not banned to write func f<T>(c: (T) -> ()) although there is no way to call c, we should not ban a specific usage without special reasons. Therefore, what is problem is whether banning has enough inevitability and conviction. But the problem is not inevitable. I suggest using another keyword for the sugar of generics to avoid problems.

In terms of user-friendliness, it's hard to understand why something that was introduced as a generic sugar behave in the complex rule of subtyping, because the original generics can be explained as simple replacement of type parameter into concrete type. Shouldn't the sugar also be a simple replacement like this?

func foo(value: generic P) 
func foo<T: P>(value: T)

func foo() -> generic P
func foo<T: P>() -> T

---

Some people claimed that the proposed usage of some P creates 'dual' or 'symmetry', but if such a aesthetics is important, the asymmetric behavior of (some P) -> () closure is a significant defects, I suspect...

[xwu (Xiaodi Wu)]

@ensan-hcl

Shouldn't the sugar also be a simple replacement like this?

func foo(value: generic P) 
func foo<T: P>(value: T)

func foo() -> generic P
func foo<T: P>() -> T

The argument, fundamentally, is that the syntax is more useful as a simple replacement like this--

func foo(value: xxx P) 
func foo<T: P>(value: T)

func foo() -> xxx P
func foo() -> <T: P> T

where xxx is impl in Rust and some in Swift. The reasons for that are outlined in ample fashion in the Rust RFC.

Other, less used or usable, permutations of 'who chooses' the parameter is rightly left out of the shorthand spelling and can be enabled in unsugared form when the need arises:

// Already supported but discouraged:
func f<T>() -> ((T) -> Int)  { ... }

// Not yet supported officially, but could be enabled by
// named opaque return types:
func f() -> <T> ((T) -> Int) { ... }

// Not supported, but spelling is self-explanatory
// if/when it becomes so:
func f() -> (<T>(T) -> Int)  { ... }
func f() -> ((T) -> <T> Int) { ... }

[xwu (Xiaodi Wu)]

@Douglas_Gregor

I suppose you could say that the second bit of reasoning could apply to this proposal as well: because the caller is choosing the parameter type of the closure parameter in your foo example, and you don't have another way to name that type, you can't really call that closure. We could say that that's enough reason to also ban the use of some P in covariant positions within parameters.

I think the behavior is non-obvious enough that it may be best that we do apply the same reasoning to ban some P in covariant positions within parameters.

---

For clarity, is the following permitted?

func foo(f: () -> some P) { ... }

...and if it is, does it desugar to func foo<T: P>(f: () -> T) { ... },
or does it mean (notionally) func foo(f: () -> <T: P> T) { ... }?

IMO, it has to mean the latter or else it would be very confusing with other uses of () -> some P enabled in SE-0328. And if that's the case, then that gives good reason not to permit (some P) -> () in parameter position with the desugaring that's proposed.

[ensan-hcl (ensan)]

@xwu

// Not yet supported officially, but could be enabled by
// named opaque return types:
func f() -> <T> ((T) -> Int) { ... }

// Not supported, but spelling is self-explanatory
// if/when it becomes so:
func f() -> (<T>(T) -> Int)  { ... }
func f() -> ((T) -> <T> Int) { ... }

Could you please explain the difference of these codes? I can understand the first one well, but I don't understand the second and third ones. Is the second one 'generic closure' and is the third one 'reverse generic closure'?

[xwu (Xiaodi Wu)]

@ensan-hcl

Is the second one 'generic closure' and is the third one 'reverse generic closure'?

Yup

[ensan-hcl (ensan)]

I still don't understand the replacement rules. My understanding is as follows. Is this correct?

If some P is found, check whether it is argument side or return side in relation to the (structurally) nearest arrow, add <T: P> at the head if it is argument side, or immediately after the arrow if it is return side, and replace some P with T.

func f0(value: some P)
func f0<T: P>(value: T)

func f1(value: () -> (some P))
func f1(value: () -> <T: P> T)

func f2(value: (some P) -> ())    // error
func f2(value: <T: P> (T) -> ())  // error, because it represents 'generic' closure, which is not supported currently

func f3() -> some P
func f3() -> <T: P> T

func f4() -> () -> (some P)
func f4() -> (() -> <T: P> T)

func f5() -> (some P) -> ()      // error
func f5() -> (<T: P> (T) -> ())  // error, because it represents 'generic' closure, which is not supported currently

[xwu (Xiaodi Wu)]

@ensan-hcl

If some P is found, check whether it is argument side or return side in relation to the (structurally) nearest arrow, add <T: P> at the head if it is argument side, or immediately after the arrow if it is return side, and replace some P with T .

I agree that this is a very understandable way of desugaring.

As I asked earlier, it seems that as proposed, because (some P) -> () is not permitted otherwise, f2 is currently allowed and desugars instead to func f2<T: P>(value: (T) -> ()). I'd agree with concern that this is inconsistent and is best off not permitted as you suggest here and as @Douglas_Gregor contemplated earlier.

[Michael_Ilseman]

@GreatApe

From the outside, we don't know what x is, but we know it will always be the same type.

That's dependent on many things, such as the type of a (and the caller might itself be parameterized over it).

I think there's a lot of complexity and nuance (and asymmetry) smuggled in under the concept of "fix"ing a function.

The callee may be a dynamically-dispatched method (via protocol or class), or the callee might itself be generic (like the identity function), or it may change over time under separate compilation, etc. Fixing a function requires use of language-level tools and reasoning at the call-site for this same-type-every-time inference to hold, and those tools and reasoning themselves are asymmetric.

If you fix the callee, for a complex and nuanced notion of fix, then the caller can make that kind of inference.
If you fix the caller, for a complex and nuanced notion of fix, then the callee can make that kind of inference.

There's asymmetry surrounding the relationship between callers and callees, from how the calling convention works, to language tools for reasoning about or reducing dynamism, to how libraries evolve over time differently than their clients. Reasoning about all the ways you actually call a library function is different than reasoning about all the ways your library function could be called. Similarly, this same-type-every-time inference (assuming there's value to it) may be more useful in one direction than the other.

As far as I can tell, the use of some syntax for an unnamed generic type is actually one of the (few) symmetries surrounding callers and callees.

[ensan-hcl (ensan)]

Thank you. Maybe now I understood your interpretation.

However, I suspect the rule is broken, because SE-0328 introduced () -> (some P) as f6, not as f7. I think f6 is not equal to f7.

func f() -> () -> some P
func f6() -> <T: P> () -> T
func f7() -> (() -> <T: P> T)

As you explained, (<T: P> (T) -> ()) is a 'generic closure'. 'Generic closure' is the feature that enables to treat generic functions as objects, while it is still generic. Likewise, (() -> <T: P> T) is 'reverse generic closure'. It enables to treat reverse generic functions as objects, while it is still 'reverse generic'. Therefore, f7 would be able to have the following implementation, unlike f6.

func x() -> <T: P> T { PX() }
func y() -> <T: P> T { PY() }

func f7() -> (() -> <T: P> T) {
   Bool.random() ? x : y
}

If so, the rule of replacement for () -> (some P) in return position is not consistent with actual behaviors. Therefore, your interpretation is broken in this point.

But if my concern is wrong, I hope your interpretation will be clearly stated in the proposal and (some P) -> () closures are banned.

[Douglas_Gregor]

@xwu

For clarity, is the following permitted?

func foo(f: () -> some P) { ... }

...and if it is, does it desugar to func foo<T: P>(f: () -> T) { ... } ,
or does it mean (notionally) func foo(f: () -> <T: P> T) { ... } ?

IMO, it has to mean the latter or else it would be very confusing with other uses of () -> some P enabled in SE-0328. And if that's the case, then that gives good reason not to permit (some P) -> () in parameter position with the desugaring that's proposed.

As written, it means for former. I agree with you and @ensan-hcl that the semantics of some in function types within parameters, as in return types of SE-0328, is sufficiently confusing that we should prohibit it in this proposal.

Doug

[xwu (Xiaodi Wu)]

@ensan-hcl

However, I suspect the rule is broken, because SE-0328 introduced () -> (some P) as f6 , not as f7 . I think f6 is not equal to f7 .

func f() -> () -> some P
func f6() -> <T: P> () -> T
func f7() -> (() -> <T: P> T)

Eek, I think you are right, and I think that’s a problem I don’t recall being discussed. I wonder if we can amend SE-0328 to postpone this coming into effect until further discussion.

[ensan-hcl (ensan)]

The difference among the interpretation of proposal, the interpretation of @xwu, and the interpretation I've been suggesting can be summarized as the difference in possible ranges of <T> position for some.
First, in my interpretation, the <T> that some replaces is always at a fixed position "after -> of func". In this case, the position of some is irrelevant.

func f0() -> some P
func f0() -> <T: P> T

func f1(value: some P) -> Void
func f1(value: T) -> <T: P> Void

func f2(closure: (some P) -> (some P)) -> (some P) -> (some P)
func f2(closure: (A) -> B) -> <A: P, B: P, C: P, D: P> (C) -> D

Next, as for the interpretation of the proposal, depending on whether some is argument side or result side with respect to -> of func, there are two possible positions of <T> that some can replace.

func f0() -> some P
func f0() -> <T: P> T

func f1(value: some P) -> Void
func f1<T: P>(value: T) -> Void

func f2(closure: (some P) -> (some P)) -> (some P) -> (some P)
func f2<A: P, B: P>(closure: (A) -> B) -> <C: P, D: P> (C) -> D

Finally, from @xwu's point of view, depending on whether some is argument side or result side for the structurally closest ->, the position of <T> that some replaces changes.

func f0() -> some P
func f0() -> <T: P> T

func f1(value: some P) -> Void 
func f1<T: P>(value: T) -> Void

func f2(closure: (some P) -> (some P)) -> (some P) -> (some P)
func f2(closure: (<A: P> (A) -> <B: P> B)) -> (<C: P> (C) -> <D: P> D)

Among these three interpretations, my position requires rejecting SE-0341 (or using other keywords), the position of this proposal requires allowing (some P) -> () in both return position and argument position, and @xwu's position requires amending SE-0328 and SE-0341.
Since most of people in this thread don't seem to be supportive of my position, I suggest as the next best thing that SE-0328 and SE-0341 will be modified so that @xwu's consistent interpretation can be adopted.
At the very least, I absolutely disagree with disallowing (some P) -> () as an exception while keeping the position that opaque parameters some are syntax sugars for generics. If we are going to push that position, then (some P) -> () should always be allowed, otherwise it's not consistent.

[Alejandro_Martinez]

I have to say that this is a very nice way to seeing the differences of what everybody is talking about. Thanks for clarifying that.