Reverse generics for argument type?

[ensan-hcl (ensan)]

Reverse generics and opaque result types

Introduction I've been trying to wrap my head around SE-0244: Opaque Result Types and recently I had some realizations that helped me understand the proposal better. I noticed I'm not the only one confused about it, so I thought I'd write down some things that might help others to get a better grasp of the concept as well. In doing so, I'd also like to try and rebrand opaque result types, or at least the underlying mechanism, to something I'm calling 'reverse generics'. I think viewing this al…

Improving the UI of generics

@Douglas_Gregor wrote the Generics Manifesto almost three years ago, which provided the roadmap for the core of Swift's protocols and generics features. Since then, we've implemented almost all of the basic model that it envisioned. Although there are further features we could add, we think we have a solid baseline; meanwhile, we've gained practical experience with the model in developing the standard library, we've absorbed lots of feedback from the larger Swift development community, and we've…

SE-0309: Unlock existential types for all protocols

Hi everyone. The review of SE-0309, "Unlock existentials for all protocols", begins now and runs through May 1, 2021. Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager. When emailing the review manager directly, please keep the proposal link at the top of the message. What goes into a review? The goal of the review process is to improve the …

I read these discussions, and I now have a concern about it.

In theory, we can think about what happens if reverse generics was applied in argument type considering the next example.

// Reverse generics in result
func callee<opaque T: FixedWidthInteger>() -> T { /* ... */ }

// Reverse generics in argument
// The actual type of `T` is `Int64`, but from the caller it seems to be opaque
func callee<opaque T: FixedWidthInteger = Int64>(_ value: T) { /* ... */ }

<opaque T: FixedWidthInteger = Int64> is my instant syntax, which says the actual type of T is Int64, but tells user only that the T conforms to FixedWidthInteger.

It is easy to reject using this, but this has possibility to be used.

Just for example, FixedWidthInteger has constraint to have static let max: Self. Then we can use this function by using this.

// Call the function with leading dot syntax
callee(.max)

Although caller cannot know the actual type of the argument, constraint ensures that the type has static value max. Therefore, caller can use .max.

But caller cannot use this with explicitly written type, because caller doesn't know this is correct.

// Error, because caller cannot know the actual type of argument is Int64 or not.
callee(Int64.max)

By considering it, we can have clear explanation of generics and reverse generics.

Who decide actual type \ position	Argument Type	Result Type
Caller	Generic Argument	Generic Result
Callee	Reverse Generic Argument	Reverse Generic Result

Without it, the cell of "Argument Type/Callee" got empty.

I don't argue reverse generic argument is highly useful, but in some cases it would be useful, for example, the situation you don't want to make it visible what the actual argument type is. If you are using module private types for the argument, you can use reverse generic argument so that you don't have to make the type public.

Again, I don't think such situations are common and I don't feel it important to implement this feature, but we have to admit that such feature can be considered.

However, at least, the next syntax should be reconsidered, because potentially it excludes the opportunity to use reverse generic argument with some keyword. This is quote from here

@orobio

Additionally, the proposal mentions a possible future direction that provides a similar way to define anonymous normal generic parameters:

// Possible future direction with 'some' used
// for anonymous normal and reverse generic types:
func makeCollection(with number: some Numeric) -> some Collection {
  return [number]
}

// This corresponds with the following syntax
// using named normal and reverse generic parameters:
func makeCollection<T: Numeric, ^C: Collection>(with number: T) -> C {
  return [number]
}

// But it also could be this:
func makeCollection<^T: Numeric = Int64, ^C: Collection>(with number: T) -> C {
  return [number]
}

How do you think about these things? I couldn't find any previous refers in the forum about this. I'm sorry if this discussion has been already held. Thank you!

[Moximillian (Mox)]

EDIT: as discussed downthread in Reverse generics for argument type? - #14 by xAlien95,
using some as sugar for normal generics in function parameter might not actually make sense, and alternatively could be used as syntax for reverse generics in function parameter. This post has been rewritten.

Examples of uses for some

protocol P {}
extension Int: P {}
extension String: P {}

func (value: Int) -> some P { // this is the Opaque type
      return value
} 
var value: some P = 1 // works in Swift 5.4
func (value: some P) {} // not currently supported

[ensan-hcl (ensan)]

(EDIT: This reply doesn't make sense now because targeted message edited.)

Just for checking my understand, but how do you write the type like [some Hashable: any Numeric]?

From your example, it seems that your way to use them is prefixing some/any before the type part like some [P].

However, I suspect it would not work in this case. Type like some any [Hashable: Numeric] is hard to read and difficult to determine which keyword modifies which protocol.

[tem (GalaxySwift)]

It's not possible to have some [Hashable: Numeric]. The brackets is a dictionary literal, i.e. Dictionary<Hashable, Numeric> but opaque types only works with protocols as constraint, not concrete (generic) types. Does that make sense?

[ensan-hcl (ensan)]

No, I'm saying [some Hashable: any Numeric] . You can make this type like:

func someFunction() -> [some Hashable: Numeric]  { /* ... */ }

For me it seems possible to do this if this is possible.

@Moximillian

func () -> some [P] {} // this is the Opaque type

And if we agreed to use any for existential type, it becomes any Hashable, so, we would have this.

func someFunction() -> [some Hashable: any Numeric]  { /* ... */ }

If there is fault to use dictionary, I can say the same thing for tuple.

func someFunction() -> (some Numeric, any Numeric)  { /* ... */ }

Doesn't it make sense?

[xAlien95 (Stefano De Carolis)]

We already have reverse generics/opaque types in argument position. Just write the following in a playground and check the type of f:

protocol Protocol {
  func foo(_ p: Self)
}

struct Struct: Protocol {
  func foo(_ p: Self) {}
}

let s: some Protocol = Struct()
let f = s.foo
// f is (some Protocol) -> Void
// f is (some Protocol (type of 's')) -> Void

As you said, unlike generics, with reverse generics the type is decided by the callee (in this case is Self, which is deduced by the protocol interface).

Thus, this code snippet doesn't make sense:

@ensan-hcl

// Possible future direction with 'some' used
// for anonymous normal and reverse generic types:
func makeCollection(with number: some Numeric) -> some Collection {
  return [number]
}

You cannot use an opaque type without actually providing the underlying type (directly or by means of the context in which the function is declared). That function shouldn't be callable by design (you know that it accepts a specific conforming Numeric type, but you don't know which type).

[ensan-hcl (ensan)]

It's my fault, but this is quotes from here. I fixed the expression.

Reverse generics and opaque result types

Introduction I've been trying to wrap my head around SE-0244: Opaque Result Types and recently I had some realizations that helped me understand the proposal better. I noticed I'm not the only one confused about it, so I thought I'd write down some things that might help others to get a better grasp of the concept as well. In doing so, I'd also like to try and rebrand opaque result types, or at least the underlying mechanism, to something I'm calling 'reverse generics'. I think viewing this al…

@xAlien95

Thus, this code snippet doesn't make sense:

@ensan-hcl

> // Possible future direction with 'some' used
// for anonymous normal and reverse generic types:
func makeCollection(with number: some Numeric) -> some Collection {
  return [number]
}
> ```

You cannot use an opaque type without actually providing the underlying type (directly or by means of the context in which the function is declared). That function shouldn't be callable by design (you know that it accepts a specific conforming Numeric type, but you don't know which type).

[sighoya]

@xAlien95

As you said, unlike generics, with reverse generics the type is decided by the callee (in this case is Self , which is deduced by the protocol interface

No, in this case it is deduced by the caller of f. The type some P in argument position isn't decided by the callee, it is decided by the (nth) caller, hence we get into trouble not considering some P as existential type.

[xAlien95 (Stefano De Carolis)]

@sighoya

No, in this case it is deduced by the caller of f . The type some P in argument position isn't decided by the callee, it is decided by the (nth) caller, hence we get into trouble not considering some P as existential type.

In order to call foo you need an instance of a conforming Protocol type, in this case it's s. Protocol.foo is of type (Self) -> Void, and Self in this context is the type of the instance, i.e. (type of 's'). You cannot decide which type the argument should have, it's the method foo that knows and constrains the type of its argument to be (type of 's').

[tem (GalaxySwift)]

@xAlien95

Thus, this code snippet doesn't make sense:

@ensan-hcl

// Possible future direction with 'some' used
// for anonymous normal and reverse generic types:
func makeCollection(with number: some Numeric) -> some Collection {
  return [number]
}

You cannot use an opaque type without actually providing the underlying type (directly or by means of the context in which the function is declared). That function shouldn't be callable by design (you know that it accepts a specific conforming Numeric type, but you don't know which type).

I thought the some P in the parameter position is just a (proposed) shorthand for <T: P> after the function name and then using T in the parameter position.

It does make sense because the caller provides the missing information needed for the return type.

[ensan-hcl (ensan)]

@xAlien95

In order to call foo you need an instance of a conforming Protocol type, in this case it's s . Protocol.foo is of type (Self) -> Void , and Self in this context is the type of the instance, i.e. (type of 's') . You cannot decide which type the argument should have, it's the method foo that knows and constrains the type of its argument to be (type of 's') .

Yes. This can be confirmed if we make static member constraint. (EDIT: I added a test)

protocol Protocol {
    func foo(_ p: Self)
    static var value: Self { get }
}

struct Struct: Protocol {
    func foo(_ p: Self) {}
    static let value: Struct = Self()
}

let s: some Protocol = Struct()
let f = s.foo
// f is (some Protocol) -> Void
// f is (some Protocol (type of 's')) -> Void
// it works.
let _ = f(s)
// it also works.
let _ = f(.value)
//Error: Cannot convert value of type 'Struct' to expected argument type 'some Protocol'
let _ = f(Struct())

This is reverse generic argument type because no one except f knows the exact type of its argument but it still works.

@tem

I thought the some P in the parameter position is just a (proposed) shorthand for <T: P> after the function name and then using T in the parameter position.

It does make sense because the caller provides the missing information needed for the return type.

Also yes. If I understand correctly, current direction is making the some P shorthand of normal generics in argument type. And I'm worried about these two things, because they seems cannot work together.

[sighoya]

@xAlien95

You cannot decide which type the argument should have, it's the method foo that knows and constrains the type of its argument to be (type of 's') .

No, foo must accept any type provided, the only problem you would have is if you pass two instances of Self, you have to assure that the caller passes the same type for both some P arguments.
Anyway, the caller decides with some restrictions from the callee.

[sighoya]

@ensan-hcl

This is reverse generic argument type because no one except f knows the exact type of its argument but it still works.

No, f doesn't know the exact type.

I must admit, the caller likewise because some P is constructed from another callee in this case here.
But for the caller, some P is then the concrete type passed by the caller to the callee ;).

However, passing Struct should also be possible.
It isn't yet as shown by you:

//Error: Cannot convert value of type 'Struct' to expected argument type 'some Protocol'
let _ = f(Struct())

However, later improvements should allow for this.

[xAlien95 (Stefano De Carolis)]

@sighoya

No, foo must accept any type provided

Yes, "any" feels quite natural, doesn't it?

If you were going to really declare a function which can accept any Protocol, you would have spelled it as func foo(_ p: any Protocol) instead of func foo(_ p: some Protocol). If we stick with the IMHO simpler model that some is decided by the callee independently of the position (either argument or result position), we can obtain a model coherent with the current implementation of opaque types.

I don't think that treating func foo(_ p: some Protocol) as a sugared alternative to func foo<T: Protocol>(_ p: T) can lead to a sound type system. After all if it were, then you would also have the following as alternatives to the same function which is not true:

func foo() -> some Protocol
func foo<T: Protocol>() -> T

The difference between generics and reverse generics is in who decides the type, the caller in the first case and the callee in the second one. Opaque types are modeled after reverse generics and not generics.

---

@sighoya

However, passing Struct should also be possible.

It shouldn't. Struct may be defined in another module as private and may change among future releases of the module. If you were allowed to pass the underlying type, you would "break the contract" of opaque types.

[ensan-hcl (ensan)]

@xAlien95

If you were going to really declare a function which can accept any Protocol , you would have spelled it as func foo(_ p: any Protocol) instead of func foo(_ p: some Protocol) . If we stick with the IMHO simpler model that some is decided by the callee independently of the position (either argument or result position), we can obtain a model coherent with the current implementation of opaque types.

Here, you use any Protocol for "normal generics", right? Or do you use it for "existential type"?

[sighoya]

@xAlien95

After all if it were, then you would also have the following as alternatives to the same function which is not true:

func foo() -> some Protocol
func foo<T: Protocol>() -> T

No, the latter signature is invalid for any function determining the type itself. You have to rewrite it to:

func foo() -> (T:Protocol)

where we can clearly see it is an existential.

@xAlien95

If we stick with the IMHO simpler model that some is decided by the callee independently of the position (either argument or result position), we can obtain a model coherent with the current implementation of opaque types.

Interesting, how would you even do that?
How does the caller at all know what to pass to the callee?

@xAlien95

It shouldn't. Struct may be defined in another module as private and may change among future releases of the module.

I don't understand that. F has to accept any conforming type to Protocol.
And yes, I see the irony here.

I'm interested how some P types evolve in Swift, when you really want them callee decided everywhere, then they are different to impl Trait in Rust.

[Moximillian (Mox)]

@ensan-hcl

Also yes. If I understand correctly, current direction is making the some P shorthand of normal generics in argument type. And I'm worried about these two things, because they seems cannot work together.

My apologies, seems like I had not fully grasped the implications of normal generics vs. reverse generics in relation to using some in various locations. After reading the discussion in this thread I have now changed my post to talk about possibly using func foo(some: P) {} for reverse generics, not for normal generics.

[sighoya]

@tem

suggesting that -> some P could have been alternatively designed as -> <T: P> T but wasn't due to aesthetics).

func foo() -> <T:Protocol> T

Is not the same as:

func foo<T: Protocol>() -> T

the latter would reject callee site determination of the type parameter T. The former isn't a type parameter at all, it decays to a type, an existential type.

I think part of the confusion stems that some P is either sides seen as a dual to generics, otherwise some P is seen as type alias, which is contradictory.

[sighoya]

@Avi

By that definition, some P isn't existential. some P is shorthand, with the added benefit of hiding the real type from the caller. But that's all a compiler trick. some P is a stand-in from the perspective of the developer , never the compiler.

some P behaves like the following:

// magic modifier to hide the real type from everyone else.
private(type) typealias HiddenP = Int

public func f() -> HiddenP {}

One would not consider HiddenP to be an existential, and some P is nothing more than Swift's actual implementation of my made-up typealias modifier.

I give you right, here. We can actually consider some P in return type position as type aliases, but for the argument position, it seems not to working anymore.
The type aliases are created from other callees than the actual callee accepting the some P type, so we actually have to deal with different type aliases.

[xAlien95 (Stefano De Carolis)]

@ensan-hcl

Here, you use any Protocol for "normal generics", right? Or do you use it for "existential type"?

No, I'm using any to refer to protocol types as the Swift Programming Language Guide calls them. Generics and protocol types are identical on the call site:

func foo<T: Protocol>(_ p: T) {}
func bar(_ p: any Protocol) {}

you can pass any Protocol-conforming type to both foo and bar, but the difference lays in the possibility to specialize the former.

---

@sighoya

How does the caller at all know what to pass to the callee?

The caller doesn't know the underlying type which can be passed and that's the point, since that underlying type might be private, therefore unreachable. However, based on the protocol interface, it can be guided to the "correct" opaque type.

private struct Private: Equatable {}

let p: some Equatable = Private()
let q: some Equatable = Private()

p == q  // error: p and q may have different underlying types

The diagnostics can help: you can write p == p, q == q or even let p2 = p; p2 == p, but you cannot use == on unrelated some Equatables.

Who decides which types == (lhs: some Equatable, rhs: some Equatable) -> Bool can accept in argument position? It's the function ==, not the caller:

protocol Equatable {
  static func == (lhs: Self, rhs: Self) -> Bool
}

The declaration of == constrains both lhs and rhs to have the same type, which is the type Self of the instance (in this case the instance is p).

---

With this design in place, which I'd like to remark that it's just one of the possible designs that can extend the current status of opaque types consistently, would it make sense to allow declaration of functions having opaque types in argument position? No, it wouldn't. In the declaration of the following function

func == (lhs: some Equatable, rhs: some Equatable) -> Bool { ... }

the types of lhs and rhs cannot be inferred from the declaration context. You would need to use reverse generics explicitly if they were available

func == <^T: Equatable = Private>(lhs: T, rhs: T) -> Bool { ... }

but such a function would be almost useless. T is known to conform to Equatable, it is provided by the function but it is unrelated to any other Equatable-conforming type, so you cannot have an instance of the "correct" opaque type to pass as argument.

An example in which the function can still be called has been provided by @ensan-hcl:

@ensan-hcl

// Call the function with leading dot syntax
callee(.max)

but it would be quite unusual to provide such a function in a module for the sake of making the parameter type private. If you want the parameter type to be private, then the function declaration should most likely be private too.