Opaque result types

(Richard Wei) #121

How about a generic-parameter-list-like syntax after the arrow?

func foo() -> <T : Collection> T where T.Element == String

(Anthony Latsis) #122

We want to avoid confusion with generics, since opened existentials and opaque types aren't generic parameters. The name is merely to provide a readable way to express constraints; something that a generic parameter list syntax doesn't really help in understanding.

(Greg Titus) #123

This motivation is actually the most exciting part of the proposal from my point of view. Extending your silly example just slightly:

func foo<C: Collection>(_ c: C) -> LazyCollection<ReversedCollection<LazyFilterCollection<LazyMapCollection<C, ()>>>> {
  return c.lazy.map{ String(describing: $0) }.filter { Bool.random() }.reversed()

Look at that return type! I don't want to have to spell that anywhere. When I'm prototyping in code (and even more often in playgrounds) I often end up wrapping my functional-style map/filter chains in return Array(...) so that this sort of thing can be spelled foo<E: Element>(_ c: [E]) -> [E] instead, even though I'm now losing all of the standard library lazy smarts and all the Collection genericness at all of my internal API boundaries. Maybe once I've got everything prototyped I go back and do it the right way.

So this is a plea to try to make some lightweight version of the proposed syntax work for in-same-resilience-domain APIs where the compiler knows the concrete type already, so the declaration can be spelled something like func foo<C: Collection>(_ c: C) -> opaque Collection {}.

(Joe Groff) #124

In discussions about generalized existentials, my hope has been that rather than have explicit "opening" operations to use dynamic types, we could use "path dependent types" to refer to associated types relative to an existential:

let a: Collection
let b: a.Index = a.startIndex
let c: a.Element = a[b]

which is nice because it'd make operations relative to a generalized existential "just work" as expected without the mental overhead of having to think about a new opened type. Applying this idea to declarations, it might be interesting to allow constraints to be applied directly to arguments in a similar way. If return values could also be named, you could write things like this:

extension Collection {
  func concat(_ other: Collection) -> returned: Collection
    where Element == other.Element, Element == returned.Element

  func map(_ transform: (Element) -> returned.Element) -> returned: Collection

which feels nice and expressive to me, avoids the "angle bracket blindness" of explicit generic arguments, and could be made to do what you mean in both contravariant positions (where you want type parameters) and covariant positions (where you want opaque types). This would all be nicer if we hadn't already used protocols in type positions for existential types, although at least for non-inout arguments, taking an existential and taking an argument whose type is a unique generic type argument are equivalent.

(Michel Fortin) #125

Naming the return value is an interesting idea. But you can't use variable names in constraints. It's the type you need to name.

Maybe this would work better:

extension Collection {
  func concat<Other: Collection>(_ other: Other) -> opaque Result: Collection
    where Element == Other.Element, Element == Result.Element

  func map(_ transform: (Element) -> Result.Element) -> opaque Result: Collection

(Richard Wei) #126

I agree, and that is why I put it after the arrow. While this can be confused with generic parameter lists, this is just a syntax with familiarity.

Yes, I understand that. I don't know about you but a generic parameter list syntax does help me understand that the return type is bound and concrete. When <T : Collection> is placed after the arrow, it suggests that the callee decides what the concrete type is.

(Chéyo Jiménez) #127

where also goes after the arrow for Other.
Where do we put the where when using opaque return types in that case?

(Richard Wei) #128

I like that, but variable a being a value feels weird. What if a has an instance member that's called Index? Maybe we can use "metatype dependent types"?

let a: Collection
let b: type(of: a).Index = a.startIndex
let c: type(of: a).Element = a[b]

(Anthony Latsis) #129

Fair enough. The difference is that I additionally take the fact that we can't have more than one return type as an allusion to omit that syntax and simply state the parameter right away. Besides, we need a single mechanism for naming generalized existentials and opaque types both as return types and type aliases.

(Joe Groff) #130

IIRC nested types are treated as both instance and static members, and you aren't allowed to "overload" a type name with another instance or static member. (Maybe that's changed since I last touched name lookup.)

Being able to use values in type positions is what I was proposing. Every immutable value has a fixed type; it's not unreasonable to consider allowing that type to be named indirectly through a value with the type.

(Richard Wei) #131

According to the proposal, an opaque type can be part of the return type (so it doesn't have to be the return type). Here are two questions:

  1. Can the opaque-keyword syntax express a function returning a 2-tuple of the same opaque type?

    // It is unclear whether the first tuple element type is equal to the second.
    func foo() -> (opaque Collection, opaque Collection) where ...
    // The following is clear.
    func foo() -> <T : Collection> (T, T) where ...
  2. Are multiple opaque return types allowed in a function? How can I add constraints on them?

    // Not sure how to add constraints using the `_` syntax.
    func foo() -> (opaque Collection, opaque Sequence) where ...
    // Adding constraints is possible in this syntax:
    func foo() -> <T : Collection, U : Sequence> (T, U) where T : ..., U : ...

(Anthony Latsis) #132

We need the opaque keyword to differentiate from opened existentials, so with your syntax it would be

func foo() -> <opaque T : Collection> (T, T) where ...

func foo() -> <opaque T : Collection, opaque U : Sequence> (T, U) where T : ..., U : ...

P.S. Were the questions addressed to me?

(Richard Wei) #133

It would not make sense to use the generic parameter list syntax on existentials because they are not concrete. The syntax I proposed would be used exclusively for opaque result types. So there's no need to differentiate between opaque and existential using an opaque keyword.

My questions are related to this:

... because the proposal did not indicate that we can't have more than one opaque return type.

(Richard Wei) #134

This compiles:

struct T {
  typealias Index = Int
  var Index = 0

(Anthony Latsis) #135

Both syntax variants work well with compound opaque types etc., the only difference is the parameter list which makes it slightly more verbose. But I must say I'm not a fan of drawing such a thick line between opaque types and opened existentials in terms of syntax. We only need those names for convenience rather than semantics after all; I think a single attribute carries out the semantic part much better in this case.

Just in case, this was the syntax I was referring to.

(Richard Wei) #136

I agree that opaque U: Collection is cleaner. But the tricky case is when U is used multiple times in the return type.

When an opaque type is used multiple times, where would you specify the conformance? On the first occurrence?

func foo<T>() -> (opaque U: Collection, opaque U)

Maybe in this case the conformance should only be specified using a where clause.

func foo<T>() -> (opaque U, opaque U) where U: Collection

(Anthony Latsis) #137

Let's build some silly example with a tuple:

func foo<T>(_ arg: T) -> (opaque U, opaque Q) where // Or maybe opaque (U, Q) ...
                         U: Collection,
                         U.Element == Q,
                         T: MutableCollection,
                         T.Iterator == U.Iterator,
                         U: RandomAccessCollection when
                             T.Element: RandomAccessCollection { ... }

It's just as listing normal constraints, you can arrange them as you like. Conformances better be allowed only in the where clause after the return type with this syntax variant. In your case, the best way of expressing the same thing would be

func foo<T>(_ arg: T)<U: Collection, Q> -> (U, Q) where // U: Collection can go here as well
                         U.Element == Q,
                         T: MutableCollection,
                         T.Iterator == U.Iterator,
                         U: RandomAccessCollection when
                             T.Element: RandomAccessCollection { ... }

Quite similar, as you see. But this is my main concern:


I've seen this said a few times but I haven't been fully convinced that this is true yet. You can already use _ as a variable name and place constraints on its type

let _: String = "test"

and I saw the use of the _ in opaque types in a similar way, i.e. there is some specific return type but you can't refer to it by name elsewhere. Though, having said that, the difference here is that the constraints themselves can't be written in “point-free style”, so they can't avoid referring to the type in some way.

Perhaps this conflict makes it almost inevitable that the opaque type will need to be named in some way, and the syntax should just be designed from the start to require a name. This solves some other issues too, like allowing multiple opaque types in the same declaration. It doesn't have to be verbose if you give the opaque type a single character name, e.g.

-> opaque O where O: Collection, O.Element == Int

where I've put all the constraints in the where clause. Perhaps there could be a convention to call a single opaque type “O” in the absence of a better contextual name, so it would stand out from generic types (which are conventionally T, U, etc).

(Zachary Waldowski) #139

Ok, sure, thanks! I did a closer reading now, I apparently didn't see the link to the generalized existentials draft. The section you link to comes across as a point in favor of opaque types ("Ah, I see! Static is better!") until chewing on that other draft; this pitch delegates a lot of the understanding of the problem space to that other draft.

So, now, I kinda understand after reading both that opaque types won't allow getting rid of AnyCollection<T>, but will instead allow me to avoid using it… that's preferable? Why? That distinction needs a smidge of ELI5, sorry if that's not helpful.

The posts upthread say a few times that the explicit annotation is necessary to differentiate when we get generalized existentials that will use approximately the same syntax… why does this distinction matter to me? Swift doesn't ask me to specify how it packs the bits in an enum with associated values; it tries to do the right thing. If I write an expression that looks like a generalized existential but can statically be determined to fit within the bounds of an opaque type, it should interpret it as such, IMHO. Or vice versa if the optimization falls the other way.

(Adrian Zubarev) #140

It did, but for now it would be just a restriction which we can lift later. The main issue here is that we need an unambiguous way referencing a specific opaque type that we want to constrain. For example I used the closure like $ syntax above, but as Doug already pointed out correctly it will be ambiguous when used within a closure with anonymous parameters.

func f() -> opaque Collection { ... }
func g() -> opaque Collection where $0.Element : Equatable { ... }

// If we can already do this then the last example is a natural consequence.
let tuple1 = (f(), g())

// Only the elements of the second opaque collection required to conform to `Equatable`.
// Without further constraints the elements of both collections are considered different.
func h() -> (opaque Collection, opaque Collection) where $1.Element : Equatable {

let tuple2 = h()