Opaque result types

IMHO, the issue I have with opaque types is that the concrete type is inferred from a call site. As a result, would this not be a heterogenous array?

func foo() -> opaque Collection & Equatable { return "foo" }
func bar() -> opaque Collection & Equatable { return "bar" }

let foobar = [foo(), bar()]

Would I be unable to do something like this?

if foo() != bar() { ... }

If these aren't possible, then uses like this seem to undermine the usefulness of opaque types in a (hopeful) future with generalized existentials.

Opaque types do not eliminate all use cases for existentials. In your examples, foo() and bar() do indeed have different types from the type system's perspective, and you'd need an existential type or type-erasing container to be able to work with both as one type.

Or use an opaque type alias to declare that both foo() and bar() return the same opaque type as discussed in some previous posts:

typealias Foo = opaque Collection & Equatable as SomeConcreteType

func foo() -> opaque Foo { ... }
func bar() -> opaque Foo { ... }

let foobar = [foo(), bar()] // homogeneous array containing elements of SomeConcreteType

if foo() != bar() { ... } // valid code
1 Like

How about a generic-parameter-list-like syntax after the arrow?

func foo() -> <T : Collection> T where T.Element == String
4 Likes

We want to avoid confusion with generics, since opened existentials and opaque types aren't generic parameters. The name is merely to provide a readable way to express constraints; something that a generic parameter list syntax doesn't really help in understanding.

This motivation is actually the most exciting part of the proposal from my point of view. Extending your silly example just slightly:

func foo<C: Collection>(_ c: C) -> LazyCollection<ReversedCollection<LazyFilterCollection<LazyMapCollection<C, ()>>>> {
  return c.lazy.map{ String(describing: $0) }.filter { Bool.random() }.reversed()
}

Look at that return type! I don't want to have to spell that anywhere. When I'm prototyping in code (and even more often in playgrounds) I often end up wrapping my functional-style map/filter chains in return Array(...) so that this sort of thing can be spelled foo<E: Element>(_ c: [E]) -> [E] instead, even though I'm now losing all of the standard library lazy smarts and all the Collection genericness at all of my internal API boundaries. Maybe once I've got everything prototyped I go back and do it the right way.

So this is a plea to try to make some lightweight version of the proposed syntax work for in-same-resilience-domain APIs where the compiler knows the concrete type already, so the declaration can be spelled something like func foo<C: Collection>(_ c: C) -> opaque Collection {}.

1 Like

In discussions about generalized existentials, my hope has been that rather than have explicit "opening" operations to use dynamic types, we could use "path dependent types" to refer to associated types relative to an existential:

let a: Collection
let b: a.Index = a.startIndex
let c: a.Element = a[b]

which is nice because it'd make operations relative to a generalized existential "just work" as expected without the mental overhead of having to think about a new opened type. Applying this idea to declarations, it might be interesting to allow constraints to be applied directly to arguments in a similar way. If return values could also be named, you could write things like this:

extension Collection {
  func concat(_ other: Collection) -> returned: Collection
    where Element == other.Element, Element == returned.Element

  func map(_ transform: (Element) -> returned.Element) -> returned: Collection
}

which feels nice and expressive to me, avoids the "angle bracket blindness" of explicit generic arguments, and could be made to do what you mean in both contravariant positions (where you want type parameters) and covariant positions (where you want opaque types). This would all be nicer if we hadn't already used protocols in type positions for existential types, although at least for non-inout arguments, taking an existential and taking an argument whose type is a unique generic type argument are equivalent.

7 Likes

Naming the return value is an interesting idea. But you can't use variable names in constraints. It's the type you need to name.

Maybe this would work better:

extension Collection {
  func concat<Other: Collection>(_ other: Other) -> opaque Result: Collection
    where Element == Other.Element, Element == Result.Element

  func map(_ transform: (Element) -> Result.Element) -> opaque Result: Collection
}
1 Like

I agree, and that is why I put it after the arrow. While this can be confused with generic parameter lists, this is just a syntax with familiarity.

Yes, I understand that. I don't know about you but a generic parameter list syntax does help me understand that the return type is bound and concrete. When <T : Collection> is placed after the arrow, it suggests that the callee decides what the concrete type is.

where also goes after the arrow for Other.
Where do we put the where when using opaque return types in that case?

I like that, but variable a being a value feels weird. What if a has an instance member that's called Index? Maybe we can use "metatype dependent types"?

let a: Collection
let b: type(of: a).Index = a.startIndex
let c: type(of: a).Element = a[b]

Fair enough. The difference is that I additionally take the fact that we can't have more than one return type as an allusion to omit that syntax and simply state the parameter right away. Besides, we need a single mechanism for naming generalized existentials and opaque types both as return types and type aliases.

IIRC nested types are treated as both instance and static members, and you aren't allowed to "overload" a type name with another instance or static member. (Maybe that's changed since I last touched name lookup.)

Being able to use values in type positions is what I was proposing. Every immutable value has a fixed type; it's not unreasonable to consider allowing that type to be named indirectly through a value with the type.

1 Like

According to the proposal, an opaque type can be part of the return type (so it doesn't have to be the return type). Here are two questions:

  1. Can the opaque-keyword syntax express a function returning a 2-tuple of the same opaque type?

    // It is unclear whether the first tuple element type is equal to the second.
    func foo() -> (opaque Collection, opaque Collection) where ...
    
    // The following is clear.
    func foo() -> <T : Collection> (T, T) where ...
    
  2. Are multiple opaque return types allowed in a function? How can I add constraints on them?

    // Not sure how to add constraints using the `_` syntax.
    func foo() -> (opaque Collection, opaque Sequence) where ...
    
    // Adding constraints is possible in this syntax:
    func foo() -> <T : Collection, U : Sequence> (T, U) where T : ..., U : ...
    
1 Like

We need the opaque keyword to differentiate from opened existentials, so with your syntax it would be

func foo() -> <opaque T : Collection> (T, T) where ...

func foo() -> <opaque T : Collection, opaque U : Sequence> (T, U) where T : ..., U : ...

P.S. Were the questions addressed to me?

It would not make sense to use the generic parameter list syntax on existentials because they are not concrete. The syntax I proposed would be used exclusively for opaque result types. So there's no need to differentiate between opaque and existential using an opaque keyword.

My questions are related to this:

... because the proposal did not indicate that we can't have more than one opaque return type.

1 Like

This compiles:

struct T {
  typealias Index = Int
  var Index = 0
}

Both syntax variants work well with compound opaque types etc., the only difference is the parameter list which makes it slightly more verbose. But I must say I'm not a fan of drawing such a thick line between opaque types and opened existentials in terms of syntax. We only need those names for convenience rather than semantics after all; I think a single attribute carries out the semantic part much better in this case.

Just in case, this was the syntax I was referring to.

I agree that opaque U: Collection is cleaner. But the tricky case is when U is used multiple times in the return type.

When an opaque type is used multiple times, where would you specify the conformance? On the first occurrence?

func foo<T>() -> (opaque U: Collection, opaque U)

Maybe in this case the conformance should only be specified using a where clause.

func foo<T>() -> (opaque U, opaque U) where U: Collection
1 Like

Let's build some silly example with a tuple:

func foo<T>(_ arg: T) -> (opaque U, opaque Q) where // Or maybe opaque (U, Q) ...
                         U: Collection,
                         U.Element == Q,
                         T: MutableCollection,
                         T.Iterator == U.Iterator,
                         U: RandomAccessCollection when
                             T.Element: RandomAccessCollection { ... }

It's just as listing normal constraints, you can arrange them as you like. Conformances better be allowed only in the where clause after the return type with this syntax variant. In your case, the best way of expressing the same thing would be

func foo<T>(_ arg: T)<U: Collection, Q> -> (U, Q) where // U: Collection can go here as well
                         U.Element == Q,
                         T: MutableCollection,
                         T.Iterator == U.Iterator,
                         U: RandomAccessCollection when
                             T.Element: RandomAccessCollection { ... }

Quite similar, as you see. But this is my main concern:

2 Likes