Opaque result types

[Douglas_Gregor]

@jechris

I guess there is something I did not understand, but I would expect opaque typealias to be more something like:

typealias Q = opaque A where B: C

func d() -> Q {
    return ...
}

func e() -> Q {
    return d()
}

With no reference to any concrete type as it does not appear inside function signature (when not using typealias).

There is a discussion of this in the proposal. We shouldn't try to infer the underlying concrete type of Q from the bodies of both d and e; rather, we need the underlying concrete type to be specified as part of the "definition' of the type alias.

@jechris

As for opaque naming, what about anonymized ?

func makeMeACollection<T>(_: T.Type) -> anonymized Q: MutableCollection & RangeReplaceableCollection where Q.Element == T {
    return [T]()
}

I like this direction! This feature is effectively saying "there's a type here, but I'm not going to state what it is". Along the same lines but shorter: unnamed, nameless.

Doug

[beccadax (Becca Royal-Gordon)]

@Douglas_Gregor

Along the same lines but shorter: unnamed , nameless .

Somehow, “unnamed” and “nameless” sound to me like features of the underlying type, while “anonymous” sounds like features of this use of the type. Maybe that’s just me, though.

[Joe_Groff]

@Vogel

protocol A { associatedtype B } struct C { } func d() -> opaque A where B: C { return ... } //just some kind of convenience method that always ends up calling d: func e() -> opaque A where B: C { //other stuff... return d() } let f = [d(), e()] //OH NO, ERROR, NOT THE SAME TYPES (even though they actually are, but we can't express that.)

Yeah, it wouldn't be appropriate to use this feature if the fact that d and e return the same type is important. If this were expanded out to allow for opaque typealiases, you could share an alias between these two definitions. It's hard for me to see how the standard library using opaque types for collections would affect its current level composability, since the use of separate concrete wrapper types already in practice means that you'll get different types for different computations. You can't put [x.lazy.map(...), x.lazy.filter(...)] into an array either without forcing both elements to a common concrete collection type, and you could do the same if the return types of map and filter were opaque.

Although it's true that the standard library's primitive transformations would still need to implement their wrappers, this feature would allow those to be implementation details that can change without affecting source compatibility. Furthermore, this feature is useful for functions that build on top of those primitives, since it both saves you from having to write a long composed return type, and allows you to change the implementation to something with a different return type without breaking your clients. Right now, if you change:

func foo(x: [Int]) -> LazyMapCollection<[Int], String> {
  return x.lazy.map { ... }
}

to:

func foo(x: [Int]) -> LazyFilterCollection<LazyMapCollection<[Int], String>> {
  return x.lazy.map { ... }.filter { ... }
}

that's an ABI- and possibly source-breaking change. If the return type is opaque, you're guaranteed to be able to change the lazy implementation of foo without breaking binary or source.

[Nevin]

@Nevin

I think it would read best like this:

opaque typealias Q: (A where B: C) = MyConcreteType

…or perhaps:

opaque typealias Q = MyConcreteType: A where B: C

or

typealias Q = opaque MyConcreteType: A where B: C

[Joe_Groff]

I've updated the prototype toolchain to use the latest master branch: [WIP] Opaque result types prototype by jckarter · Pull Request #21137 · apple/swift · GitHub There isn't any additional opaque type specific functionality from the previous toolchain, it's just built against a newer compiler.

[benrimmington]

Could you repurpose the @unknown attribute?

protocol Shape { ... }
func translucentRectangle() -> @unknown Shape { ... }

[Jumhyn (Frederick Kellison-Linn)]

To me, unnamed or nameless would be confusing in the contexts were you have actually given the type a name, e.g. in a typealias. Maybe it's just because the feature was introduced under the title "Opaque result types," but it seems like everyone is pretty comfortable discussing these as "opaque types", and that there's no urge to say, we're really discussing "anonymous types" (which to me, invoke the truly "anonymous" classes of Java, or structs/unions in C). The types returned as opaque results are declared elsewhere, with names, but we are blocking the true type name from view. I still believe opaque is the best terminology.

I also think that the syntax typealias usage and the inline usage should be identical, or at least as consistent with one another as possible (unless we go the route of disallowing inline definition). The issue I have with the following from @Nevin:

opaque typealias Q = MyConcreteType: A where B: C
typealias Q = opaque MyConcreteType: A where B: C

is that they both appear to be constraining the concrete type, which doesn't make a whole lot of sense. It's not wrong, since MyConcreteType indeed satisfy those constraints, but it fails to adequately express that these are actually constraints on our opaque type Q.

@Joe_Groff

Yeah, it wouldn't be appropriate to use this feature if the fact that d and e return the same type is important.

Have we decided to definitively disallow this? It seems like it would be useful to be able to express something like this:

func makeMeACollection<T>(_: T.Type) -> opaque Q: MutableCollection & RangeReplaceableCollection where Q.Element == T, Q: Equatable {
    return [T]()
}

func makeMeACollection<T>(containing element: T) -> makeMeACollection<T>(_:).Q {
    var collection = [T]()
    collection.append(element)
    return collection
}

var collection1 = makeMeACollection(Int.self)
collection1.append(1)
let collection2=. makeMeACollection(containing: 1)
collection1 == collection2 // true

[Joe_Groff]

Personally, it would make more sense to me to name the opaque type with a separate declaration than to invent a way to refer to another declaration's opaque return type once you need the same type in multiple places.

[Jumhyn (Frederick Kellison-Linn)]

Would we disallow clients from implementing wrappers which return the same opaque type as methods in another module, then?

[Joe_Groff]

It would be interesting to me to see how often the desire to do that comes up in practice. Rust has had impl Trait for a while now; does anyone using Rust have any experience with whether being unable to name the impl-ed type has been a problem for them?

[Jumhyn (Frederick Kellison-Linn)]

Without it, what would be the proper way to return the result of a method with an opaque result type? Would it be:

func makeCollectionAndAppend<T>(_ element: T) -> opaque Q: MutableCollection & RangeReplaceableCollection where Q.Element == T {
    var collection = makeMeACollection(T.self)
    collection.append(element)
    return collection
}

[Joe_Groff]

@xwu

I do not see any reason to try to be clever in the choice of keyword. If this particular feature will be described as an "opaque type," then it is appropriate to use the keyword opaque or opaquetype , just as we use the keyword typealias for type aliases, associatedtype for associated types, and class for classes--without worrying that a user would be confused that class Room models a classroom.

I think it's arguable whether "opaque result type" is the best name for this feature in general, regardless of how it's spelled in the language. The word "opaque" is already pretty heavily overloaded in the compiler and implementation model. That largely doesn't leak out into the user model unless you go peeking into the implementation or try to understand the optimizer, but it has given me pause while implementing this feature as to whether there's a clearer name for what it's trying to accomplish.

[Joe_Groff]

Yeah, that will work, though the function's own opaque type will still be considered as distinct from the callee's.

[terhechte (Benedikt Terhechte)]

I do some Rust on the side and I never felt that this was a problem.

[John_McCall]

"Abstract" is perhaps a better name except for the potential confusion with abstract classes and methods (and who knows, maybe we'll add those some day to Swift; but even if we don't, we can't deny the potential for confusion).

[michelf (Michel Fortin)]

@Joe_Groff

Yeah, it wouldn't be appropriate to use this feature if the fact that d and e return the same type is important.

I'm not sure there is a situation where it is not important in one way or another. Although that depends on what you mean by "important".

Would you be able to write convenience functions like this if lazy map was returning an opaque type?

func lazyFirstNames() -> SomethingLazySomething {
     return persons.lazy.map { $0.firstName }
}
func lazyLastNames() -> SomethingLazySomething {
    return persons.lazy.map { $0.lastName }
}

What return type do you use if you want the result of the two convenience functions to be interchangeable?

let lazyNames = somePreference ? lazyFirstNames() : lazyLastNames()

If lazy map was using an opaque typealias as its return type, you could refer to it by name and reuse it as the return type for both of your convenience functions. I'm a bit worried that you can't do this with the current proposal since each convenience function would have to redefine its own semantically different opaque type.

[Joe_Groff]

@michelf

Would you be able to write convenience functions like this if lazy map was returning an opaque type?

func lazyFirstNames() -> SomethingLazySomething {
     return persons.lazy.map { $0.firstName }
}
func lazyLastNames() -> SomethingLazySomething {
    return persons.lazy.map { $0.lastName }
}

Yeah, by having the functions also

@michelf

What return type do you use if you want the result of the two convenience functions to be interchangeable?

let lazyNames = somePreference ? lazyFirstNames() : lazyLastNames()

If lazy map was using an opaque typealias as its return type, you could refer to it by name and reuse it as the return type for both of your convenience functions. I'm a bit worried that you can't do this with the current proposal since each convenience function would have to redefine its own semantically different opaque type.

That would only be possible to begin with if lazyFirstNames and lazyLastNames had the exact same chain of lazy transformations. Once the types on the branches of the conditional diverge, you lose most of the efficiency benefit of generic specialization, and so you'd probably want either force both branches to Array or sink the conditional into the chain of transformations somehow.

[Jumhyn (Frederick Kellison-Linn)]

Doesn't that essentially defeat the fact that an opaque result type "saves you from having to write a long composed return type"? If the opaque result is truly something simple like opaque Equatable then you save a lot of effort, but even in the initial proposal we saw monster signatures like:

public func reversed() -> opaque BidirectionalCollection
        where _.Element == Element
        where Self: RandomAccessCollection -> _: RandomAccessCollection
        where Self: MutableCollection -> _: MutableCollection {      
    return ReversedCollection<Self>(...)
}

Which would only expand as more conditional conformances were added. With the amount of conditional conformance in the standard library, I'm skeptical that this would make the long composed return types any simpler without having the ability to say "this method just returns the opaque result of another method.

[Joe_Groff]

You don't have to replicate that entire signature, only the part that's interesting to your specific API. The standard library has to be generic enough to handle all possible different clients, but most of those parameters are likely to be concretized in your specific use of the API.

[michelf (Michel Fortin)]

@Joe_Groff

That would only be possible to begin with if lazyFirstNames and lazyLastNames had the exact same chain of lazy transformations.

Maybe the one who designed the lazy filter API would think so, or maybe it was just simpler to write it as an opaque return type without really trying to take this downside into consideration.

I do like the concept of opaque return types, but I also don't think having a couple of convenience accessors using the same filter function with a different closure is that far fetched. So for cases like this, perhaps there should be a way to write a signature this way:

func lazyFirstNames() -> #returnType(of: persons.lazy.map)
func lazyLastNames() -> #returnType(of: persons.lazy.map)

I guess that'd have to be a different proposal.