Opaque result types

(Greg Titus) #181

On another forum I was discussing ways to try to clean up long where clauses, and came upon this sort of construction, which works great in Xcode 10 betas (though not in Swift 4.1):

extension Collection {
    public typealias OtherCollection<C: Collection> = C where C.Element == Self.Element

extension Collection where Element : Hashable {
    func sameElements<T>(with: OtherCollection<T>) -> Bool {
        return Set(self) == Set(with)

That is, we can avoid repeating common sets of requirements by offloading them onto a named generic typealias and the resulting code looks pretty nice.

Personally, I would love to see the stdlib interface someday look like:

extension Collection {
    public typealias OtherCollection<C: Collection> = C where C.Element == Self.Element

    var lazy: opaque OtherCollection<_>
    func filter((Element) -> Bool) -> opaque OtherCollection<_>
    func reversed() -> opaque OtherCollection<_>
    /* etc. */

That is, a "collection with the same element type as this collection" is a very common thing to want to name and refer to, and I'd love to be able to do so with a concise and common shared spelling like this one.

However! This use of type aliases to offload requirement clauses would be broken by the proposed opaque type alias future direction in this proposal, because it would require all of those OtherCollections to be the same concrete type, which they obviously are not.

So I just wanted to throw that out as a possible bump in the road with regard to that section of the proposal.

(Matthew Johnson) #182

I don't think there is a conflict here. You have reversed() -> opaque OtherCollection<_>. In this context, the _ is the placeholder for the opaque type. That opaque type is passed to the generic OtherCollection typealias. I would read this syntax as meaning that the opaque type must meet requirements of that non-opaque typealias, allowing you to use this syntax optimization to factor out constraints on an opaque result type.

(Dante Broggi) #183

So far, of the proposed syntaxes for opaque, concrete values I have seen on this thread, the one I like the most is:

typealias MyCollection<T> = [T] as MutableCollection & RangeReplaceableCollection where Self.Element == T
func makeMeACollection<T>(_: T.Type) -> MyCollection<T> {
   return [T]()   // okay: `MyCollection<T>` is `Array<T>`

// or for a simpler function:
func makeMeANumeric() -> Int as Numeric {
   return 5

(Greg Titus) #184

Ah! You're right. Okay, so while the difference might be subtle to coders who come upon it, we'll be able to express both these things.

(Dave DeLong) #185

I've been mildly following this proposal, given that I'm extremely interested in type erasure and this sort of implementation detail hiding.

A couple of thoughts:

  1. I do not understand why additional keywords are required. I already find the existing where syntax for generic constraints quite tedious and distracting, and the thought of extending it with more keywords and longer clauses makes me despair. I will be exceptionally opposed to any proposal that adds more keywords. Keywords mean "compiler magic", and I'm of the opinion that, as much as possible, we should be putting this sort of functionality into the standard library, and not in the compiler.

  2. I would like to see constraints expressed as part of the type itself, using existing syntax. @gregtitus already mentioned an idea I put forward in a slack community, but there's even more you can do here. For example, something like this would avoid the creation of additional keywords without losing expressivity:

    func returnSomeInts() -> Collection<Element = Int, Index = Int>

    This would be a function that returns some sort of Int-indexable collection of Ints. There is no need for where clauses; there is no need for $0.Element noise. All of the constraints using existing syntax of AssociatedType = Blah or AssociatedType: SomeProtocol, which is instantly recognizable to anyone already familiar with generics.

    For a protocol with a single associated type, the compiler could infer that the generic parameter applies to the single associated type:

    func returnSomeStuff() -> ProtocolWithSingleAssociatedType<Stuff>

    This approach also would provide for nice typealiases, like so:

    typealias AnyArray<T> = RandomAccessCollection<Element: T, Index = Int> & MutableCollection<Element: T, Index = Int> & ...

    These typealiases, like @gregtitus mentioned, can be namespaced within an existing type for terseness in method signatures, or it could be top-level typealiases for use anywhere.

(Douglas Gregor) #186

Opaque result types are a new concept, so I'd prefer to identify them with a new keyword rather than (say) reinterpret existing syntax to have the semantics presented. Their closest analogue is existential types, so we could try to make the keyword describe the key differences between opaque result types and existentials. concrete and specific are meant to do that.

This doesn't avoid the need for a new keyword, because that Collection<...> type would likely still be treated as existential. What you propose could be an alternate syntax for both opaque result types and generalized existentials, but it's only an alternative to the proposed where clauses if it can subsume all uses of where clauses.


(Joe Groff) #187

I think having something like Rust's Trait<AssocType = T> associated type constraint sugar would be great. If we allowed both same-type and protocol constraint bindings inside angle brackets, then it seems to me like it should be possible to express all constraints relative to the opaque result type without resorting to the where clause; you can express protocol constraints on the opaque type itself as a protocol composition, then require all its associated type constraints to use the new sugar to be expressed under angle brackets:

func map<U>(_ f: (Element) -> U) -> opaque Collection<Element == U>

I believe Rust's impl Trait feature doesn't allow you to express constraints on the unnamed type independent of the trait constraint either. If we go with the "multiple -> arrows" syntax for conditional conformances, this also makes the notation a bit more manageable IMO, since the conditional constraints are detangled from the implied constraints on the output type:

func reversed() -> opaque BidirectionalCollection<Element == Element>
  -> opaque RandomAccessCollection<Element == Element> where Self: RandomAccessCollection
  -> /* etc. */

I guess writing Foo == Foo is a bit weird (taking the LHS inside brackets to always refer to an associated type like in Rust), but Rust users don't seem to mind?

edit: Ah, the one thing you still wouldn't be able to spell without mentioning the opaque type is a same-type constraint between two associated types of the opaque type.

(Matthew Johnson) #188

How would this syntax be distinguished from functions that return functions? Would it be partly by the lack of parentheses on the return type? I generally like the direction but am not entirely sure how we avoid confusion between the two (given the syntactic similarity).

(Jon Hull) #189

I would like those too, but I think the main point I am trying to get across is that rather than having syntax which always infers the internal type:

keyword TypeAsSeen

We should have:

Type keyword TypeAsSeen

and if we really want to be able to infer the internal type, we would use:

_  keyword TypeAsSeen

I have proposed as for the keyword so we don't have to burn a new one (and because it is short):

[Int] as C:Collection where C.Element == Int

or using sugar as others proposed:

[Int] as Collection<Element == Int>

or if you want to infer the type:

_ as Collection<Element == Int>

I know it is an extra character (+ space) from your proposal, but this has a lot of advantages:

  • We can explicitly define the internal type when needed/desired
  • The same syntax can be used for opaque type aliases (i.e. you only have to learn 1 syntax)
  • The same syntax can be used for properties (i.e. you only have to learn 1 syntax)
  • The same syntax can be used for etc...
  • All of the information about the type can be found in one spot (you don't have to search the func)
  • It is more concrete to explain to people
  • It doesn't use _.Element which I find a very troubling syntax

Scala like placeholder for types
(Jordan Rose) #190

We really do care about inferring the type. Even if it's written explicitly in source sometimes, it has to be hidden in the generated interface view of a module. That means we need some syntax that doesn't require spelling the underlying type.

(I personally think it's a good idea anyway. Swift usually makes you write types when you're committing to something, but you're not committing to anything here unless you make the function inlinable.)

(Jon Hull) #191

@jrose: Would something like:

_ as Collection where ...

work for that, or is there an issue I am missing?

Edit: Also, would we eventually like to be able to infer non-opaque return types in some cases too?

(Jon Hull) #192

As an example of my point about being able to use the same syntax everywhere, the proposal has the following syntax for opaque typealias:

public typealias LazyCompactMapCollection<Elements, ElementOfResult>:
  opaque Collection where _.Element == ElementOfResult
    = LazyMapSequence<
             LazyMapSequence<Elements, ElementOfResult?>

Notice you have this weird _.Element == ElementOfResult = LazyMapSequence... bit where you have == and = together. Also notice that we have invented a new form of type alias which uses a colon to define the opaque part.

Now look at this:

public typealias LazyCompactMapCollection<Elements, ElementOfResult> = 
      LazyMapSequence<LazyFilterSequence<LazyMapSequence<Elements, ElementOfResult?>>,ElementOfResult> 
      as C:Collection where C.Element == ElementOfResult

So the types are still crazy, but what we have is a standard typealias (no new syntax) and our standard syntax for an opaque type. We were able to compose this naturally without having to create additional syntax for each area the concept is applied.

(Jordan Rose) #193

Sure, I just think that's weirder to see in a return type than opaque Collection where .... Remember that these things are going to be read a lot more than they're written.

(I don't like any of the syntaxes presented so far, even Doug's. My #1 problem is around the syntax for conditional conformances, since those really want to use where again.)

(Jordan Rose) #194

Maybe for private and local things, but not otherwise. It's important not to have to look at a function body across file boundaries when doing a debug build. (A whole-module-optimized build will certainly be able to look through opaque result types for functions declared in the same module.)

(Thomas Roughton) #197

As an alternative syntax: how about something like:

extension BidirectionalCollection {
  public func reversed() -> opaque 
                       : BidirectionalCollection where .Element = Element,
                       : RandomAccessCollection where Self: RandomAccessCollection,
                       : MutableCollection where Self: MutableCollection {
    return ReversedCollection<Self>(...)

func makeMeACollection<T>(_: T.Type) -> opaque 
                : MutableCollection & RangeReplaceableCollection 
                  where  .Element = T {
   return [T]()

The idea is that the return type is just opaque, and it's assumed to be of Any type. That can be refined with a list of conditional or unconditional conformances; all are specified using : Type, and there can also be a where clause containing conformances (where Self : RandomAccessCollection), same-type requirements (where Self.Element == String), and associated types (using .AssociatedType = ConcreteType).

I also toyed with the idea of having an if clause for conditional conformances, so you could do something like:

extension BidirectionalCollection {
  public func reversed() -> opaque 
                       : BidirectionalCollection where .Element = Element,
                       : RandomAccessCollection where .Element = Element if Self: RandomAccessCollection,
                       : MutableCollection where .Element = Element if Self: MutableCollection {
    return ReversedCollection<Self>(...)

although that gets a little confusing since it's not clear when reading whether the 'if' refers to the associated types (e.g. that opaque.Element == Self.Element only when Self : RandomAccessCollection) or the conformance (e.g. that opaque : RandomAccessCollection only when Self : RandomAccessCollection).

As another option for keywords: you could replace the where before the associated types with whose and then use == instead of =, saving where exclusively for conditional conformances, although you still run into the issue where it's not clear when reading what the condition applies to (even though it would be clear in the grammar of the language).

Note that I've used .Element = Element in a few places; you could spell that more explicitly as .Element = Self.Element.


This would break composabillity, because:

protocol A {
    associatedtype B

struct C { }

func d() -> opaque A where B: C {
    return ...

//just some kind of convenience method that always ends up calling d:
func e() -> opaque A where B: C {
    //other stuff...

    return d()

let f = [d(), e()] //OH NO, ERROR, NOT THE SAME TYPES (even though they actually are, but we can't express that.)

(Jon Hull) #199

I can agree with you on that. I think opaque P does read nicer in that exact use case, but it quickly starts to need more info in more complex contexts (forcing weird things like _.Element and a new form of typealias).

I guess my main concern is that this be a modular concept which can be applied wherever it makes sense without having to alter it's syntax or the syntax of what it is interacting with. I would also want the ability to be explicit about the type in the original use-case when I want to be.

One note: Just because the type shows when written in this form: Int as Comparable, doesn't mean that it is visible to anything external. The world sees it as this Opaque Comparable thing. It is only inside the function that we see it as Int.

Crazy idea! What if we were to spell it:

let a:opaque P = Int as P

That is, the external signature (and thunk if needed) is spelled 'opaque P', but 'Type as P' is how you create such things explicitly. You can't create 'opaque P' directly (except using as) because it doesn't contain the necessary info, but you can infer it from somewhere that does have that info. Thus foo()-> opaque P is you asking for it to infer, but you could still do foo()-> Int as P explicitly wherever you needed/wanted to.

I'll keep thinking about Conditional Conformances.

To be clear, I really like the idea of Opaque types... I just want us to find a compossible/reusable syntax for it that will allow us to unlock it's full potential over time.

(Dante Broggi) #200

If we have a functions like:

typealias MyCollection<T> = [T] as MutableCollection & RangeReplaceableCollection where Self.Element == T
func makeMeACollection<T>(_: T.Type) -> MyCollection<T> {
   return [T]()   // okay: `MyCollection<T>` is `Array<T>`

// or
func makeMeANumeric() -> Int as Numeric {
   return 5

The generated interface could be:

typealias MyCollection<T> = _ as MutableCollection & RangeReplaceableCollection where Self.Element == T
func makeMeACollection<T>(_: T.Type) -> MyCollection<T> 

func makeMeANumeric() -> _ as Numeric

(Jon Hull) #201

@Douglas_Gregor: Ah, I see your point.

Assuming it is only used for existentials, opaque types, and typealiases of either, what do you think of when syntax? It reads really clearly, but it does burn a keyword (but only in that position)... aka I couldn't have a type named when.

I do also have other ideas for where when could be used that rely on the upcoming compile time static code. Namely, you could conditionally run a piece of code at compile time based on the exact type. That is beyond the scope of the feature we are talking about here, however.

(Jon Hull) #202


I was thinking if there were enough valid use-cases, we could create a syntax that means "I explicitly want the compiler to infer and fill in this type at compile time". For example:

let x:??? = a.returnsAnInt() 
//same as let x = a.returnsAnInt()

func foo() -> ??? { //Compiler would infer String here
   return "foo"

Then the syntax would normally compose from the two concepts:

func bar() -> ??? as Comparable {...}

It would also work like Scala's "I'll think of a type/name later" placeholder because it wouldn't error until compile time when the type can't be inferred...

But if it isn't useful much beyond this single use case, then it doesn't make sense...

Edit: Ok, because of the Scala thing, I talked myself into checking whether this is something people want in a separate thread.

Scala like placeholder for types