SE-0244: Opaque Result Types

(Manolo van Ee) #57

I came up with the ^ prefix because I wanted a way to declare the output types that wouldn't break up the space between the -> and the return types with an extra sort of generic type parameter list <C,D>. However, perhaps a good alternative is to declare the output types in the existing generic type parameter list, but with a separation to make it clear that these are not generic parameters, but output types. A pipe could be used for example. If we do that, we don't have the visual noise of ^, but everything else stays the same:

func makeTwoOpaqueCollections<T | C, D>(with element: T) -> (C, D) where C: Collection, D: Collection {
  return ([element], [element] as Set)
}

Both have advantages I guess. The ^ prefix makes it very clear that those types are opaque, but without the ^ it looks cleaner.

(Yuta Koshizawa) #58
  • What is your evaluation of the proposal?

+1.

I like the keyword some because it seems natural that some Foo type and other some Foo type mean different types.

var a: some Foo = foo1()
let b: some Foo = foo2()
a = b // Error: `some Foo` and `some Foo` are different types

I think that it is less obvious if we adopt the keyword opaque.

Although I think the proposed syntax works well enough, I am not sure if it is the best one. I think generic arguments and opaque result types are dual. For example, for the following two functions,

func useFoo<F: Foo>(_ foo: F) { /* ... */ }
func makeFoo() -> some Foo { /* ... */ }
  • users of useFoo decide a concrete Foo type and implementers of the function use an abstract Foo type
  • implementers of makeFoo decide a concrete Foo type and users of the function use an abstract Foo type

Both of them are counterparts of existential types. They have similar roles for arguments and return values respectively.

However they don't have syntactic relations. It is possible to imagine some syntactic relations between them. For example, as implementers of functions give temporary type names for arguments by type parameters, it may be also possible that users of functions which return an opaque result type give temporary type names to return types. Such syntax makes it possible to avoid errors caused by assignments between (same-looking) some Foo types as shown above by giving them different names.

  • Is the problem being addressed significant enough to warrant a change to Swift?

Yes.

  • Does this proposal fit well with the feel and direction of Swift?

Yes.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Just read the proposal.

5 Likes
(Joe Groff) #59

You're correct that they're duals. I touched on this briefly in "future directions". It would be great to eventually support some Foo as an argument type as well, which would let you write:

func useFoo(_ foo: some Foo) { /* ... */ }
func makeFoo() -> some Foo { /* ... */ }

making the duality more apparent.

6 Likes
(Svein Halvor Halvorsen) #60

I do like the direction this is going, but I feel strongly that the word some is cutesy and unnecessary.

The feature is clearly called "opaque types" as evident by how we talk about the feature in this very thread. Why not use a very precise term of art opaque? It seems easier to google and better matches how we talk about it orally. The counter argument that it may be confusing in cases such as

func translucentRectangle() -> opaque Shape { /* ... */ }

… seems very contrived and rare. However users will probably discover some every day, and have a hard time dealing with the fact that it is hard to google and hard to talk about.

Cute, but unnecessary.

7 Likes
(Tino) #61

In general, I agree that re-use of existing terms to minimize the number of keywords doesn't make sense (I wouldn't propose enum instead of some ;-).
But I think as is less ambiguous for those without prior knowledge of Swift, and also for existing users:
At least for superclass-relationships, as can already be used to achieve a transformation like the one needed for opaque types (also I guess the protocol-case is much more important for the pitch).

(I consciously posted in the discussion-thread; I don't mind the text being moved over, but pease don't complain about distraction from the review ;-)

(Manolo van Ee) #62

Exactly this duality tells me we need to design a counterpart to generics. I don't know if there is an existing term for that. Perhaps 'reverse generics'? I wil use this term for the rest of this post. Please correct me if that doesn't make any sense.

Refining my earlier idea a little bit, using the ^ in the generic parameter list to designate reverse generic parameters:

func useFoo<F: Foo>(_ foo: F) { /* ... */ }
func makeFoo<^F: Foo>() -> F { /* ... */ }

So, to be clear:

  • Normal generic type parameters are made concrete by the caller.
  • Reverse generic type parameters (prefixed ^) are made concrete by the callee.

Type inference will most of the time do its job, but explicitly typing something as the reverse generic type parameter of makeFoo() could perhaps be done as follows:

// Explicitly typing a variable with the return type of makeFoo():
var myFoo: makeFoo.F = makeFoo()

// Explicitly typing an array of the type returned by makeFoo():
var myFooArray = [makeFoo.F]()
myFooArray.append(makeFoo())

This notation also makes it possible to constrain functions to return the same concrete type, which could make @Vogel happy:

// Returns an opaque collection
func makeCollection<T, ^C: Collection>(with element: T) -> C where C.Element == T {
  return [element]
}

// Returns the same type opaque collection, with Element == String
func makeCollectionWithString<^C>() -> C where C == makeCollection.C, C.Element == String {
  return makeCollection(with: "Foo")
}

// The following will type check correctly:
var myCollection = makeCollectionWithString() + makeCollection(with: "Bar")
print(myCollection) // ["Foo", "Bar"]

Using this notation there is no need for a keyword like 'opaque' or 'some'. I see that @brentdax posted a similar idea in the Protocol<.AssocType == T>... thread and he notes that 'some' could be used as shorthand syntax. In my opinion the following would be the right course of action:

  1. Decide on the basic design for a 'reverse generics' system.
  2. See how ergonomics can be improved with additional shorthand syntax (if there is a need for it).

I think this will provide a better foundation for future directions.

I'm no language expert, so if I said anything that makes no sense or if I butchered any terminology, please let me know.

(Ryan Sobol) #63

Opaque types are new to me. As such, one of the aspects if this proposal I’m still having trouble understanding is under what scenarios is an opaque type is preferred over an existential type. Would anyone be willing to write up a brief ELI5 summary?

(Steven Van Impe) #64

I was in the same boat, but after some reading up, I believe the difference is as follows. I'll use these functions as an example:

func one() -> Equatable { ... }
func two() -> some Equatable { .... }

The return type of one can be of any type that is Equatable. Different invocations of one may return instances of different types. The function only guarantees it will return an Equatable, not that it will return the same type every time. The return type of two is some type that is Equatable. Unlike one, it's the same type every time; it's just hidden from the user.

As a result, instances returned from one will have to use dynamic dispatch, as the compiler doesn't know the exact type, so the decision has to be made at runtime. Instances returned from two can use static dispatch if the compiler can infer the concrete type from the implementation.

So:

  • If you want a function that can decide which type to use at runtime, use one (this is regular OO polymorphism).
  • If you want to hide the type you return because it's an implementation detail and shouldn't be exposed, but you want to avoid the overhead of dynamic dispatch, use two.

I hope that's a correct summary. If not, someone will surely correct me :slight_smile:

4 Likes
(Chris Lattner) #65

I am surprised the alternatives considered section only includes two names here, when many many others were discussed over on the pitch thread.

I have the same position as before - this is an obscure feature that most people will not see very often, and is not a core part of the language that all users should be expected to know to be effective in Swift. Therefore, it makes sense to use a long (and probably compound) keyword for this concept. Using a short name harms 'web searchability' and reduces the odds that someone (who doesn't already know what is going on) can figure out what they keyword is doing without doing that research.

This is not going to show up pervasively in Swift code, I see no advantage to huffman encoding it down to a 4 character identifier, I'd rather see a 15 character compound name of some sort.

It would also be nice for the proposal to be revised to include all the alternatives that the pitch thread considered.

-Chris

23 Likes
(William Moss) #66

Here are some examples proposed in the pitch thread in use of an opaque return type. I'm not assuring that it's complete or accurate, but it's an attempt to throw things at the wall for discussion.

For those that are having trouble with visual connotations of words like opaque or hazy, remember that's going to happen with any colloquial word before one understands it's purpose and use. If you didn't know lazy was a term-of-art, your brain will try to imply sloth on a lazy Collection of OfficeWorkers. It may take time for one to read code that calls for an opaque Shape rather than an opaqueShape but with use the visual connotations will fall away. On the positive side of a colloquial word or phrase, it should be easy to look up and less likely for conflict with future words Swift might want to reserve for later.

Throwing things against the wall from the pitch thread:

func translucentRectangle() -> opaque Shape { ... }

func translucentRectangle() -> unspecified Shape { ... }

func translucentRectangle() -> anyOld Shape { ... }

func translucentRectangle() -> hazyButSpecific Shape { ... }

func translucentRectangle() -> someSpecific Shape { ... }

func translucentRectangle() -> someConcrete Shape { ... }

func translucentRectangle() -> nonspecific Shape { ... }

func translucentRectangle() -> unclear Shape { ... }

func translucentRectangle() -> arbitrary Shape { ... }

func translucentRectangle() -> someThing Shape { ... }

func translucentRectangle() -> anonymized Shape { ... }

func translucentRectangle() -> nameless Shape { ... }

func translucentRectangle() -> unnamed Shape { ... }

7 Likes
#67

It looks like you missed:

func translucentRectangle() -> doNotLookBehindTheCurtain Shape { ... }
4 Likes
(David Hart) #68

We might as well keep the original ‘opaque’ name. It was fine by me.

(Joe Groff) #69

Indeed, this is pretty much what opaque types boil down to, is the ability for a function to "output" generic parameters in addition to taking them as inputs. Another way we could conceivably notate this is by putting the output parameters after the ->, as a number of people suggested:

makeFoo() -> <F: Foo> F

The proposal tries to explain it, though the section about it is quite lengthy and I can see how it can be unclear. @Douglas_Gregor, maybe we should go over this section again and try to make it more to the point. @svanimpe's description is a good summary.

I understand the concern about searchability, and I checked—Google does a fine job providing relevant answers for swift some today. (There's a legitimate concern that having two somes could dilute the waters, I agree.) I however don't agree that this feature will remain obscure for long. The equivalent impl Trait feature in Rust is used pervasively; On Github I find 5,000 hits for impl Fn arguments and returns, 1,000 for impl Iterator<Item=...>, and 400 for impl Future alone. Although Swift uses type erasure for function types, obviating the first to some degree, we could deploy this feature for standard library collections as the proposal suggests, and we could well end up with protocol-oriented futures, key paths (and maybe even generic function objects like Rust's, since there are optimizations that enables that our current closure design makes difficult) in the future. Part of the goal of this feature is to make those sorts of libraries easier and more ergonomic to design and use.

As for "Huffman coding" the identifier, I have a couple other reasons I think going for something concise that reads well is a good idea. I'd really like to see this feature eventually extend to also allow generic arguments to be written as some Protocol, the same way that Rust lets you use impl Trait as an argument, since that will make writing a lot of simple generic functions clearer and easier. For this sugar to be worth using, it ought to be at least as concise and readable as the syntax it replaces, and a bulky, obscure keyword would kill that quickly. Furthermore, I also see a bit of a let vs var situation with regard to existential notation. @dabrahams and others have noted that protocol existentials in their full generality quickly become a rather obscure, complex feature, that doesn't necessarily deserve the pride of place it currently has in our type syntax (and that if anything, opaque types as provided by this feature are in fact closer to what you really want a good percentage of the time we currently tell people to use existentials). Rust eventually decided that making existentials undecorated was a bad idea and added the dyn keyword to encourage their use to be explicit. In previous discussions of existential syntax, Any or any seemed like the clear favorite as the modifier keyword for explicitly introducing an existential. Both linguistically, and in the spirit of fairness of not syntactically biasing one style over the other, Some or some strikes me as the corresponding modifier for specific opaque types.

14 Likes
#70

+1 for the concept, but -1 for the syntax because it doesn't allow explicit constraints, and doesn't scale to multiple opaque types. I quite liked the syntax proposed in the original discussion thread by @anthonylatsis (here: Opaque result types) which explicitly named the opaque "thing" so it could be constrained in a where clause. To summarize what I think the linked post was getting at, it would look like:

func foo() -> some T
    where T: Collection, T.Element == Int
{ 
    return [1, 2, 3]
}

This has several advantages. For one, it doesn't require any new syntax for the constraints and instead re-uses the existing where clause that is already familiar. This allows it to scale to multiple types:

func foo() -> (some T, some Q)
    where T: Collection, T.Element == Int, Q: Numeric
{ 
    return ([1, 2, 3], 4)
}

And finally it allows opaque constraints to mix with generic constraints (composability!) which I think will be really important for the opaque type concept in general:

func foo<R>(input: R) -> some T
    where T: Collection, T.Element == R
{ 
    return [input]
}

I would of course support type inference to allow omission of some or all of the above constraints if they can be inferred from the return type:

func foo() -> some T: Collection
{ 
    // T.Element == Int is inferred
    return [1, 2, 3]
}
2 Likes
(Joe Groff) #71

Since part of the intent of this feature is to provide more control over the interface a function represents, I don't think it would be a good idea to infer constraints that weren't written. For instance, while you may want to expose the Element type of the returned collection, you most likely don't want to expose its Index, since in most cases that'd stick you with one underlying collection implementation in practice.

6 Likes
#72

That's a really good point, and I agree but correct me if I misunderstand but the proposal as written would infer both Index and Element form the return type, wouldn't it?

(Joe Groff) #73

As written, nothing is inferred; it doesn't yet propose a way to describe constraints on the returned type. Your notation would be one way to do so; other approaches are being discussed in this thread:

#74

I guess I have a fundamental misunderstanding of how this works. From the original proposal:

func makeMeACollection<T>(with element: T) -> some MutableCollection & RangeReplaceableCollection {
   return [element] // ok: an array of T satisfies all of the requirements
}

Does this not infer .Element == T for the underlying type? If not, how can the program actually compile?

I guess I'm worried that since this proposal explicitly does not name the opaque thing, by accepting it we are tacitly vetoing any syntax which seeks to do so. I think the syntax for constraining opaque types needs to be nailed down before accepting a proposal introducing opaque types.

1 Like
(Joe Groff) #75

It works similar to what you'd get if you passed in a generic argument constrained to <T: MutableCollection & RangeReplaceableCollection> with no Element constraints. You know that the value has an Element type, and that it's consistent among values returned by the same function, but you don't know exactly what type it is.

A number of people suggested mirroring the generic argument syntax on the other side of the <>, which seems nice and uniform with generic arguments:

func makeMeACollection<T>(with element: T) -> <C: MutableCollection & RangeReplaceableCollection> C
   where C.Element == T

and could allow the some syntax to still be used in either argument or return position in common cases where there are no secondary constraints among arguments or results.

4 Likes
#76

I went back to the proposal and found this:

Following the some keyword is a class, protocol, Any , AnyObject , or composition thereof (joined with & ).

It's maybe a degenerate case, but some Any doesn't seem to read too well. :slight_smile:

I notice that list of things that can follow the keyword is more or less the same list that can follow a : indicating type conformance (e.g. struct T: P). If you want to consider something longer than some, what about one of these:

    … -> conformanceTo P
    … -> typeConformingTo P

or something along that line?