Opaque result types

We could do like recent versions of Rust do, and make it so that the existential type requires a modifier (which could be Any) before the constraint:

struct Thing {
  var data: Any Collection<Element == Int>
}

which would mirror how Doug suggests using opaque for the hidden concrete type case.

8 Likes

I read your comments. I think you're missing my point, which is not earth shattering: we could save a keyword if we eventually want to introduce newtype because if it is used in the position where we're using opaque currently, it can be thought of as an “anonymous newtype”. You'd still have anonymous newtype typealiases that act just like the proposed typealias X = opaque … construction.

Though now that I type it, the name “anonymous newtype typealias” seems to suggest the wrong thing…
…so maybe this idea has value < 0. :stuck_out_tongue_winking_eye:

In turn, I think you missed Doug's point—the opaque result type cannot be a newtype in the general case, because that would require forwarding of the exposed conformances from the underlying concrete type to the newtype, which is not always possible to automate if there are recursive associated types or Self requirements in structural positions.

I suppose that's true if you think an anonymous newtype has to work the same way as a nominal one. Maybe this violates the expected meaning of newtype too much to tolerate, but my idea was that a typealias for an anonymous newtype would act exactly the way a typealias for an opaque type is proposed to act.

Been following along for some time now, and wanted to offer a super straw-man idea for improving the syntax.

Would something like this work?

func dance<A, B>() -> 
  opaque Foo<
    AssocType1 == A, 
    AssocType2 == Result<B, Swift.Error>> // assuming the Boogaloo happens :slight_smile: 
  where
    A: Equatable
    B: Equatable

It retains the conditional where-clause flexibility (at the cost of extra generics admittedly), but also provides a bit of 'scope' to the relevant parts of the opaque type itself.

Also, is there ever a reason that you'd want to say, express opaque type of opaque types? That sort of context is not really distinguishable in the _.assocType flavor. It seems like good information to have/not lose perhaps.

For instance, is this even meaningful to want?

func randomMatrix() -> opaque Collection<opaque Collection<Element: Double>>

Certainly such an idea is expressible unambiguously with a type erasers.

I have a minimal prototype implementation of opaque result types in this pull request:

https://github.com/apple/swift/pull/21137

Limitations include:

  • The opaque type is spelled __opaque Protocol or __opaque ProtocolA & ProtocolB [& ProtocolC...] as placeholder spelling.
  • Opaque types can only appear as the return type of func declarations, not properties or closures.
  • where clause constraints on the opaque type are not yet supported, nor are conditional conformances on the opaque type. However, the protocols in the __opaque type can be arbitrary protocols, with any number of associated type or Self type requirements. __opaque types can in turn be inferred as associated types of protocol conformances.
  • The runtime and resilience aspects are not yet implemented. Changing the underlying type of a public opaque type will break ABI.

Some known issues:

  • Because of the missing runtime support, I substitute out opaque types during SILGen in a rather hacky unprincipled way. It's likely that the optimizer, particularly generic specialization, may still cause crashes.
  • The type checker currently crashes if there's a type error in the underlying type, such as if it does not conform to the required protocols.
  • Diagnostics involving opaque types will print the type with a compiler-internal representation involving the original decl name and generic arguments, instead of anything human-understandable.

I asked CI to build a toolchain: https://ci.swift.org/job/swift-PR-toolchain-osx/170//artifact/branch-master/swift-PR-21137-170-osx.tar.gz I'd appreciate help kicking the tires on this. Thanks!

37 Likes

Should it be conveyed to the programmer that the opaque value has value/reference semantic?
It seems like something programmer will want to know, (or else we would restrict its assignment to multiple location, implying unique semantic, which is irrelevant here, and is a different beast entirely).

So the question is, should it be conveyed to the programmer that the opaque value has value/reference semantic?

1 Like

There is nothing about opaque result types that make them unique in this respect from any other generic or existential context where the concrete type is unknown. Right now, AnyObject can be used to consorting to reference semantics. Perhaps in the future we will be able to use AnyValue to constrain to value semantics (see Strict Value Semantics).

4 Likes

I think a subset of this proposal should be ready for review soon. One thing that'd be good to sort out is the keyword we use to introduce an opaque result type. The word "opaque" makes sense to describe the language, but isn't terribly evocative if you read it at a declaration site, and furthermore, it has existing meanings in other domains, particularly graphics, that could be confusing:

protocol Shape { ... }
func translucentRectangle() -> opaque Shape { ... }

One alternative that comes to mind is some. This has the right connotation, that the function returns a value of some type, that's still specific, and it nicely complements our use of Any* in dynamically type-erasing containers. some is however already used as one of the case names for Optional (although it's rare to need to spell it out explicitly because of Optional's sugar). How does some sound? If anyone has other ideas, I'd like to hear them too.

10 Likes

I like some. It matches very well. On the other hand, it might limit our options when we get to generalized existentials.

some sounds very nice to me +1

Awesome, looking forward to this!

I like it.

This doesn't bother me.

We probably don't want to reconsider it now, but it seems like if source compatibility wasn't an issue we might choose wrapped for the Optional case to be consistent with the accepted design of Result. If there was appetite for going there it would be a way to eliminate the different uses of some.

I don’t have any objections to some. I’d also say specific would be a reasonable name - it’s a specific type, even though that type is not known. specific might also read a little better in error messages; e.g. “The return value of someFunction is a different specific type than the return value of someOtherFunction”.

1 Like

I agree that specific is a nice alternative to some. Along the same lines I can only offer the inferior concrete and fixed. But some just reads so well...

One concern I have with some — and perhaps I'm off-base to worry about this — is that it relies on a fairly nuanced sense of the English word that might be unfamiliar or confusing to non-native speakers. It's "some" specifically in the sense of "a non-specific example of", like "some kid ran into me in the park", not "some" as an alternative to "none" or "lots". So not only would we be using "some" in two different fairly core places in the language, but we'd actually be using a different sense of it in each place.

11 Likes

I actually like concrete better than specific. We often speak of "concrete" types but don't often speak of "specific" types.

As a non-native speaker, I don't think this is a concern. Just about anything can be a stumbling block to a learning speaker, but if we concern ourselves too much with that, we'll quickly run out of names for anything interesting.

2 Likes

I think some X is fine when paired with a singular noun, like some Shape. I could see it reading a bit more oddly when paired with a type name that is a plural noun, but I think users would adjust quickly.

Regarding specific or concrete, I'd like to propose going the opposite direction: unspecified Shape. I think that gets the observation across that the return type is unspecified at the point of declaration more clearly.

Last alternative: anyold Shape :man_shrugging:

7 Likes

A while back I jokingly proposed "a specific" and "an arbitrary" as keywords for existentials and universals, and I keep wondering if I was actually joking as much as I thought I was.

I guess spaces in keywords is probably a bridge too far heh.

(I like 'some' just fine)

2 Likes

I think similar things could be said about the current usage of some as used in the Optional context, or other keywords like where (EDIT: this is regarding the nuanced usage, the duplication point stands). Even the philosophy to have calls read as grammatical phrases sometimes involves somewhat convoluted constructions that might be confusing to a non-native speaker. I like some the best of the proposed options I've seen.

I'll also throw in my objection to _.Element. I like the cleanliness of the unnamed version, but I think that if you want to do some nontrivial constraints on your opaque type, you should be forced to give it a name, even if that name is local to the result type declaration of the method you are declaring. I'm new to this thread and haven't read each and every post here, so apologies if it has already been proposed, but I would imagine something like this:

func makeMeACollection<T>(_: T.Type) -> opaque Q: MutableCollection & RangeReplaceableCollection where Q.Element == T {
    return [T]()
}

Q would be inaccessible from outside the context of makeMeACollection(_:), and I could go either way on whether Q could be used in the body. Also, the type name could be omitted if you didn't need to access any associated types, so this would also remain valid:

func makeMeAnyCollection() -> opaque MutableCollection & RangeReplaceableCollection {
    return [Int]()
}

EDIT: This looks very similar to the syntax @anthonylatsis was proposing last year.

3 Likes