Future work on opaque result types

Development Compiler
,
1

Hello Swift community,

I worked this summer as an intern on the Swift compiler team. I mostly finished an implementation of "structural" opaque result types as described in this evolution proposal.

However, this is my last day on the team, and there is more work to be done on opaque result types that I did not have a chance to finish this summer. I wanted to leave a record of what that work is, for myself and the community.

Structural Opaque Result Types

There are a few, relatively minor details that need to be cleaned up with structural opaque result types and I'll talk about these up front because they will not be the main focus of my post. In fact, I have already been working on fixes for a few of these things, though I may or may not have the time to finish my changes.

First, there three very particular cases described in a TODO toward the top of test/type/opaque_return_structural.swift that do not work.

Secondly, support for multiple opaque result types is not fully implemented. This might seem significant, but it should not be particularly hard to implement because I have already sufficiently generalized the type checker. It's just a matter of tweaking type resolution (which was written with this generalization in mind), serialization, and deserialization.

Named Opaque Result Types

On to the main topic of this post. The team would like to finish the implementation of “named" opaque result types as described here.

To summarize the desired feature: currently, opaque result types cannot be given a name so they cannot be constrained in a where clause or used in multiple positions in the return type. As an example of how named opaque types might be useful, consider the Swift standard library function zip, which has the following declaration:

func zip<Sequence1: Sequence, Sequence2: Sequence>
(_ sequence1: Sequence1, _ sequence2: Sequence2) -> Zip2Sequence<Sequence1, Sequence2>

With named opaque result type support, we can reap the benefits of opaque result types in our API and instead write:

func zip<Sequence1, Sequence2>
(_ sequence1: Sequence1, _ sequence2: Sequence2) -> <Sequence3> Sequence3
  where Sequence1: Sequence,
        Sequence2: Sequence,
        Sequence3: Sequence,
        Sequence3.Element == (Sequence1.Element, Sequence2.Element)

Since named opaque result types can be constrained in a where clause, we can expose the element type of the return sequence without a purpose built Zip2Sequence struct.

Note that named opaque result types should work wherever opaque result types work. That means, in addition to working as function return types, they should also work as the declared types of properties and variables. For instance, let x: <T> T = 1.

The Current State of the Compiler

I have already implemented type checking in the presence of multiple opaque result types, which is necessary for named opaque result types because of constructions like <T> (T, T), as mentioned above.

I have also modified the parser to use the new parsing function parseTypeWithOpaqueParams, which produces the new type repr NamedOpaqueReturnTypeRepr if a generic parameter list is supplied in the source code and the --enable-experimental-structural-opaque-types compiler flag is set. For instance, func f() -> <T, U> (T, U) would have return type repr (type_named_opaque_return (type_tuple (type_ident (component id=’T’)) (type_ident (component id=‘U’))).

Currently, the parsed generic parameter list is stored in the NamedOpaqueReturnTypeRepr but completely ignored. TypeResolver::resolveType cases on resolving a NamedOpaqueReturnTypeRepr and simply resolves the type repr nested inside that type repr (this probably does not need to change, see the next section).

All tests for named opaque result types are in swift/test/type/opaque_return_named.swift.

What Needs to Change

By the time opaque result types get to type inference, they should all be represented by OpaqueTypeArchetypeTypes, regardless of whether they were named. Therefore, implementing named opaque result types probably only requires modifying type resolution and not type inference.

In terms of type resolution, a good place to start looking is probably resolveTopLevelIdentTypeComponent and NameLookup.cpp. When the compiler sees the Ts in the tuple type in <T> (T, T), or any other unqualified type name, resolution will end up in resolveTopLevelIdentTypeComponent. In order to make that find the named opaque result types, one probably has to modify the compiler so that the right ASTScopes are produces for the generic parameters in the NamedOpaqueReturnTypeRepr, which is where name lookup comes into the picture. For named opaque result types, the ASTScope for a parameter should end after the where clause:

func f() -> <T> T where T: Sequence { /* ... */ }
//             <------ASTScope------>

One wrinkle that I have not mentioned so far is that we probably want to allow where clauses in some new locations, namely in property and variable declarations, e.g. let x: <T> T where T: P = 1. I have not touched where clause parsing.

29 Likes
2

I've been working on these, and addressed most of the remaining issues:

  • Structural opaque result types completed the implementation of structural opaque result types. There were scattered issues throughout the compiler.
  • Named opaque result types implemented the ASTScope name lookup changes @bdriscoll describes and a few of the other fixes needed so one can use named opaque result types so long as they don't add any additional requirements.
  • Generic environments and opaque types made it possible to use where clauses with named opaque result types, although the where clause has to go inside the angle brackets, e.g., func f() -> <T where T: Sequence> T { /* ... */ }.

My primary interest was in implementing SE-0328 "Structural opaque result types", which is now done on main. If someone were inspired to complete named opaque result types, here's my TODO list of things that would need to be done:

  • Check an opaque result type to make sure it mentions all of the named parameters somewhere, the same way we do for generic functions. For example, func f() -> <T> Int { } should complain about T not being mentioned in the result type, just like func g<T>() { } complains that T is not used anywhere in the function type.
  • Figure out what it means to name one of the generic parameters within the body of the function, or ban such uses entirely: func f() -> <T> T? { ... T ... }
  • Allow the function's where clause to specify requirements on the named opaque type parameters (along with other requirements), rather than the odd syntax that works now with the where inside brackets. The tricky part here is to sort out the requirements that apply to the caller vs. requirements that constraint the result type.

I'm not planning on working on any of these myself in the near term.

Doug

12 Likes
3

Just so it isn't lost: does this (the remaining tasks not yet to be assigned) include effects? Such as: if a type implements a throws requirement as non throwing?

4

No, I was not considering that in my list. It’sa good point, and we’d need a design for it.

Doug