Function builders implementation progress

Hi all,

We've been making quite a bit of progress on the implementation of function builders within the Swift compiler's type checker. While I'm not quite ready to reopen the big design discussion, I wanted to call attention to some of the work we're doing (some of which has made it into Swift releases, some that is still landing in master) to bring it closer to the proposal and get this feature to realize its full potential. The hope here is to get the technology into place to allow us to really explore all of the cool ideas that have come up in the design of this feature, improving the development experience along the way.

Here's a summary of what we've been up to:

  • One-way constraints are a change to the way in which we perform type checking for function builder closures. At a high level, this change makes type checking flow from the "top" of a closure down to the "bottom" (but not backwards). This better matches how normal function type checking works and also provides a significant improvement in type checking performance; technical details are in the code base and early pull request. This was shipped as part of Swift 5.1.1.
  • The new diagnostic architecture allows us to properly diagnose errors deep within function builder closures by honing in on the specific problems that prevent a proper type check. This is technically separate from function builders, but is critical to their usability for developers. The bulk of this landed in Swift 5.2, and you can try it out with the Swift 5.2 development snapshots, and we hope the difference is night and day.
  • buildExpression support: one can introduce a static method buildExpression into a function builder type, and it will be called to map each raw expression within a function builder closure to a value that will then be passed to buildBlock. buildExpression can be overloaded to provide some contextual translation of expressions that would be unwieldy with buildBlock alone.
  • #warning and #error support, which are small-but-useful additions.
  • Delayed constraint generation for single-expression closure bodies lets the type checker use more contextual information to type check closure bodies. For function builders, it means that single-expression closures will correctly make buildBlock and buildExpression calls (they were previously skipped in this case).
  • Statement-based translation of function builders makes the implementation model in the compiler match what is specified in the proposal. Prior to this change, function builders were implemented by folding the entire closure into a single-expression closure with one giant expression. With this refactor, it becomes easy to support aspects of statements that cannot be described in expressions, such as multiple conditions in an if and support for if #available. It's scaffolding for supporting let declarations, if let, and switch statements to make function builders more expressive, although that generalization is still a work-in-progress.

Not coincidentally, a lot of these changes impact on our ability to evolve the language further in the future, particularly in the area of closures. For example, the delayed constraint generation for single-expression closures could allow code like this to compile:

let f: (Int, Int) -> Bool = { $0.isMultiple(of: 2) }

Right now, one would have to add something like (x, _) in to the closure to show that it takes two parameters.

The generalization of function builders to work with more-or-less arbitrary statements opens up the potential for better type inference through multi-statement closures. For example, right now a single-expression closure can infer a closure result type:

let result = someCollection.map { $0.method() }

but a multi-statement closure does not:

let result = someCollection.map { 
  if $0.someProperty {
    return $0.method() 
  }
  return $0.someOtherMethod()
}

and one will have to provide some type annotations to make this code compile. We should be able to infer the result type of the closure to avoid the need for extra type annotations.

No promises, of course, but it's fun to see where improvements in the architecture and implementation of the type checker can take Swift, especially in making function builders more powerful and more expressive.

Doug

64 Likes

Inferring result type of multi-expression closures would be a BIG win! Looking forward to the day this ships, if possible :grinning:

22 Likes

I wonder if this would allow omitting the return type declaration on functions and computed properties to further reduce the visual noise:

var radius: Double
var diameter { radius * 2 }

@Douglas_Gregor that's a great insight into what's been going on and some fantastic progress made by you and the team. The beauty of a lot of these features is that they concentrate on a lot of 'quality of life' improvements which are a huge step forward and massively appreciated in the day to day world of Swift development. Having posts like this to highlight the extraordinary effort needed to make these improvements possible really brings those achievements to the forefront.

8 Likes

This feature would be possible, yes. I would oppose such a feature, for both a philosophical reason and a practical one:

  1. When looking at a declaration, it should be obvious what the type of that declaration is, so that you know how to use it. If you have to reason through the implementation of a function to figure out a return type, it affects code readability. This is the reason Swift doesn't allow one to omit the result type of a function in the declaration, even though we could (technically) implement it in the compiler.
  2. Compile-time performance suffers when there is no barrier between the declaration and body of a function, because it means that every source file that makes a use of that function (e.g., to call it) must also type-check the entire body of the function.

Swift does allow

var circumference = Double.pi * diameter

which I regret due to (1) and has been demonstrated to cause compile-time performance problems in real-world Swift projects (2).

Doug

15 Likes

It is fun! Thanks for the update.

+1, both this and not having to explicitly discard unused arguments would be awesome!

2 Likes

Am I correct to assume that the opaque return types are what is intended to be used when the return type isn’t known?

It depends. Opaque result types are the right tool when you don't want to expose the concrete type you're producing to your clients. If the problem is that you don't know how to write out the type... I'd suggest writing the wrong type and letting the type checker tell you.

Doug

Have Xcode 11.4 beta 1 toolchain include these changes?

2 Likes

Xcode 11.4 contains Swift 5.2. Only those changes that I’ve said are in 5.2 or earlier will be there. “Master” development snapshots from Swift.org will have the latest.

1 Like

Exciting! Might this at last lead to a fix for this longstanding issue?

1 Like

It should make that easy to fix.

5 Likes
Terms of Service

Privacy Policy

Cookie Policy