[Pitch #2] Function builders

[John_McCall]

That's just not supportable at the level of generality you're thinking of without abandoning a ton of other language goals. You'd basically need the preserve the function source and recompile it at runtime.

[lassejansen]

I don’t want to inspect what foo returns just to figure if it is to be included. What if it has @discardableResult ? The answer to the latter question will dissatisfy users half of the time, and that’d be a pretty big footgun.

I think @discardableResults should be passed to the handleBuilderValue method. I think the chances that the code behaves unexpectedly to the developer is very low:

• @discardableResult is used quite rarely in my experience.
• If a function does use @discardableResult, then there must be an overload of handleBuilderValue that matches the return type of the function.
• If the return type of the function matches an overload AND it's not desired that this value is used by the builder, the user can use underscore syntax to ignore the value: _ = methodWithDiscardableResult().

This one elevates the building closure to a new class of function, which IIRC was already challenged during the first pitch.

I'm not sure what "a new class of function means" in this context.

To be clear, I don't want to replace function builders with my proposal, my argument is just that function builders are a special case that could be embedded in a generic builder feature.

The generic builder would run arbitrary Swift code and emit "free" (a.k.a some definition of "unused") values to its parent, without the need of implementing all those buildExpression, buildBlock, buildArray, ... functions.

If for performance or semantic reasons it's not desired to run arbitrary Swift code in a build block, or if access to AST information is necessary (like for SwiftUI), one could opt into the "low-level" API of @functionBuilders, as it is described in this proposal.

[TizianoCoroneo]

Not even something like this?

Wild pseudocode warning

@functionBuilder struct UITestBuilder: FunctionBuilder {
    typealias Representation = Test
  
    // implementation 
}

struct Button<Builder: FunctionBuilder>: Builder.Representation {
    @Builder var body: some Builder.Representation {
        // Some kind of button
    }
}

let view = Button<ViewBuilder>()
let test = Button<UITestBuilder>()

[Lantua]

@lassejansen

The generic builder would run arbitrary Swift code and emit "free"

This is what I mean when I say a new class/kind of function. It's a closure that keeps emitting new values, as opposed to a closure that just returns the value once. You can not treat a function that uses builder as a normal closure, which you can still do in the current pitch. You also need to maintain the source code if you want to have different level of control on the same closure, which IIUC is what you're trying to do with Builder.apply.

[lassejansen]

Ok, I see what you mean. Although I'm not sure why this would be a show stopper, except for "too much work".

[Lantua]

It's not a show stopper, but I don't see it as a generalization to the current design. It's more of a counter-proposal than a future direction.

[lassejansen]

Well, from the perspective of a DSL consumer both variants look largely the same:

• You provide a closure block to a function
• "free"/unused values are in one way or another used to construct the desired object graph
• You can used Swift syntax to control the end result

In general I assume that a DSL implementor doesn't want to restrict which Swift syntax construct a DSL consumer is able to use. So I would assume that most DSL implementors would implement all the methods that are provided by this proposal.

My point is now while it's sometimes necessary to have this low level control over the AST and to prevent certain syntax constructs – e.g. for performance, to prevent heap allocations – that this is not needed in the general case.

Also, most DSL implementors would probably choose to allow the full Swift syntax, instead of some subset.

The reason I see my proposal as a generalization is that the end result of my proposal looks the same to a DSL consumer as in the function builder proposal. The function builder proposal would be a lower-level API to create the same consumer API (with some added restrictions to achieve specific goals).

[Mordil (Nathan Harris)]

I've been thinking of this feature more of like a closureEvaluator or something similar

[lassejansen]

It's more of a counter-proposal than a future direction.

To bring the syntax of my suggestion more in line with the existing function builder proposal, a way to switch between the high-level and the low-level syntax could be to use instance methods of the function builder type for the high-level API instead of static functions. The UIStackView example from above would then look like this:

class UIStackView: UIView {

  @functionBuilder
  struct Builder {
    let stackView: UIStackView

    func addExpression( _ expression: UIView) {
      stackView.addArrangedSubview( expression)
    }
  }

  @discardableResult
  init( axis: Axis = .vertical, @Builder content: () -> Void) {
    super.init()
    self.axis = axis
    Builder(stackView: self).evaluate( content)
  }
}

The differences in summary:

High-Level / Dynamic API

• Would be enabled by implementing the instance method addExpression on the builder type for each type that should be captured.
• Would be triggered by calling evaluate( ...) on a builder instance.
• Would allow the DSL consumer to use arbitrary Swift syntax in the builder closure.

Low-Level / Static API

• Would be enabled by implementing some or all of the static functions from the function builder proposal
• Would be triggered by calling the closure block directly (or maybe by calling a static evaluate function on the Builder type, for symmetry reasons).
• Would only allow the subset of Swift syntax features that were enabled by implementing the respective buildSomething... methods from the function builder proposal.

I'm aware that this is probably out of scope for this proposal, I just want to bring it up as a possible future extension, because I think that the high level API would be sufficient for most use cases and much easier to use for "day-to-day" DSLs.

[Lantua]

Rather than the high-level API be part of a generalization, maybe it's a special case of the low-level API? Since it sounds like a boilerplate implementation:

@functionBuilder
struct ArrayBuilder<Value> {
  typealias Content = [Value]

  static func buildExpression(_ expression: Value) -> Content { [expression] }
  static func buildBlock(_ components: Content...) -> Content { components.flatMap { $0 } }
  static func buildOptional(_ component: Content?) -> Content { $0 ?? [] }
  ...
}

[Lantua]

I'm still not sure if we really need to have a separate buildDo instead of only buildBlock. I'm not even sure if it should be visible to the builder.

I think if the grouping is important, most DSL would do better by giving it a name, like Group { ... }.

---

Should we start bikeshedding the function names at this stages?

I'm thinking about:

build(expression:)
build(array:)
build(optional:)
build(withLimitedAvailability:)

buildEither(first:)
buildEither(second:)

buildBlock(components:)
buildDoBlock(components:)

buildFinalResult(components:)

[cukr]

I don't have a candidate for the better names, but the current ones are all over the place without any coherent theme, other than the build prefix which imo makes it feel like a mess.

abstract roles

• buildExpression
• buildBlock
• buildEither
• buildFinalResult

specific constructs in the swift syntax

• buildDo
• buildLimitedAvailability

named after arguments to the builder function

• buildOptional
• buildArray

I think at least the last group should be renamed to something in the first or second category

[Douglas_Gregor]

@Paulo_Faria

Schema<String> {
    Component()
    Component()
}

Per the syntactic transformation, this closure becomes:

Schema<String> {
    let v0 = Component()
    let v1 = Component()
    return Builder<String>.buildBlock(v0, v1)
}

Note that there's no "context" for the initializers of v0 and v1, hence the error. If you add buildExpression to your function builder, like this:

static func buildExpression(_ component: Component<Root>) -> Component<Root> {
    component
}

then the closure is transformed to:

Schema<String> {
    let v0 = Builder<String>.buildExpression(Component())
    let v1 = Builder<String>.buildExpression(Component())
    return Builder<String>.buildBlock(v0, v1)
}

and all your examples work. (Note: I also had to fix your buildBlock to be variadic with ... rather than take an array).

@Paulo_Faria

This is exactly the kind of API that function builders should solve and unfortunately they did not help us here.

Please try with buildExpression---which is, intentionally, the only way in which one can have the builder type influence the types of the subexpressions---and see if your statement above still holds.

Doug

[Douglas_Gregor]

@Paulo_Faria

@Douglas_Gregor do you think this is a reasonable statement? I can't help to think that we're missing a lot of use cases where this capability would be required. I think a good design goal of function builders would be to allow users to swap any initializer they can write today with arrays or variadic arguments with function builders and type inference should work exactly the same way. I believe this is a reasonable expectation since, as I understand it, this is the design goal of function builders, to make these kinds of APIs more ergonomic for developers.

As I just posted, the example from which you drew this conclusion is fixed by adding buildExpression to your function builder. That post has more of an explanation.

Doug

[Paulo_Faria]

Thank you very much, Doug. I'm sorry for the oversight! I'll see if that solves the issue and reply accordingly.

[DevAndArtist (Adrian Meister)]

Diving into the thread mid-flight. Just want to add a few cents about the general functionality name if it‘s still on the table to be potentially rebranded.

@functionBuilder isn‘t really describing the type it is applied to compared to @proppertyWrapper. I do agree on the builder part though. The final type contains functions to build a specific result from the given context inside some closure, function or getter scope. What would be the most general term of art for such result?

I tend towards some @*result*Builder where result would be replaced with something else than result to not confuse with Result type or function as it‘s not the final general result type.

[mtsrodrigues (Mateus Rodrigues)]

So it's unlikely for ... in and forEach working with SwiftUI in the future?

[John_McCall]

That's the SwiftUI team's decision, not mine. But personally, I think it would be somewhat unfortunate to translate loops as lazy maps.

[Douglas_Gregor]

@lassejansen

I'm aware that this is probably out of scope for this proposal, I just want to bring it up as a possible future extension, because I think that the high level API would be sufficient for most use cases and much easier to use for "day-to-day" DSLs.

I can certainly understand the desire to have a simpler way to declare new function builders, particularly simple ones only aggregate values and don't need to propagate types the way more advanced builders like SwiftUI's ViewBuilder do. However, I don't think that it makes sense to approach this as two different APIs from the language perspective. Rather, we should try to make the feature we have composable enough to enable the simple interface. I think we're almost there (more on that in a moment), but first I want to disagree with this bit:

@lassejansen

Would allow the DSL consumer to use arbitrary Swift syntax in the builder closure.

Philosophically, function builders are intended to be more declarative in nature, similar to what fits well if the entire closure were a single expression. Arbitrary control-flow (like break, continue, etc.) goes against that philosophy. There us a short paragraph about this in future directions, but I sorta feel like it's not a good direction. And I certainly don't think we should do it for "high-level" function builders and not "low-level" ones.

Back to making the simple case simple. @anreitersimon showed an example that introduces a FunctionBuilderProtocol that opts in to all of the syntax that a function builder can support, wrapping it up in default implementations so a client can define a new, fully-featured function builder by only implementing buildFinalResult. I think your "Builder" example can be layered on top of that fairly easily.

Note that @anreitersimon's example doesn't work today because of an oversight in the way we did name lookup for the build functions. I just put up a bug fix to the function builders implementation that makes it work. I've also added this example to the proposal so others can find it more easily.

Perhaps some form of that protocol belongs in the standard library. That would address one of @davedelong's concerns as well:

@davedelong

My other piece of feedback is related to @anreitersimon's suggestion. When implementing one of these builders, I struggled to know what I actually needed to implement in order to get things to work. I like the idea of making @dsl / @functionBuilder imply conformance to a standard-library-provided protocol, so that implementors have something they can go look at to see what they need to implement.

@JoeyKL notes that having such a facility would cover a lot of use cases:

@JoeyKL

I think the most obvious takeaway for this is that the function builder API should expose a way to more easily implement function builders for the most common use case, building static data structures under a monoid operation. This could be accomplished by either by defaulting to buildArray when no buildBlock is specified or by separating it into one implementation explicitly for loops and one for a single-method way to define a full builder that accepts control flow.

That we can define this as our own protocol on top of the existing proposal indicates that the language itself doesn't need another "simpler" mode. Rather, we can point people at this protocol (or something like it) as the way to eliminate boilerplate for a class of use cases.

Doug

[EDIT: Added link to the new discussion in the proposal of a "simple" function builder protocol, and response to @JoeyKL's comment that came in a moment after my original post.]

[JoeyKL]

I was thinking about why the current design makes me somewhat uncomfortable, and I think I've partially figured it out. Of 28 uses of @_functionbuilder in projects linked in the awesome-function-builders repository:

• 14, half, only implement buildBlock and all to a common return type.

• 9 implement methods other than buildBlock, but all to a common return type, and only to implement a simple monoid operation (or in 1 case a semigroup operation).

• 5 implement methods with differing return types / special behavior different methods. (One I think unnecessarily; three for in UI libraries (two of which are modeled after SwiftUI specifically); one for GraphQL queries).

I think the most obvious takeaway for this is that the function builder API should expose a way to more easily implement function builders for the most common use case, building static data structures under a monoid operation. This could be accomplished by either by defaulting to buildArray when no buildBlock is specified or by separating it into one implementation explicitly for loops and one for a single-method way to define a full builder that accepts control flow.