SE-0289: Function Builders

I originally tried to simplyfy the rules surrounding buildOptional (partly due to my misunderstanding the buildOptional-only case). Since the rule then becomes: "Always use injection tree; skip buildEither if not defined". That it helps with homogeneous case is actually a fallout.


I see. It sounds harzardous that the builder just eat @discardableResult silently. Maybe we can add a warning, and silent it with an extra parenthesis?:

foo() // warning
(foo()) // ok

If anything, it would be reasonable to treat @discardableResult as non-producing, so it's somewhat ambiguous here.

The rule you're describing only works for the homogeneous case, though, because there is a single vMerged and it will need a consistent type for the buildBlock calls in every "then" block. There is a buildOptional-only formulation that works for the heterogeneous case, where you create a unique vMerged for every "then" block and all of the other vMerged values get assigned nil. But again---buildOptional and buildEither have specific uses, and homogeneous cases that use something like ArrayBuilder won't need to think about any of this.

Doug

Fair point. I still think that it's weird that the taxonomy of the optionally-executed block is:

  • No optional block,
  • Up to one optional block, and
  • Multiple optional blocks.

Requiring buildOptional to include buildEither might be a less surprising choice.


During the digging, I also found

func buildIf(_: Component?...) -> Component

It's an interesting idea to explore, that the author can opt-in to different transformation strategies based on the functions present, like:

  • buildOptional to use optional + injection tree,
  • buildEmpty to use injection tree only,
  • buildSelect(_: Components...) -> Component to combine all optional blocks using one function, etc.

buildSelect would be useful for cases like homogeneous blocks that needs to preserve the block identity, so we probably want to at least leave it as a possibility.


Btw, what do you think of ignoring post-buildExpression Void values. I don't know if it current type checks buildExpression before or after inserting buildBlock, so I don't know if it's viable. This is most definitely an exception to the rule, but it should provide a pretty good value.

Hmm, doesn't work well when the author defines buildExpression though (you still need to add (Void) -> Void one).

I agree that the demand exists, but a proven design does not. I referenced one potential design in the proposal, but is it the right one? If all people want is ArrayBuilder, then that FunctionBuilder protocol shouldn't be part of the proposal (and Either definitely shouldn't sneak in through this proposal). Perhaps we should do a standalone ArrayBuilder like this one:

@_functionBuilder
public enum ArrayBuilder<Expression> {
  public static func buildExpression(_ component: Expression) -> [Expression] {
    [ component ]
  }

  public static func buildBlock(_ components: [Expression]...) -> [Expression] {
    Array(components.joined())
  }

  public static func buildOptional(_ optional: [Expression]?) -> [Expression] {
    optional ?? []
  }

  public static func buildEither(first components: [Expression]) -> [Expression] {
    components
  }

  public static func buildEither(second components: [Expression]) -> [Expression] {
    components
  }

  public static func buildArray(_ components: [[Expression]]) -> [Expression] {
    Array(components.joined())
  }
}

... but this is less flexible. It doesn't look like the shape of the library API we end up here will influence the language proposal, and enough questions remain about the library that it's not worth tying the two together.

Doug

2 Likes

This was us thrashing around with an initial prototype implementation before we settled on buildOptional. It's only supported by the implementation for backwards compatibility.

I fundamentally think you're going in the wrong direction here. We have complaints upthread that function builders already have too much API surface in the ad hoc protocol; I'm strongly against making it any more complicated than it is.

It's possible, in the sense that we could have a more complicated expression that drops Void values from the arguments to buildBlock. It would be a pain to implement, and personally I think the

static func buildExpression(_: ()) -> Empty { ... }

Approach captures this use case well, without adding special cases to the model.

Doug

2 Likes

I think of buildEither as simpler than buildOptional since it guarantees that the user always provides a value. If the DSL in question doesn’t have optional values or a reasonable default value, you can’t implement buildOptional, but buildEither is still useful. (I do think the compiler should give a specific diagnostic if a client tries an if-without-else, suggesting to add the else block.)

2 Likes

On that note, the revision should probably also mention the additional restrictions introduced by that PR, namely, that buildBlock must be a static method and cannot be 1) a static var with function type, 2) a static var of a type which provides a callAsFunction method, or 3) an enum case.

Thanks @Douglas_Gregor — it was a bit tongue in cheek :smile:

The hierarchy is still weird though:

  • No optional block,
  • Exhaustive optional block,
  • Arbitrary optional block.

I think the second line X will always be weird no matter what it is, since we don't really have a construct that's half-way between the first and the third ones (I'm somewhat reluctant to say that switch is in this case). That's why I think requiring both buildOptional and buildEither to enable all branchings might be a more intuitive choice down the line.

Personally, I couldn't figure out how the half-way case could be useful, and there's no one that implements it either.

The transformation for Availability case seems to be missing. It does explain the use case, but I don't see whether it wraps the entire branching blocks, or only ones with #available.

I don't use much of #available, so I don't know its behavior very well. Is it possible for limited availability item to be in the else case, like with deprecation or something?

Speaking of, do we still need to support those cases, when a private feature becomes public? Since people needs to migrate them anyway.

It seems entirely sensible to me. Either your DSL only supports composition and not alternation or absences, or it also supports alternation but not absences, or it supports absences but not alternations, or it supports everything.

That said, I re-read the "no buildEither" behavior and I'm a little surprised that it unconditionally injects values into optionals, rather than having that depend on the exhaustiveness of the switch. Was that intentional, Doug? If that weren't there, then "no buildEither" builders would get alternation-without-absences for free as long as there's a common type. (Also there's both v and vCase and I think those are supposed to refer to the same thing.)

I think a DSL with a fixed number of elements could be interested in the "alternation but no absences" thing, like Rorschach. I am having trouble coming up with an "absences but no alternations" example…but that makes sense, since alternations could always be resolved outside the builder. (I think that's another reason to make the "no buildEither" case "just work".)

1 Like

We should also clarify how types propagate through variables:

@FooBuilder
func foo(@FooBuilder closure: () -> Foo) -> Foo { ... }

//  (() -> Foo) -> Foo
// NOT @FooBuilder (@FooBuilder () -> Foo) -> Foo
let a = ```

This would be akin to treating builder annotation as part of the function name, not function type. So maybe we'll need to permit it when referring to function names:

let a = foo(@FooBuilder _:)

We shouldn't allow builder annotation at non-argument location in the protocol requirement:

protocol Foo
  //   Bad             Ok
  //    |               |
  @FooBuilder func foo(@FooBuilder _: () -> Foo) -> Foo

One reason is that we never permit implementation details in the protocol requirements, even for property wrapper, which is half-implementation half-interface*. It calls into question of how much should be allowed in a protocol declaration, which I think should only be information relevant to the caller.

Another, more important reason is that it's confusing to the reader to omit a builder. It's fine for the writer to look up the protocol requirement and add the appropriate signature (sans builder annotation). On the reader side though, it's not clear why this compiles:

extension Conformer {
  func foo() -> Foo {
    foo1()
    foo2()
    if condition {
      foo3()
    }
  }
}

The reader needs to know that Conformer conforms to a protocol, that it requires foo, and that foo is annotated with @FooBuilder in the protocol requirement.

It's probably much less of a problem for well-known frameworks like SwiftUI. Even then, the fact that the obscurity of this depends on the framework popularity is somewhat unsettling.


* It could also be because of the implementation difficulty, nonetheless, that's the current behavior.

1 Like

General comments, mostly in agreement with what's been said:

  • If you implement buildExpression for some types, do expressions with other types just get passed through verbatim? I'd suggest "no"; if you want that behavior you can opt in with a fallback buildExpression that has an unconstrained generic parameter. You only get the "default" behavior if there are no overloads of the thing.

  • I agree that "function builder" isn't actually a good name for these, because that makes it sound like they build functions, and they don't. "Builder functions" isn't bad as a name for the feature, but the type that provides these methods isn't a function, so it'd be weird to see on an attribute. "Function transform" or "function transformer" seems most accurate, if not very interesting…but sometimes boring is good. (Adopting types would probably still be called "builders", a la ViewBuilder.)

  • As for the "what is a function builder" complaints, however, I think it's actually stated pretty well at the top of the proposal:

    This proposal describes function builders, a new feature which allows certain functions (specially-annotated, often via context) to implicitly build up a value from a sequence of components.

    The basic idea is that the results of the function's statements are collected using a builder type[.]

    In effect, this proposal allows the creation of a new class of embedded domain-specific languages in Swift by applying builder transformations to the statements of a function.

    That's what they are and what they do. They add logic to a function to collect the results of top-level expressions and build the return value out of that. This can be used in making declarative DSLs. What's missing?

  • I agree with @ctxppc: it is weird that the builder methods are static. I understand why the building isn't accumulating state with mutating methods like string interpolation does—it's so that the types flow through—but using instance methods instead would allow for some sort of configuration syntax even if we didn't add it now. (And inline parameters in the configuration syntax has precedent with property wrappers.)

  • I agree with @Lantua about dropping Void results by default. That would allow the use of assertions and assignments in the middle of a builder, making it more like "normal code with collected results". Having to make your component type optional just so you can filter out Void seems icky.

    Never is a bit trickier, because really I'd want Never in a branch to eliminate that branch. This would happen automatically with Never-as-bottom-type, but without that it'd be another special case to put in.

  • I also agree with @Lantua that inheriting a builder annotation from a protocol is too subtle, SwiftUI notwithstanding. The main thing we infer from protocols today is @objc, but that doesn't really change the meaning of the rest of the witness for a requirement—it's mostly just an additional capability. However, I do think we want the protocol author to be able to suggest an annotation, so that the compiler can help someone who forgot it (not to mention code completion). I just think it shouldn't be enforced…and that is a subtlety on the protocol side, the same way the default implementation for an associated type isn't a requirement.

  • I agree with @davedelong and others that having a bunch of optional syntactic customization points makes it hard for the compiler to help you define a function builder. However, I really do want them to stay optional, so for now I agree with @suyashsrijan as well that code completion is going to be our best way to combat that: if you're in a @functionBuilder type, code completion should give you templates for all the customization points.

I don't really like this direction for DSLs at all, but I don't think that opinion's really relevant given the positions of both Apple and the core team.

14 Likes

If buildExpression is defined, it always use buildExpression* and fails if there's no declaration with matched type.

I thought about it too that Never block should be untransformed. It's also impossible to emulate with the current design. The compiler will always try to buildExpression other expressions in the block.


* Edit: Fixed the link. The original link was two comments above what I meant to use.

2 Likes

If you have an alternative in mind that you’d be willing to briefly sketch out, I think that’s quite relevant to the proposal review.

3 Likes

I'm going to hide this because I don't think it changes anything about the review, but if you're interested here's my response:

I was still at Apple when the SwiftUI DSL was being designed, so I dropped in on some of the early conversations (though I was never a main participant). I had my chance to say this was overkill and overly complicated, but all the other ideas at the time (subscripts with "clever" signatures, or just plain initializers and array literals) were considered insufficient for various reasons:
  • Too many expressions to type-check at once, Swift's longstanding Achilles heel.

  • Preserving static types is a must for SwiftUI's update model to be efficient (this is also why opaque types were introduced when they were). I didn't have much insight into how much of a difference this made, and probably ought not to share it anyway if I did, but I do think that if variadic generics were already part of the language, the discussion might have been different even if the conclusion was the same.

  • Mixing parens and square brackets gets weird, since you have to match them but it's not always super obvious why one call uses square brackets and another uses parens when they're both just part of a builder DSL.

  • Can't use normal Swift syntax like if and for. I don't consider that convincing in general but admit that #available doesn't have great expression-level options today.

Harlan's HTML example in the proposal is reasonably compelling in showing how the linear form loses the structure but the non-linear form loses flexibility, and I don't have an immediate alternative to that. But it's not necessarily the problem I would choose to solve, and while I'm not sure if I want Rust's super-powerful macros in Swift, it's very possible that declarative DSLs would fit better in some other feature than the specific syntactic transformation that's described here.

EDIT: It's also entirely possible this is early-adoption wariness, and in another two years I'll feel the same as @JohnEstropia, where it feels totally sensible. That's actually a reason why I didn't endorse John's point about wanting an introducer keyword; I felt that way in the past, but know that such introducers can end up just feeling like noise once everyone's gotten used to the feature.

5 Likes
I want to discuss some `ArrayBuilder` shape. This could be split into a new thread if it's irrelevant.

We could make the ArrayBuilder a protocol to make it more customizable:

public protocol ArrayBuilder {
  associatedtype Element
  associatedtype Result = [Element]
  typealias Component = ArrayBuilderComponent<Element>

  static func finalize(_ elements: [Element]) -> Result
}

public enum ArrayBuilderComponent<Element> {
  case none, single(Element), nested([ArrayBuilderComponent])

  func merge(into result: inout [Element]) {
    switch self {
    case .none: break
    case let .single(element): result.append(element)
    case let .nested(blocks): blocks.forEach { $0.merge(into: &result) }
    }
  }
}

extension ArrayBuilder {
  static func convertToComponent(_ element: Element) -> Component { .single(element) }

  static func buildBlock(_ components: Component...) -> Component { .nested(components) }
  static func buildOptional(_ component: Component?) -> Component { component ?? .none }
  static func buildEither(first: Component) -> Component { first }
  static func buildEither(second: Component) -> Component { second }
  static func buildArray(_ components: [Component]) -> Component { .nested(components) }
  static func buildFinalResult(_ component: Component) -> Result {
    var result: [Element] = []
    component.merge(into: &result)
    return finalize(result)
  }
}

extension ArrayBuilder where Result == [Element] {
  static func buildExpression(_ element: Element) -> Component { convertToComponent(element) }
  static func finalize(_ elements: [Element]) -> Result { elements }
}

The conformer then only needs to define Element. If the Result is not [Element], it can customize buildExpression and buildFinalResult. There is also a convenience method called convertToComponent to help with buildExpressions:

// "Simple" case
@_functionBuilder
struct FooBuilder: ArrayBuilder {
  typealias Element = Int
}

// "Complex" case
@_functionBuilder
struct BarBuilder: ArrayBuilder {
  static func buldExpression(_ element: Element) -> Component {
    convertToComponent(element)
  }
  static func finalize(_ elements: [Int]) -> String {
    "\(elements.reduce(0, +))"
  }
}

I also use merge(into:) strategy for merging instead of flatMap when finalizing, and do it at the very end instead of eagerly. Not sure if there's any noticeable difference, though.

1 Like

Sorry for repeated bikeshedding, but the current naming really, really bothers me.

We're not building X. We're building components using X. We're building component from Expression, etc. So if we adhere to factory method naming, it would be much closer to:

createComponent(expression:)
createComponent(subcomponents:)
createFinalResult(component:)

createOptional(component:)
createComponent(firstBranch:)
createComponent(secondBranch:)

createComponent(loopComponents:)

createComponent(limitAvailibilityComponent:)

If we switch from "build component using X" to "transform X (into component)" as a few people have suggested, we will get a much Swiftier name:

transform(expression:)
transformBlock(components:) 
transform(finalComponent:)

transform(optional component:)
transform(firstBranch:)
transform(secondBranch:)

transformLoop(components:)

transform(limitedlyAvailable component:)

And we should also rename @functionBuilder to @functionTransformer or even @closureTransformer.

8 Likes

Mm, I agree that we're not building the expression, but the way in which the function* is "transformed" has nothing to do with the way the expression is "transformed", so it'd be nicer to have different language to talk about the two of them. (I think the original intent of "build" is "build from expression", but it's not great.) I don't think I'd really say the intermediate results are being "transformed" into an aggregate result, anyway, except in the sense that every pure function is a transformation.

I do find "component" to be a little weird; it's not a term we use anywhere else in the language yet, and it's not like you're limited to a single component type. But I don't have a ready alternative.

I think part of the problem in picking a verb (create, make, build, transform) is that the verb isn't really the important part; these could be handle(expression:) and handle(block:) and such and it wouldn't matter. Maybe that suggests a noun-ish naming, like your names but dropping "create"?

* "function", not "closure", since it also applies to standalone function and accessor declarations. Same reason for why @closureTransformer doesn't really seem apt.

2 Likes

The current implementation allows a build method to take extra arguments with default values. Example:

@_functionBuilder
struct FB {
    static func buildExpression(_ label: String, file: StaticString = #file, line: UInt = #line) -> Void {
        print("\(String(reflecting: label)) at \(file):\(line)")
    }

    static func buildBlock(_: Void...) -> Void { }
}

@FB
func test() {
    "hello"
    "world"
}

test()

This is great but I don't know if it's intentional. If so, it would be nice for it to be documented.

5 Likes