Function builders

[Jumhyn (Frederick Kellison-Linn)]

@Alejandro_Martinez

I think that the other thread about changing the binding of self in closures could be revisited and aligned with this.

The idea of rebinding self reminds me of some of the work I did for the prototype implementation to allow implicit self in escaping closures. It explicitly disallows implicit self when you have a capture of the form [self = other], but maybe that restriction could be removed...

[Jumhyn (Frederick Kellison-Linn)]

It seems that the static nature of the methods in function builders is necessary to enable generic type inference, so that calls to different build functions can be on HTMLBuilder types with different generic parameters, so I think your suggestion would be better modeled as:

let v_0 = div.initialValue
let v_1 = useChapterTitles? HTMLBuilder.buildExpression(h1("1. Loomings.")) : v_0
let v_2 = HTMLBuilder.buildExpression(p("Call me Ishmael")
let v_3 = HTMLBuilder.buildExpression(p("There is now…")

let b_1 = HTMLBuilder.buildBlockPartial(nil, v_1)
let b_2 = HTMLBuilder.buildBlockPartial(b_1, v_2)
return HTMLBuilder.buildBlock(b_2, v_3)

or progressively build as in your example as:

let v_0 = div.initialValue
let v_1 = useChapterTitles? HTMLBuilder.buildExpression(v_0, h1("1. Loomings.")) : v_0
let v_2 = HTMLBuilder.buildExpression(v_1, p("Call me Ishmael")
let v_3 = HTMLBuilder.buildExpression(v_2, p("There is now…")

return HTMLBuilder.buildBlock(v_3)

[Douglas_Gregor]

@Jumhyn

I'm curious what the current hurdles are for getting to a point where for statements could be supported.

There aren't that many hurdles, really: it's a straightforward transformation. However, we'd want to use something different from buildBlock, because buildBlock is meant to take N independent expressions (which may have N different types, where N is fixed), whereas the result of a for..in loop will be (e.g.) an array of some variable number of elements.

Doug

Doug

[marcrasi]

@wiml

What if instead of collecting the statement values into locals and passing them to a factory function, the values were accumulated in a reducer-type pattern, but in which the "reduce" operation allows generic (type-based) dispatch?

@wiml

(Possibly the largest advantage) The type of the v_n values can vary as well, based on the return type of .build() . This is a rather powerful feature; for example, it can be used to statically enforce a schema on the created value (a DFA that describes the schema can be embedded into the type system). In the above case, the type system enforces that v_1 has the same type whether or not useChapterTitles is true.

I see another large advantage related to this!

This makes it really easy for the compiler to point out exactly which line has a forbidden statement, something that I think might not be possible with buildBlock!

I'll write my explanation in terms of reduce(_ state: State, _ element: Element) -> State, which takes the previous state (or Void if there is none), the value of the current element, and returns the next state.

e.g, with buildBlock:

struct Builder {
  static func buildBlock(_ a: String...) -> [String] { ... }
  static func buildBlock(_ a: Int...) -> [Int] { ... }
}

{
  "hello"
  "world"
  3
  "foo"
}

// All the compiler can say in this case is that there aren't any overrides
// matching (String, String, Int, String). It can't point at a line that is
// at fault.

with reduce:

struct Builder {
  static func reduce(_ state: Void, _ element: String) -> [String] { ... }
  static func reduce(_ state: Void, _ element: Int) -> [Int] { ... }
  static func reduce(_ state: [String], _ element: String) -> [String] { ... }
  static func reduce(_ state: [Int], _ element: Int) -> [Int] { ... }
}

{
  "hello"
  "world"
  3  // error: no `reduce` with arguments `([String], Int)`
  "foo"
}

[Jumhyn (Frederick Kellison-Linn)]

@marcrasi

This makes it really easy for the compiler to point out exactly which line has a forbidden statement, something that I think might not be possible with buildBlock !

It would also enable something like what @gwendal.roue was looking for with runtime validation that can be attached to the point where an invalid partial result was actually built.

[Douglas_Gregor]

@Jakub_Suder

This feature changes how the code inside the passed block will be interpreted on a language level, and it's completely unclear by just looking at the code for someone not familiar with the API. When I use an unknown API and pass a block to it, I might not know what the called method will do with it, when / how / how often it will call the code inside the block, but I can at least expect that when it does, the statements in the block will be called one by one without any modification, and then the last one with return will be the only thing returned from the block. This feature breaks this assumption, and now when using an unknown API I will need to look at its code or documentation to not only know what it will do with the code in the block, but even how the code in the block will work .

The statements in the closure will be called one by one without any modification. The difference with the DSL is that the results of each expression produced are fed into the function builder to produce a combined result. Your assumptions about how the code in the closure will execute are still valid.

As for the return statement, let's say we wrote DSL code like this:

div {
  p { "hello" }
  p { "world" }
}

but the trailing closure parameter to div isn't using a function builder. You'll get warnings about "unused value" for both p expressions, and probably an error that you didn't return anything. There's no difference from how Swift has always worked.

The case that speaks more to your concern is when div is using a function builder, and you add a return:

div {
  p { "hello" }
  return p { "world" }
}

Now you'll get an "unused value" warning from the value that will get dropped, because you're no longer using the function builder (you've taking over the return yourself). That's fine--it lets you be more imperative when you want it--and the tools help you know when you've tried to mix the declarative and imperative styles.

Doug

[nukka123]

I think @discardableResult leads to confusion.
In the following code, "m1" and "m2" are displayed.
Some (almost?) swift users (including me) may not be concerned with the return value of removeFirst.
I hope, there need a compiler warning.

struct ContentView : View {
    var body: some View {
        var messages = [
            Text("m1"),
            Text("m2"),
        ]
        return VStack {
            messages.removeFirst()
            messages[0]
        }
    }
}

[Lantua]

@nukka123

I think @discardableResult leads to confusion.

I share the same concern with you. I think the result should be emitted regardless, but it also emit the warning.

var messages = [ Text("m1"), Text("m2"), Text("m3"), ]
return VStack {
    messages.removeFirst() // emit Text, + "@discardableResult used in builder" warning
    _ = messages.removeFirst() // not emitting
    (message.removeFirst()) // emit Text, silencing warning
}

[Lantua]

@wiml

What if instead of collecting the statement values into locals and passing them to a factory function

It’s slightly more restrictive than the current design.
The order of generating type must form a regular language for new design.

I don’t think it’s overly restrictive. I kinda like this level of restriction really. It could help make the compiler decide whether or not it’s a valid block much faster. But I can see that for more complex DSL, the author of the library can get real creative.

EDIT:
Now that I think about it, Swift’s function signature is already restricted to be regular.
So I suppose it’s not any more restrictive that current design.

[Jakub_Suder]

@Douglas_Gregor

As for the return statement, let's say we wrote DSL code like this:

div {
  p { "hello" }
  p { "world" }
}

but the trailing closure parameter to div isn't using a function builder. You'll get warnings about "unused value" for both p expressions, and probably an error that you didn't return anything. There's no difference from how Swift has always worked.

What I meant was: when I join a project that uses some unknown library with an API like this (that I'm not familiar with) that does use a function builder, and I look at this code, my understanding is that there are some methods div() and p() that take a block and do something with it, I don't know what, but regardless of what they do, when the outer block is called:

• the method p will be called once with a { "hello" } block and the result of that call will be ignored
• then the method p will be called again with a { "world" } block and the result of that call will be ignored again
• nothing will be returned from the block

Because that's how normal Swift code works. I may not know what the methods div and p actually do, but at least I do know this much. But if it turns out that the results of calls to p that don't get assigned to anything aren't in fact ignored, and that div does in fact get a result from this block even if there's no return keyword - then it suddenly feels like I'm coding in a slightly different language.

I would also assume, looking at this code without any context, that if I write:

for x in ["hello", "world"] { p { x }}

(or any other kind of control statement or a mix of them) it would have the same result. I will not know that I'm not supposed to use for/switch/etc. inside the block if I'm not aware that the block is passed to a builder, having checked the library's documentation first.

[Karl]

After having a bit more time to consider the specifics of this proposal, I have some more comments:

• Firstly, I think the name "function builders" is confusing - this isn't really about building functions, but rather about special functions that are more amenable to the builder pattern. builder functions, if you will.

• Secondly, I think the examples we have seen are overly simplistic (e.g. Apple's SwiftUI examples). In reality, your Views will not only define their layout hierarchy; there may be methods dealing with user interaction, animations and drawing, and all sorts of other code. What does it look like when some parts of your code behave differently to others (this is the "embedded" part)? Or when certain control-flow statements are allowed in some contexts but not in other, identical-looking contexts?

  By its nature, a DSL is not exactly Swift and doesn't follow the regular language rules. When I look at the examples, there is just a lot of unwritten context which is vitally important to somebody reading the code. I'm thinking of an overworked programmer, hacking away late at night and whose view-code is a lot more complex than the simple examples we have seen. They will have to be switching contexts (or brain modes) regularly.

• Lastly, going back to the problem of making functions that are more amenable to the "builder pattern", it seems to me that we want a builder-function which spits-out children to the builder and then resumes, until the entire tree is built. So basically, we want coroutines:

  var body: some View {
    VStack {
      guard let user = self.user else { yield LoginButton() }
      yield HStack {
        if let pic = user.profilePicture { yield Image(pic) }
        yield Text(user.name)
      }
    }
  }
  

  This turns the implicit thing (the unused results of expressions) in to an explicit thing. It would seem to remove the limitations imposed by not having variadic generics (because each builder receives its children one-at-a-time), remove the need for the builder to handle control flow and make the whole thing fit more nicely in the context of actual Swift.

  Of course, it does introduce a bit of punctuation. We need to consider where we draw the line with this feature: if we wanted a DSL that was exactly like, say, HTML, that already exists - it's called HTML! Why not just embed real HTML in to your Swift code? There is some philosophical debate about what exactly an eDSL is (some would say that any sufficiently complex API is an eDSL), but IMO, the whole point of embedding it is that it should be reasonably like the host language.

  This thing is a little bit like Swift, but it's different enough that I think it will be confusing and difficult to use in the real world.

[Lantua]

I’ve been contemplating about the mechanism for transforming the Expression.
So far 2 (2.5?) mechanisms has popped up;

• Collecting method that collects all Components at the end of closure (currently the proposed design), and
• Iterative method that keeps the current state of builder (either via builder instance, or static function’s return type) and change that state as new object emits.

Iterative method using builder instance doesn’t catch much of my attention since it lacks the type checking that the other 2 methods offer, which to me is a big plus.

So I would like to share my thought in comparing these two

1. Iterative Method is nice with control flow. It works almost naturally with looping. We could add buildBeginLoop, buildEndLoop for loop enclosure, and buildNone for block that generates no Expression. This also play pretty well into return, break, continue so long as the branching paths has the same Type.

   Collecting method seems to struggle somewhat with looping, we can either have buildLoop(_: [Component]) -> Component for dynamic loop, but I’m not sure How I feel about for/while being something akin to an expression. We could also flatten the result within the block for loop that we can unroll.

2. Collecting method compose different structures nicely. The exemplar buildFunction Generally is of the form buildFunction(_: Component...) -> ReturnValue, but it can take other signature (IIUC; at least this is not being corrected where mentioned). Say, you want structure A to be of either the form B C D, or B E F then you can declare

   buildFunction(_: B, _: C, _: D) -> X
   buildFunction(_: B, _: E, _: F) -> X
   

   If Iterative method is to provide this level of type checking, it’d be much harder (tedious?) since author must keep track of the state machine oneself.

   buildFunction(partial: PartialResult0, _: B) -> PR_B
   buildFunction(partial: PR_B, _: C) -> PR_BC
   buildFunction(partial: PR_BC, _: D) -> PR_BCD
   ...
   

   This shouldn’t be much of a problem if the struct has simple grammar (as I mentioned before, both methods permits type order of any regular language).
   Depending on the scope of eDSL of this feature, it may or may not be a big hurdle for Iterative method.

[DeFrenZ (Davide De Franceschi)]

@skagedal suggested elsewhere (maybe jokingly? But I think it's interesting) that we could use yield-style approach, but with a • operator, so it looks like

func body() -> some View {
  • div {
    • p { "hello" }
    • p { "world" }
  }
}

and I would like as well some annotation on the function itself, like it's necessary for mutating at the moment. e.g. builder func body() -> some View { ... }

[jayton (Jens Ayton)]

Quick summary of the pitch’s motivation:

• Swift array literals have a problem with constrained heterogeneous types
• Also we’d like to allow local variables and conditions in our list-like constructs

If we assume that Sufficiently Generalized Existentials could solve the first part, we’re left with a seemingly obvious question: why can’t you allow local variables and conditions inside collection literals? Is it because ALGOL didn’t?

Such a feature would obviously have a wider set of use cases than ones where a DSL is reasonable (the pitch describes the tradeoffs in such a decision very well). It would also remove a class of thorny design decisions: this list-like thing might sometimes have conditional members, so should I take a collection here, or a builder, or maybe both?

I recognize that SwiftUI wants a solution which reflects the full generated structure at the type level in order to do build-time structural optimizations. However, this use case is not emphasized in the pitch, and with no indication of how great an advantage it will give, it cannot reasonably be evaluated as an argument for a more complex, less orthogonal feature.

---

Given that a change in direction is unlikely, I do have some comments on the pitched solution:

• I think the namespace issue is a bigger issue than John does. SwiftUI ducks it by using constructors as it terms, but using functions as in the HTTP example is very reasonable, yet littering the global namespace with functions violates the API Design Guidelines.
  I have some amount of sympathy for those labouring under Apple’s product cycle, but I would like us to see this as a known defect that should be addressed soon. (I was going to suggest a design, but it’s basically what ferranpujolcamins said).

• I’m worried about confusion that might arise when people accidentally add return statements to closures that are intended to be builders. I’ve already seen what seems to be an example, although I’m told it’s not.
  In general, I worry about the difficulty of generating good diagnostics for builder errors. I’m disinclined to accept “we’ll fix it later” here, given that type inference errors in Swift are still so bad that TI is more of a time sink than a benefit.

• Several commenters have suggested introducing builders with a specific keyword or annotation.
  The pitch brushes this off by saying that if you do this, you must be explicit about all nested builders and their types, and this is too great a syntactic burden. This is a false dilemma; if no annotation is OK, then an annotation with limited information must also be at least as OK.
  A build keyword at the top level would be enough to tell human readers that a special thing is happening. Not specifying which builder is in use is just a case of implicit type context, which is not a new concept in Swift.

[plorenzi (Pierre Lorenzi)]

The idea of coroutines is interesting. To make it more lightweight, it could be combined it with a block notation such as {: }, that would yield all the values declared inside.

The example would become:

func body() -> some View {:
  div {:
    p {: "hello" }
    p {: "world" }
  }
}

It is still magical but more general.

[lmcd (Lee M)]

Edit: I've moved some ideas into a new thread: @builderOperator attribute

[lmcd (Lee M)]

I also have an example that can produce deeply nested HTML structures with the expanded version of this syntax. Works just fine. If anyone would like me to expand on this, let me know.

[jayton (Jens Ayton)]

Forgot to say: as others have pointed out, Kotlin’s receiver functions can be used to very similar effect, and deserve some Alternatives Considered love – especially as they have already been pitched for Swift.

[JJJ (Jens Jakob Jensen)]

Just thinking out loud, I guess, but an alternative approach could perhaps be (mis-)using default arguments:

func div(makeChildren: (builder) -> Void) -> HTMLNode { ... }

func p(builder: Builder = implicit, ...) { ... }

div { builder in
  implicit builder

  if useChapterTitles {
    h1("1. Loomings.")
  }
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

Where implicit is a new keyword, and arguments defaulting to implicit only has a default if the innermost scope has an implicit declaration.

This would make it clear that the rest of the current scope includes magic

To me the @functionBuilder attribute is like user defined operators. It's just even more hidden and magic. I think it would make Swift code more readable if there was a clear marker that the following code redefines the usual rules.

[masters3d]

@jayton

A build keyword at the top level would be enough to tell human readers that a special thing is happening. Not specifying which builder is in use is just a case of implicit type context, which is not a new concept in Swift.

I am going to agree with this. If I am reading code it is not immediately obvious that I am reading a function builder. In the case of UI code of course that’s not the case but if this feature gets used for other cases then it will be confusing. I guess the culprit here is the actual trailing closure. I am thinking about the SwiftUI case. I much rather have a method call bodyBuilder than a property just name body. A bodyBuilder method would have to be marked with the function builder annotation for example.