Function builders

If you want a block to contain exactly four values, or to require the first value to be a specific type, just define buildBlock to only accept that;

I think this, together with preserving generic type of expressions, is the best argument for this proposal.

For example, it allows HTMLBuilder to require - at compile time - the first expression to be a "head" element. And for there to be only one of them.

4 Likes

I know that some of those have already been raised but want to reiterate to emphasis that these points are a big deal imho and we should find a solution to them before going further

So even if I like the idea of allowing a DSL, my main concerns in the implementation of such a DSL by this proposal are:

  • there's no way to provide context-specific expressions only usable from a given builder. e.g. in the HTMLBuilder example, p and div are global functions; making them instance functions that would only be declared on instances of Body or some similar idea would help scope both their declaration and the places where they're allowed to be called (e.g. prevent p inside a title for this particular example) but to my understanding the current design of Function Builders would not allow me to design DSLs (like HTMLBuilder or any other) that behave that way
  • Since we don't have variadic generics yet, it leads to things like the ViewBuilder example posted by @taylorswift is very worrying to me (especially since, suddenly 10 subviews are ok but 11 aren't)
  • the fact that you can't really use any type of control flow (e.g. for is not yet supported, guard either, ...) – and that we (as a SE community) will have to redefine the behavior of each control-flow operator before it could be supported in future iterations of this feature – is also counter-intuitive and limiting
  • The fact that a simpler solution like yield was not considered – or an implicit builder closure parameter with any unused value in the closure transformed to builder.addExpression() – would feel like a simpler model to me, at least to begin with

Overall I think a solution more close to what @trs (Function builders - #79 by trs comment) or @hartbit (Function builders - #104 by hartbit), suggested above would be preferable, easier to understand, and would also allow to solve the global namespace problem (only declaring p as an instance function on body for example) – which is probably my main concern.

19 Likes

To solve the global namespace pollution problem:

We could add a type inside the builder that acts as a namespace and holds the functions that make up our DSL. Here I named this new namespace Term:

struct HTMLNode: HTML {
    var tag: String
    var attributes: [String: String] = [:]
    var children: [HTML] = []

    func renderAsHTML(into stream: inout HTMLOutputStream) {
        ...
    }
}

extension HTMLNode {
    struct Term {
        static func div(@HTMLBuilder makeChildren: () -> [HTMLNode]) -> HTMLNode {
            return HTMLNode(tag: "div", attributes: [:], children: makeChildren())
        }
        static var ol: HTMLNode { return HTMLNode(tag: "ol") }
        static let br = HTMLNode(tag: "br")
    }
}

@_functionBuilder
struct HTMLBuilder {
    typealias Expression = HTMLNode
    typealias Component = [HTMLNode]
    typealias Term = HTMLNode.Term

    static func buildExpression(_ expression: Expression) -> Component {
        return [expression]
    }

    ...
}

Example:

@HTMLBuilder
func build() {
    div { ol }
    br
}

// This could be desugared into this:

func build() -> [HTMLNode] {
    // aliasing closures
    let div = HTMLBuilder.Term.div
    let ol = { HTMLBuilder.Term.ol }
    let br = HTMLBuilder.Term.br


    let _a = HTMLBuilder.buildExpression(div { HTMLBuilder.buildExpression(ol()) })
    let _b = HTMLBuilder.buildExpression(br)
    return HTMLBuilder.buildBlock(_a, _b)
}

The proposed desugar works with the beta of Xcode 11, so no new language features are needed (of course the current implementation of Function builders would need to be adapted to generate the proposed desugared code)

Semantics:

For each function (properties also?) of the Term struct, the compiler generates an aliasing closure.

With this approach the current code generation implementation needs only to be extended to generate the aliasing closures and place them before the rest of the generated code. We are effectively bringing the identifiers defined in Term struct into the @HTMLBuilder function namespace.

Any other identifier used won't have an aliasing closure so it will be normally accessed.

Notes:

This feature also makes the discovering of the DSL terms more discoverable, since they are grouped in a known namespace.

5 Likes

I think the namespaced solution should also be able to handle Literal (String, Int, etc.), and include support for types outside of that namespace.

We can do 2-step lookup (within namespace, then general context), but we should make it clear that that's what gonna happen.

Yes, this is how it is supposed to work.

My concern about intercepting unqualified lookup in DSLs is about feasibility, not the exact language design. I’d like to have it, but if it adding it turns out to be a major project that would delay the feature out of 5.1, I think we can live without it for awhile.

1 Like

It's hard to comment on something I've only seen very briefly, because I might get used to this quickly, but at first glance, I'm worried about this:

This feature changes how the code inside the passed block will be interpreted on a language level, and it's completely unclear by just looking at the code for someone not familiar with the API. When I use an unknown API and pass a block to it, I might not know what the called method will do with it, when / how / how often it will call the code inside the block, but I can at least expect that when it does, the statements in the block will be called one by one without any modification, and then the last one with return will be the only thing returned from the block. This feature breaks this assumption, and now when using an unknown API I will need to look at its code or documentation to not only know what it will do with the code in the block, but even how the code in the block will work.

A very similar thing happens with Property Wrappers, where adding some @Attribute changes what a property declaration means, but the important difference is that the caller adds that attribute at the call site, so it's clear to them that this property does something different than it would without the attribute. Here, there is no such indication.

I totally understand the desire for readability in this case and I wouldn't want such a crucial API like SwiftUI to become more noisy because of this, but I'm wondering if we could come up with some minimal change at the call site that doesn't decrease the clarity, but still makes it clear just by looking the code (but without knowing the API) that something in this code works differently than in normal Swift code? Like, some extra symbol before/after/instead of the braces, for example?

5 Likes

Agreed. I’m taking some time to digest all of this week’s news before I comment on this design.

In the receiver closure thread we were talking about using a $ prefix on the closure. The proposal did address the downsides of this in a DSL like SwiftUI. I haven’t decided how I feel about it overall yet, but omitting usage site syntax for a semantic change like this does feel like a different direction than the rest of Swift has taken. That makes me a little bit uncomfortable, but there are certainly advantages to it as well.

If we ship 5.1 without it and later add support for it how would that impact the evolution of SwiftUI? How much of it’s API is currently living in global scope because it has to, not because it wants to (I’m still getting up to speed on it so I’m not sure of the answer). This seems like an important question.

3 Likes

I agree that's an important question. I don't think SwiftUI has very many "primitives" like this that I think they wouldn't want in the global namespace, but I can try to ask someone who's more of an expert in the framework to chime in.

3 Likes

I'm curious what the current hurdles are for getting to a point where for statements could be supported. The only real-world test case for this feature right now is SwiftUI, which already has the ForEach type to work around the lack of some sort of for support. As a concrete example, what would be the barrier to a design like:

for chapter in 1...135 {
    p { "Chapter \(chapter): \(chapterTitles[chapter])" }
    p { chapterContents[chapter] }
}

transforms to:

var v0: [HTML]
for i in 1...135 {
    let v1 = HTMLBuilder.buildExpression(p { "Chapter \(chapter): \(chapterTitles[chapter])" })
    let v2 = HTMLBuilder.buildExpression(p { chapterContents[chapter] })
    let v3 = HTMLBuilder.buildBlock(v1, v2) // or, buildFor(...) or something of the sort
    v0.append(v3)
}
return HTMLBuilder.buildBlock(v0)

Since break and continue are already prohibited, we would not have to worry about not reaching the end of the for block, right?

2 Likes

I'm really positive about this proposal. Making Swift a better language for DSLs is a great improvement.
I'm not really concern about the proposal introducing something that developers will find weird to start with, as that's the whole point of a DSL. And I really dig the compile time rewritting that this introduces.
It would be great to have some additions like support for loops and switches, but if we can add it soon is fine.

My only concern is with global namespace pollution. I think that the other thread about changing the binding of self in closures could be revisited and aligned with this.

And thanks for the great writeup in the proposal. It gives a lot of details and also explains what a DSL is and how it should be judged. Really nice!

3 Likes

A thought on sidestepping the "horrendous buildBlock(_:_:_:_:_:_:_:_:_:_:)", variadic generics, etc.:

What if instead of collecting the statement values into locals and passing them to a factory function, the values were accumulated in a reducer-type pattern, but in which the "reduce" operation allows generic (type-based) dispatch? That is, the example would be transformed into something like:

let v_0 = div.initialValue
let v_1 = useChapterTitles? v_0.build(h1("1. Loomings.")) : v_0
let v_2 = v_1.build(p("Call me Ishmael")
let v_3 = v_2.build(p("There is now…")
return v_3.finish()

This has a number of nice properties:

  • No new type-level shenanigans are required in the Swift compiler.
  • The type of the accumulator is determined by the implementor; it can be whatever makes sense for the problem domain. An HTML or JSON builder might just pass a list of items.
  • The type of each statement can vary, as long as a matching implementation of .build() is available.
  • (Possibly the largest advantage) The type of the v_n values can vary as well, based on the return type of .build(). This is a rather powerful feature; for example, it can be used to statically enforce a schema on the created value (a DFA that describes the schema can be embedded into the type system). In the above case, the type system enforces that v_1 has the same type whether or not useChapterTitles is true.
16 Likes

The idea of rebinding self reminds me of some of the work I did for the prototype implementation to allow implicit self in escaping closures. It explicitly disallows implicit self when you have a capture of the form [self = other], but maybe that restriction could be removed...

3 Likes

It seems that the static nature of the methods in function builders is necessary to enable generic type inference, so that calls to different build functions can be on HTMLBuilder types with different generic parameters, so I think your suggestion would be better modeled as:

let v_0 = div.initialValue
let v_1 = useChapterTitles? HTMLBuilder.buildExpression(h1("1. Loomings.")) : v_0
let v_2 = HTMLBuilder.buildExpression(p("Call me Ishmael")
let v_3 = HTMLBuilder.buildExpression(p("There is now…")

let b_1 = HTMLBuilder.buildBlockPartial(nil, v_1)
let b_2 = HTMLBuilder.buildBlockPartial(b_1, v_2)
return HTMLBuilder.buildBlock(b_2, v_3)

or progressively build as in your example as:

let v_0 = div.initialValue
let v_1 = useChapterTitles? HTMLBuilder.buildExpression(v_0, h1("1. Loomings.")) : v_0
let v_2 = HTMLBuilder.buildExpression(v_1, p("Call me Ishmael")
let v_3 = HTMLBuilder.buildExpression(v_2, p("There is now…")

return HTMLBuilder.buildBlock(v_3)
1 Like

There aren't that many hurdles, really: it's a straightforward transformation. However, we'd want to use something different from buildBlock, because buildBlock is meant to take N independent expressions (which may have N different types, where N is fixed), whereas the result of a for..in loop will be (e.g.) an array of some variable number of elements.

Doug

Doug

6 Likes

I see another large advantage related to this!

This makes it really easy for the compiler to point out exactly which line has a forbidden statement, something that I think might not be possible with buildBlock!

I'll write my explanation in terms of reduce(_ state: State, _ element: Element) -> State, which takes the previous state (or Void if there is none), the value of the current element, and returns the next state.

e.g, with buildBlock:

struct Builder {
  static func buildBlock(_ a: String...) -> [String] { ... }
  static func buildBlock(_ a: Int...) -> [Int] { ... }
}

{
  "hello"
  "world"
  3
  "foo"
}

// All the compiler can say in this case is that there aren't any overrides
// matching (String, String, Int, String). It can't point at a line that is
// at fault.

with reduce:

struct Builder {
  static func reduce(_ state: Void, _ element: String) -> [String] { ... }
  static func reduce(_ state: Void, _ element: Int) -> [Int] { ... }
  static func reduce(_ state: [String], _ element: String) -> [String] { ... }
  static func reduce(_ state: [Int], _ element: Int) -> [Int] { ... }
}

{
  "hello"
  "world"
  3  // error: no `reduce` with arguments `([String], Int)`
  "foo"
}
6 Likes

It would also enable something like what @gwendal.roue was looking for with runtime validation that can be attached to the point where an invalid partial result was actually built.

2 Likes

The statements in the closure will be called one by one without any modification. The difference with the DSL is that the results of each expression produced are fed into the function builder to produce a combined result. Your assumptions about how the code in the closure will execute are still valid.

As for the return statement, let's say we wrote DSL code like this:

div {
  p { "hello" }
  p { "world" }
}

but the trailing closure parameter to div isn't using a function builder. You'll get warnings about "unused value" for both p expressions, and probably an error that you didn't return anything. There's no difference from how Swift has always worked.

The case that speaks more to your concern is when div is using a function builder, and you add a return:

div {
  p { "hello" }
  return p { "world" }
}

Now you'll get an "unused value" warning from the value that will get dropped, because you're no longer using the function builder (you've taking over the return yourself). That's fine--it lets you be more imperative when you want it--and the tools help you know when you've tried to mix the declarative and imperative styles.

Doug

7 Likes

I think @discardableResult leads to confusion.
In the following code, "m1" and "m2" are displayed.
Some (almost?) swift users (including me) may not be concerned with the return value of removeFirst.
I hope, there need a compiler warning.

struct ContentView : View {
    var body: some View {
        var messages = [
            Text("m1"),
            Text("m2"),
        ]
        return VStack {
            messages.removeFirst()
            messages[0]
        }
    }
}

I share the same concern with you. I think the result should be emitted regardless, but it also emit the warning.

var messages = [ Text("m1"), Text("m2"), Text("m3"), ]
return VStack {
    messages.removeFirst() // emit Text, + "@discardableResult used in builder" warning
    _ = messages.removeFirst() // not emitting
    (message.removeFirst()) // emit Text, silencing warning
}
1 Like