Function builders

Ah I missed that, I haven't read the actual draft pitch yet... But either way I like the @builder(...) style syntax for this.

I'll read the whole draft now :)

Edit: Having read the proposal I now like the @HTMLBuilder syntax having more understanding of what is actually going on with this pitch.

1 Like

It’s not a long-term problem

otoh most long-term problems start with that kind of thinking

4 Likes

Remember that we're not actually designing HTMLBuilder.

Can you explain why you prefer @builder(HTMLBuilder) here? Just general resistance to custom attributes, or do you feel that custom attributes are a poor match for this?

1 Like

There are several reasons I didn't use SwiftUI as the example in the proposal. The type-preservation that SwiftUI needs adds quite a lot of complexity both to the exposition and to the implementation. Most DSLs will not need it and will be just fine with type erasure either via an enum or a protocol type; in either case, ordinary variadic arguments on buildBlock will be fine.

I do not think it makes sense to commit SwiftUI to an inferior language solution because of an implementation detail that we know exactly how to address in the medium term.

11 Likes

Can you add to the proposal the reasoning to use buildEither(...) instead of pushing for if/switch expressions? That part of the proposal trying to explain buildEither(...) was hard for me to follow.

2 Likes

Sure. I understand this is subtle. The answer is, again, type preservation: an if/switch expression has to require all of its cases to return the same type, but if the DSL wants the result type to reflect the structure of its component types, it's obviously unreasonable to require all the cases to have identical structure. buildEither is effectively injecting into one side or the other of an Either type without actually requiring such a type to exist; if it does exist (as it does in SwiftUI), it can carry the structure of the alternatives in its type arguments. I'll revise the proposal to make that clear.

5 Likes

Overall, I like this feature; I don't think it feels too magical and can see it being very powerful, especially once variadic generics are in place.

I do have one concern, though. The proposal says of having annotations for the top level call site:

  • Requiring an annotation on the outermost function and then propagating it to lexically-nested closures avoids “attribute overload” but is quite limiting on the DSL. For one, there may be compelling reasons for the DSL to use non-DSL functions within its body; for example, a UI hierarchy might want to directly define handler functions for various events. This suggests that there would have to be a way of turning off the lexical propagation (which would be error-prone in much the same way as non-propagation would be). Second, a DSL might actually consist of multiple similar function builder types used in different positions; this sort of implementation detail should not be allowed to become a serious usability problem for clients.

While I can see how this would apply with e.g. an @HTMLBuilder attribute, I feel as though having some signifier to specify that a function-builder context is being entered would still be useful.

Has having a generic e.g. build keyword (similar to try) been considered, to signify that a function-builder context is being entered? Within the scope of the context, build would be implicitly prefixed to each expression; it would have no effect on functions/closures that do not take a @functionBuilder type (similar to how try has no effect on functions that do not throw other than emitting a warning).

That would transform the example code into:

build div {
  if useChapterTitles {
    h1(chapter + "1. Loomings.")
  }
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

I feel that the (presumably) syntax-highlighted build keyword would aid clarity in mixed-paradigm codebases.

8 Likes

Hi John,

This is a super interesting proposal. I am so happy to see a direction that takes eDSL's seriously - this is a very exciting opportunity for Swift, one that could solve a huge range of issues and problems. It could even potentially get us to the golden world where all "statements" are just standard library features with a default statement rewrite rule in the standard prelude / stdlib.

Stepping back from that beautiful world, the quote above highlights an interesting point. I'm still trying to understand the feature itself (is there a patch implementing the feature available somewhere?), but it seems like there are two things going on here:

  1. You're providing a very general statement rewriting facility, where a syntactic transformation is changing the statement AST into a series of function calls.

  2. You're transforming if/switch "into a single result value that becomes the return value of the current function"

Have you considered decomposing these into two separate and orthogonal -- but composable features? One that allows if/switch to return values (this is something that has been highly requested for a long time) and one that allows "function builders" to transform the AST into overloadable function calls (e.g. along the lines of tf autograph)?

The composition of these two features seem like they'd be far more powerful and also simpler than the feature as proposed. I'm very interested in allowing this to eventually feature creep towards defining CUDA kernels and other accelerator functions for machine learning applications right within Swift (i.e., true DSLs, not just a thing that rewrites ignored top level expressions).

-Chris

23 Likes

That's an interesting idea. I do talk about this a little in the alternatives section; I think there are two significant problems.

The first is that I'm not sure you get the readability advantages you want if you have to consider whether each closure within a scope is actually using the builder. try can be imprecise about this because it means the same thing for every call it covers, but is it really okay under the readability argument for build to just ignore closures that aren't built?

The second is that I do think that's pretty limiting for DSLs. I talk about this a lot in the proposal, but I think a well-designed DSL will make it pretty obvious that a DSL is in effect for a particular function body. Having the build keyword in that context doesn't feel like it adds much.

But we can consider it.

2 Likes

Yes; it's linked as the implementation of the proposal, but I'll go ahead and link it again. Unfortunately, there's currently a merge conflict with Doug's recent work that I need to resolve. It is, of course, also available in the Xcode developer preview, but I understand that that's not helpful for everyone.

Yes, I have considered this. For if/switch, please see my response to Chéyo. As for autograph, I think the transformation I've described — while complicated in its own way — is much simpler and easier to apply to Swift than the autograph transformation, which (1) recursively rewrites entire expressions, not just their results, (2) relies on a sequence of control-and-data-flow-aware analyses in order to completely eliminate local variables, and (3) seems to unavoidably depend on dynamic typing.

3 Likes

I'm trying to understand how this exactly works, and my tinkering in Xcode 11 Playgrounds is not proving fruitful.

From what I gathered from the document, the function builder type is responsible for translating expressions in the closure to the type that the closure ultimately is responsible for returning?

For example:
func chain<T>(@ChainBuilder expressions: () -> [() -> T]) -> [T]

My understanding is that the closure is a series of expressions that evaluate to some value of T, and those expressions of values are passed to the ChainBuilder type to convert it into an array of closures that return a value of T, so that

let a = expressions() // [() -> T]

right?

I haven't been able to replicate this working. In most cases I either get

  1. '<Type used>' is not convertible to '() -> <Type used>'
  2. 'Cannot invoke 'chain' with an argument list of type '(() -> [() -> Int])'

Full example, lifted directly from the proposal with replacements for generics and closures:

@_functionBuilder
struct ChainBuilder {
    static func buildExpression<T>(_ expression: T) -> [() -> T] {
        return [{ return expression }]
    }

    static func buildBlock<T>(_ expression: T) -> [() -> T] {
        return [{ return expression }]
    }

    static func buildBlock<T>(_ children: [() -> T]...) -> [() -> T] {
        return children.flatMap { $0 }
    }

    static func buildBlock<T>(_ component: [() -> T]) -> [() -> T] {
        return component
    }

    static func buildOptional<T>(_ children: [() -> T]?) -> [() -> T] {
        return children ?? []
    }
}

func chain<T>(@ChainBuilder expressions: () -> [() -> T]) -> [T] {
    return expressions().map { $0() }
}

func wrapper<T>(_ value: T) -> T {
    return value
}

// side note: the compiler needs assistance with the `chain` method, even if `ChainBuilder` is hard-coded to work with 'Bool'
let a: [Bool] = chain {
    true    // Cannot convert value of type 'Bool' to closure result type '[() -> Bool]'
}
let b: [Bool] = chain {
    wrapper(true)
}    // Cannot convert value of type 'Bool' to closure result type '[() -> Bool]'
let c: [Bool] = chain {    // Cannot invoke 'chain' with an argument list of type '(() -> [() -> Bool])'
    false
    wrapper(true)
}
// the last block 'c' gets the same error, in the same spot, regardless if it's two (or more) calls to `wrapper(_:)` or are literal expressions
1 Like

Looks like the version of this that's in Xcode 11 right now doesn't actually support buildExpression (what are in the proposal the Expression and Component types need to be identical in the current build), and buildOptional is named buildIf.

3 Likes

Thanks, I forgot about that disclaimer at the top of the proposal about the current implementation.

I'll keep tinkering to see if I can get it to work.

Here's an extremely trivial working example:

@_functionBuilder
struct TestBuilder {
    typealias Component = [Int]

    static func buildBlock(_ children: Component...) -> Component {
        return children.flatMap { $0 }
    }

    static func buildIf(_ children: Component?) -> Component {
        return children ?? []
    }
}
func testBlock(@TestBuilder makeChildren: () -> [Int]) -> [Int] {
    return makeChildren()
}

let b = false
testBlock {
    if b {
        [1]
    }
    [2]
    [3]
}
2 Likes

I really like this approach. It allows writing code in the immediate mode style, which has in my opinion a drool-worthy level of readability and clarity. But, not only we wont have to rest resort to hacks like using a custom allocator or stashing things in thread local storage, with this approach, the compiler exposes enough structure is available to make some really dynamic and interesting use-cases possible. Overall, a :+1: from me.

3 Likes

buildFunction is also missing, you can’t declare local variables, some of the cases that are supposed to be ignored are actually treated as invalid, and the type-checking rules aren’t quite right. It needs some work.

3 Likes

One of SE-0228’s weaknesses is that, because it appends to a builder instance, it cannot infer the return type from the types of inserted expressions. This was an intentional tradeoff made to keep interpolated expressions from becoming too complex. Since SwiftUI needs that kind of inference, function builders need a different design.

(I’ve been wondering if we could perhaps add a defaulted initializer requirement to ExpressibleByStringInterpolation which took a closure containing the append calls; this might allow you to use a function builder to work around that limitation. But I haven’t investigated this at all.)

1 Like

Good point. I was skeptical, but this idea won me over.

6 Likes

Hello,

How do we turn a static sample code as below into a dynamic one where the paragraphs are provided as an array of strings?

div {
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

(behind my question is a "worry" about the variadic root buildBlock functions in the proposal that don't have any array form)

Edit: the answer is obvious, I'm sorry:

div {
  for paragraph in paragraphs {
    p {
      paragraph
    }
  }
}

It's correct, isn't it?

Pretty excited about this; cool to see how it dovetails with recent proposals like return omissions.
A couple thoughts after a first read:

  • buildFunction could use a motivating example. The general concept of combining components into a single result type is logical to me, but the proposal would benefit from a demonstration of this functionality.
  • buildDo is, IMO, unjustified. If a group of components has distinct semantic meaning in some context, a named function that combines the components is a better signifier than making do blocks magical. Trailing closure syntax means there's no syntactic benefit to using do; furthermore, an ordinary do is the statement that "does" the least (just introducing new scope), and I don't think changing that interpretation in the context of a function builder is intuitive.
  • I echo the previously-expressed concerns about composition. Some way to splat an array of expressions would be nice; using for as mentioned in Future Directions seems like a natural fit. As a user, I'd be naturally inclined to write something like this:
for string in subitems {
    Text(string)
}
5 Likes