Function builders

Overall, I like this feature; I don't think it feels too magical and can see it being very powerful, especially once variadic generics are in place.

I do have one concern, though. The proposal says of having annotations for the top level call site:

  • Requiring an annotation on the outermost function and then propagating it to lexically-nested closures avoids “attribute overload” but is quite limiting on the DSL. For one, there may be compelling reasons for the DSL to use non-DSL functions within its body; for example, a UI hierarchy might want to directly define handler functions for various events. This suggests that there would have to be a way of turning off the lexical propagation (which would be error-prone in much the same way as non-propagation would be). Second, a DSL might actually consist of multiple similar function builder types used in different positions; this sort of implementation detail should not be allowed to become a serious usability problem for clients.

While I can see how this would apply with e.g. an @HTMLBuilder attribute, I feel as though having some signifier to specify that a function-builder context is being entered would still be useful.

Has having a generic e.g. build keyword (similar to try) been considered, to signify that a function-builder context is being entered? Within the scope of the context, build would be implicitly prefixed to each expression; it would have no effect on functions/closures that do not take a @functionBuilder type (similar to how try has no effect on functions that do not throw other than emitting a warning).

That would transform the example code into:

build div {
  if useChapterTitles {
    h1(chapter + "1. Loomings.")
  }
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

I feel that the (presumably) syntax-highlighted build keyword would aid clarity in mixed-paradigm codebases.

8 Likes

Hi John,

This is a super interesting proposal. I am so happy to see a direction that takes eDSL's seriously - this is a very exciting opportunity for Swift, one that could solve a huge range of issues and problems. It could even potentially get us to the golden world where all "statements" are just standard library features with a default statement rewrite rule in the standard prelude / stdlib.

Stepping back from that beautiful world, the quote above highlights an interesting point. I'm still trying to understand the feature itself (is there a patch implementing the feature available somewhere?), but it seems like there are two things going on here:

  1. You're providing a very general statement rewriting facility, where a syntactic transformation is changing the statement AST into a series of function calls.

  2. You're transforming if/switch "into a single result value that becomes the return value of the current function"

Have you considered decomposing these into two separate and orthogonal -- but composable features? One that allows if/switch to return values (this is something that has been highly requested for a long time) and one that allows "function builders" to transform the AST into overloadable function calls (e.g. along the lines of tf autograph)?

The composition of these two features seem like they'd be far more powerful and also simpler than the feature as proposed. I'm very interested in allowing this to eventually feature creep towards defining CUDA kernels and other accelerator functions for machine learning applications right within Swift (i.e., true DSLs, not just a thing that rewrites ignored top level expressions).

-Chris

23 Likes

That's an interesting idea. I do talk about this a little in the alternatives section; I think there are two significant problems.

The first is that I'm not sure you get the readability advantages you want if you have to consider whether each closure within a scope is actually using the builder. try can be imprecise about this because it means the same thing for every call it covers, but is it really okay under the readability argument for build to just ignore closures that aren't built?

The second is that I do think that's pretty limiting for DSLs. I talk about this a lot in the proposal, but I think a well-designed DSL will make it pretty obvious that a DSL is in effect for a particular function body. Having the build keyword in that context doesn't feel like it adds much.

But we can consider it.

2 Likes

Yes; it's linked as the implementation of the proposal, but I'll go ahead and link it again. Unfortunately, there's currently a merge conflict with Doug's recent work that I need to resolve. It is, of course, also available in the Xcode developer preview, but I understand that that's not helpful for everyone.

Yes, I have considered this. For if/switch, please see my response to Chéyo. As for autograph, I think the transformation I've described — while complicated in its own way — is much simpler and easier to apply to Swift than the autograph transformation, which (1) recursively rewrites entire expressions, not just their results, (2) relies on a sequence of control-and-data-flow-aware analyses in order to completely eliminate local variables, and (3) seems to unavoidably depend on dynamic typing.

3 Likes

I'm trying to understand how this exactly works, and my tinkering in Xcode 11 Playgrounds is not proving fruitful.

From what I gathered from the document, the function builder type is responsible for translating expressions in the closure to the type that the closure ultimately is responsible for returning?

For example:
func chain<T>(@ChainBuilder expressions: () -> [() -> T]) -> [T]

My understanding is that the closure is a series of expressions that evaluate to some value of T, and those expressions of values are passed to the ChainBuilder type to convert it into an array of closures that return a value of T, so that

let a = expressions() // [() -> T]

right?

I haven't been able to replicate this working. In most cases I either get

  1. '<Type used>' is not convertible to '() -> <Type used>'
  2. 'Cannot invoke 'chain' with an argument list of type '(() -> [() -> Int])'

Full example, lifted directly from the proposal with replacements for generics and closures:

@_functionBuilder
struct ChainBuilder {
    static func buildExpression<T>(_ expression: T) -> [() -> T] {
        return [{ return expression }]
    }

    static func buildBlock<T>(_ expression: T) -> [() -> T] {
        return [{ return expression }]
    }

    static func buildBlock<T>(_ children: [() -> T]...) -> [() -> T] {
        return children.flatMap { $0 }
    }

    static func buildBlock<T>(_ component: [() -> T]) -> [() -> T] {
        return component
    }

    static func buildOptional<T>(_ children: [() -> T]?) -> [() -> T] {
        return children ?? []
    }
}

func chain<T>(@ChainBuilder expressions: () -> [() -> T]) -> [T] {
    return expressions().map { $0() }
}

func wrapper<T>(_ value: T) -> T {
    return value
}

// side note: the compiler needs assistance with the `chain` method, even if `ChainBuilder` is hard-coded to work with 'Bool'
let a: [Bool] = chain {
    true    // Cannot convert value of type 'Bool' to closure result type '[() -> Bool]'
}
let b: [Bool] = chain {
    wrapper(true)
}    // Cannot convert value of type 'Bool' to closure result type '[() -> Bool]'
let c: [Bool] = chain {    // Cannot invoke 'chain' with an argument list of type '(() -> [() -> Bool])'
    false
    wrapper(true)
}
// the last block 'c' gets the same error, in the same spot, regardless if it's two (or more) calls to `wrapper(_:)` or are literal expressions
1 Like

Looks like the version of this that's in Xcode 11 right now doesn't actually support buildExpression (what are in the proposal the Expression and Component types need to be identical in the current build), and buildOptional is named buildIf.

3 Likes

Thanks, I forgot about that disclaimer at the top of the proposal about the current implementation.

I'll keep tinkering to see if I can get it to work.

Here's an extremely trivial working example:

@_functionBuilder
struct TestBuilder {
    typealias Component = [Int]

    static func buildBlock(_ children: Component...) -> Component {
        return children.flatMap { $0 }
    }

    static func buildIf(_ children: Component?) -> Component {
        return children ?? []
    }
}
func testBlock(@TestBuilder makeChildren: () -> [Int]) -> [Int] {
    return makeChildren()
}

let b = false
testBlock {
    if b {
        [1]
    }
    [2]
    [3]
}
2 Likes

I really like this approach. It allows writing code in the immediate mode style, which has in my opinion a drool-worthy level of readability and clarity. But, not only we wont have to rest resort to hacks like using a custom allocator or stashing things in thread local storage, with this approach, the compiler exposes enough structure is available to make some really dynamic and interesting use-cases possible. Overall, a :+1: from me.

3 Likes

buildFunction is also missing, you can’t declare local variables, some of the cases that are supposed to be ignored are actually treated as invalid, and the type-checking rules aren’t quite right. It needs some work.

3 Likes

One of SE-0228’s weaknesses is that, because it appends to a builder instance, it cannot infer the return type from the types of inserted expressions. This was an intentional tradeoff made to keep interpolated expressions from becoming too complex. Since SwiftUI needs that kind of inference, function builders need a different design.

(I’ve been wondering if we could perhaps add a defaulted initializer requirement to ExpressibleByStringInterpolation which took a closure containing the append calls; this might allow you to use a function builder to work around that limitation. But I haven’t investigated this at all.)

1 Like

Good point. I was skeptical, but this idea won me over.

6 Likes

Hello,

How do we turn a static sample code as below into a dynamic one where the paragraphs are provided as an array of strings?

div {
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

(behind my question is a "worry" about the variadic root buildBlock functions in the proposal that don't have any array form)

Edit: the answer is obvious, I'm sorry:

div {
  for paragraph in paragraphs {
    p {
      paragraph
    }
  }
}

It's correct, isn't it?

Pretty excited about this; cool to see how it dovetails with recent proposals like return omissions.
A couple thoughts after a first read:

  • buildFunction could use a motivating example. The general concept of combining components into a single result type is logical to me, but the proposal would benefit from a demonstration of this functionality.
  • buildDo is, IMO, unjustified. If a group of components has distinct semantic meaning in some context, a named function that combines the components is a better signifier than making do blocks magical. Trailing closure syntax means there's no syntactic benefit to using do; furthermore, an ordinary do is the statement that "does" the least (just introducing new scope), and I don't think changing that interpretation in the context of a function builder is intuitive.
  • I echo the previously-expressed concerns about composition. Some way to splat an array of expressions would be nice; using for as mentioned in Future Directions seems like a natural fit. As a user, I'd be naturally inclined to write something like this:
for string in subitems {
    Text(string)
}
5 Likes

I'm not the guy you're replying to, but I'm definitely feeling this. We haven't hit Java levels of attribute abuse yet, but it seems to me that we're headed that way, fast.

I understand this attribute would be hidden in a library implementation, unbeknownst to its consumers, but I'm still not a fan.

I would love to see namespaces for attributes (e.g. @dynamic.callable, @dynamic.memberLookup, @functionBuilder.HTML, @propertyDelegate.Lazy), and a Rust-like syntax to elide repeated @ when multiple attributes are required, like:

@[ dynamic.callable, dynamic.memberLookup, functionBuilder.HTML, fooAble.bar ]
struct MyFancyStruct { }

I think it's really important for us to come up with an attribute manifesto, and figure out exactly what we want out of attribute system, and how our language evolution can progress without turning into enterprise Java.

13 Likes

Proposal suggestion: I'd prefer for there to be some way to differently/specifically handle expressions inside builder functions whose result type is Void. This would have the advantage of handling asserts and preconditions reasonably, and would obviate the need for a special rule about assignments, since assignments typecheck to Void.

Two possibilities:
(1) Expressions of Void result type should be implicitly ignored (and no Component created for them) unless the builder explicitly has a buildExpression(_ void: Void) -> Component static method.

(2) Allow for buildExpression overrides which return either Void themselves or return Component?, in which case there may be no Component for that expression in that level's buildX variadic. This is more general and let's you avoid having a (probably common) .empty case in the enum backing your DSL, but means most builders will have boilerplate buildExpression(_ void: Void) -> Void {} implementations.

4 Likes

Current Core Team thinking is that marker attributes like @dynamicCallable should just be eliminated, but you’re right that aggregating attributes in general is an interesting problem that we should be thinking about.

3 Likes

Not the original target of this question, but having the attribute namespace be filled up dynamically by previous declarations visible in the code feels ... weird. I'd prefer @builder(HTMLBuilder) simply because I expect the attributes the compiler supports to be a fixed list (for any particular compiler version).

9 Likes

Will the names for the various builders have/support qualified lookup?

Just read through the proposal and the thread. It's definitely a +1 from me.

I think there's no doubt this is useful and solves a real problem (look no further than SwiftUI).

However, I'm not a fan of the HTML use case. I prefer templated HTML pages to be HTML-like, with another language interpolated in, rather than the other way around. For one, because you can use the massive ecosystem of existing HTML WYSIWYG tools to get started with an initial HTML structure (into which you can add your dynamic bindings).

I also do not like that this proposal uses static functions for everything, it just seems needlessly restrictive. If the thinking here is that it's intended to prevent stateful behaviour, as some kind of counter-measure against abuse. I think that's futile. If people wanted stateful behaviour, they could just use thread local storage. It's just icky.

I think it's better to initialize new instances of a function builder for every "outermost block", and to call methods on that instance. This gives access to a shared state, who knows what kind of cool uses people could come up with.

Wouldn't it be funny to implement async/await using this mechanism? :rofl:

3 Likes
Terms of Service

Privacy Policy

Cookie Policy