Function builders

[dabrahams (Dave Abrahams)]

Let me say first that nothing is defined in the global scope; these things are at module scope, and it's a measure of the fact that we have a weak system for importing modules without making all the names available for unqualified lookup that we keep talking about it that way.

I think if we had the lookup mechanism described when we were designing SwiftUI, there's a good chance we'd have tried to use it. It's hard to know how that experiment would have played out, though. It really depends on the details and how they interact with real usage.

[Karl]

Just saw this on reddit from somebody attending WWDC:

But most importantly, SwiftUI changes the paradigm from MVC to just MV. Had a talk with an Apple Engineer about that yesterday and he confirmed. Logic goes into the view. States go into the view. The view is inside the view. Everything is in the view.

This reinforces my concern above about these builders looking too much like invalid/bad Swift code in real large-scale applications, even though it might look fine in a tutorial app or code snippet on a slide.

I know this pitch isn't about SwiftUI specifically, but it's also a model for any large framework that might want to use this feature.

[Max_Desiatov]

@Karl

But most importantly, SwiftUI changes the paradigm from MVC to just MV. Had a talk with an Apple Engineer about that yesterday and he confirmed. Logic goes into the view. States go into the view. The view is inside the view. Everything is in the view.

This is something that React was doing for a while in and it wasn't too big of a problem. This confusion probably could have been avoided in SwiftUI if it used a name like Component instead of View, which is what React did. Not all of the components have to contain any "view"-ish code, they can only encapsulate behavior and get embedded or wrap other views to add/compose those behaviors. Interestingly enough, recently React went away from that with the introduction of React Hooks, as compared to the higher-order components pattern that was widely used previously.

On the other hand, in Swift with generics it's much easier to build non-view logic into different types not conforming to View. E.g.

protocol Behavior {}

struct ViewStuff<B: Behavior>: View {
// ...
}

[Karl]

I'm not too concerned with the architectural side - of course, people who care and have experience developing and maintaining large, complex applications will encapsulate their business logic in to isolated, testable components that can be easily composed. But lots of people don't have the experience to inform them about why that's important or how to do it well.

My concern is mostly about readability. In React's case, HTML looks very different from JS. This is just Swift with implicit special behaviour.

[ahti (Lukas Stabe 🙃)]

I, for one, am very glad the proposal decided to not have any special syntax or other things to call out the builder-ness of closures at use site. I think it makes the DSLs designed with it much more elegant and compelling.

I also don't think this will be much of an issue. Ruby/Kotlin have the same approach of not differentiating blocks called with instance_exec/receiver closures at use site. They behave even more weirdly, since self/this inside the block is different from self/this outside the block, which means no access to instance methods/properties. It's maybe a little confusing the first time you encounter it, but in my experience pretty much a non-issue afterwards, since APIs using this just look different in use than normal code.

Sure, you could design an API that doesn't look DSL-ish, and that may confuse people (at least those not looking at the parameter types of the functions you're calling), but you can abuse any language feature to create confusing things, so I don't think this is too much of an issue either.

[Lantua]

Could someone elaborate more on the pros of having builder use static method, vs Type instance, builder instance, etc?

[dwaite (David Waite)]

@Karl

But most importantly, SwiftUI changes the paradigm from MVC to just MV. Had a talk with an Apple Engineer about that yesterday and he confirmed. Logic goes into the view. States go into the view. The view is inside the view. Everything is in the view.

This reinforces my concern above about these builders looking too much like invalid/bad Swift code in real large-scale applications, even though it might look fine in a tutorial app or code snippet on a slide.

I know this pitch isn't about SwiftUI specifically, but it's also a model for any large framework that might want to use this feature.

I've been thinking about this since I learned about SwiftUI as well, and I think I would disagree with their perspective - at least with my still very immature understanding of SwiftUI.

A view is still a representation of external and internal state, with actions that also modify internal and external state. (Internal state of the view affecting the way external state is presented, but which doesn't modify that state themselves)

For samples and small apps, you may be able to operate entirely with actions that manipulate internal and external state directly, sure. But I can't imagine a complex app won't break out the business logic of their data into a separate Model, possibly representing a subset of that state to the View. At that point you are far closer to MVVM.

[GetSwifty]

@Karl

This reinforces my concern above about these builders looking too much like invalid/bad Swift code in real large-scale applications, even though it might look fine in a tutorial app or code snippet on a slide.

As a user of Quick the Syntax feels super natural.

Right now in the beta it feels kind of clunky, but I think that's mainly due to code completion being almost non existent for a lot of it. Once that's better (or I learn it ) I think it will have a noticeably distinct experience compared writing typical (imperative) Swift.

That said, I do think it would be nice to have some sort of syntax/highlighting/etc to indicate that it's different.

[dwaite (David Waite)]

@dwaite

@John_McCall

That’s needed only because (1) SwiftUI wants to propagate sub-view types into the return type to enable some optimizations and (2) Swift doesn’t have variadic generics yet.

...as a requirement would push very hard toward this particular builder design.

Elaborating a bit more now that I feel mentally caught up - there are two main approaches for a DSL feature, both focused on reducing boilerplate to make the code 'feel' like specific language support for their domain.

The first approach is to alter the interpretation of code within a block by altering the default invocation of methods. This is very familiar for people who have Ruby experience - since ruby does late binding of types, the "binding" used to evaluate a block of code can be modified to have a different scope than context of the block. This was actually one of the very first things I pitched - days after SE started - in the form of overriding self within a closure.

More recently and more appropriately specified, this is the tact of the receiver closures pitch. You would pass in a HTML node object. For example, a system in this vein (not the proposed receivers closure syntax) might work as :

var html = HTML { 
  body {
   p("paragraph of text") 
  }

And would be interpreted as

HTML() { document in 
   document.body() { body in 
     body.p("paragraph of text")
   }
}

With @dynamicCallable, it is easy to imagine such a system being extended to allow building of say arbitrary xml or html documents with custom tags as well.

The second approach (which I have less experience with, but which function builders adheres to) is more of a code transformation, like generator functions or coroutines. This could be a visitor, a builder, or an accumulator, but the result is that method resolution of your code doesn't change - the execution context does.

This approach imposes some limitations on the HTML example - body and pre have to be resolvable not just within the DSL, but outside as well. This may mean:

• use of the DSL is limited to a type where a protocol, extension, or superclass declares these
• these are declared as functions at DSL-providing module scope
• you instead use P and Body as types, again at DSL-providing module scope. This at least prevents function/method/property name collision

Something like a custom XML builder would require either a more verbose function/method, or a type specific to each tag. While this is a big overhead for a XML builder, this is one of the benefits of this sort of system with SwiftUI - you wouldn't want to have to extend some View type with factory methods for each view you create in your app just to have them usable in a receiver closure-style DSL.

So obviously each approach has its pros and cons for different kinds of DSLs.

ViewBuilder also shows an additional limitation - the result of the DSL in Swift has a built concrete type. This is why the builder function uses static methods, and why other pitches for this sort of approach - like an accumulator or state monad - would wind up being very complex. These also wind up affecting the types of DSLs that could be made with this sort of approach, although in more nuanced ways.

---

I suppose I'm most curious about whether there is room in the swift language for both style of approaches, or if SwiftUI has pushed us firmly into the 'function builder' style only.

[GetSwifty]

So I just spent the evening making a basic XML DSL (in Xcode 11) and here's my takeaway from a user perspective.

Experience:
The thing I spent most of the night struggling with turned out to be the connection between (from the proposal)

static func buildBlock(_ children: Component...) -> Component
and
func body(@HTMLBuilder makeChildren: () -> [HTML]) -> HTMLNode

I couldn't for the life of me figure out how to have multiple elements in the same scope.

After starting with a couple different examples and plenty of trial and error, turns out I needed something like:

//Apparently you have to build it even when it's the same inferred Type
static func buildBlock(_ children: Element...) -> [Element] {
    return children
}

Maybe once the feature's finished and there are meaningful exception messages (which weren't very helpful ) some of that difficulty will be alleviated.

Thoughts: Again, coming at this from a user perspective as I'm not familiar enough with language building to know what is/isn't possible

I at least want the addition of some built in extensions e.g.

static func buildBlock<T: Expression>(_ children: T...) -> [T] {
    return children
}

which would have taken care of my issue.

I do feel there's a bit too much magic going on with this connection with little to no indication of how it's happening. Specifically with @HTMLBuilder makeChildren: () -> [HTML] I would like some indication of what method in HTMLBuilder would be triggered so it's at least possible to infer the expected code-path.

Especially since I can see wanting to extend @functionBuilders to work with my own Types, it would be helpful to have some visible (or all least clear semantic) connection. Maybe it can be done with tooling, but that seems like a cop out in this situation.

Ideas: Not sure if it's currently possible, but adding the Type like @HTMLBuilder([HTML]) would be nice. Maybe even:

func body( makeChildren: (@HTMLBuilder([HTML])) -> [HTML]) -> HTMLNode
// or 
func body( makeChildren: @HTMLBuilder( ([HTML]) -> [HTML] )) -> HTMLNode
//still callable with makeChildren()

My guess is this would increase the number of body overloads required, but generics and protocols have generally been the way that type of issue is handled and IMO it would drastically increase clarity.

[tikitu (Tikitu de Jager)]

No contribution to the technical details, but I echo the suggestion to rename from "function builders" to "builder functions". Both can mean what I think you intend (functions acting as builders), but "function builders" can also mean something confusing (things that build functions) while I don't see that ambiguity for "builder functions".

[Paul_Cantrell]

Still taking some time to digest this, but after sleeping on it, two strong reactions:

First, the good reaction: I’m thrilled to see eDSLs getting serious attention — and I really like the general flavor of this approach, which retains the type safety and careful structure we expect from Swift while also getting serious about readability and syntactic noise. Wherever this feature settles out, I’m sure it will find many exciting applications!

Second, the bad reaction: the complexity of the numerous special rules that govern the syntactic transform seem like a lot for users to keep in their heads.

My first instinct would be to prefer a coroutine approach with explicit yields, as others have mentioned. I do appreciate the concision of this proposal — nobody wants to type yield for every UI element. And it seems like this proposal maybe allows for deeper static analysis?

Still, I’m concerned about the mental burden of this feature in a language that already at time feels too much like a maze of special cases. The non-support for for and switch, and buildBlock(_:_:_:_:_:_:_:_:_:_:), aren’t just missing future features; they’re indications of how much this proposal doesn’t compose well with existing language features. It seems like users are going to repeatedly hit frustrating situations where code that works fine outside a function builder doesn’t work inside one.

These concerns are a first impression, and they might dissipate with actual usage. But let’s do think about what we’re asking of consumers of this feature.

[Lantua]

@GetSwifty

Ideas: Not sure if it's currently possible, but adding the Type like @HTMLBuilder([HTML]) would be nice. Maybe even:

This was brought up very early on in the pitch. The current design technically doesn't create new type of closure, but rather allow for builder to transform the closure inside.

@John_McCall

The information written to the right of the colon is part of the type of the parameter. @HTMLBuilder isn't part of the type; it enables the function builder transformation for arguments that are directly closure expressions, but doesn't otherwise affect anything about the semantics of the call or its parameter.

[Lantua]

@Paul_Cantrell

Second, the bad reaction: the complexity of the numerous special rules that govern the syntactic transform seem like a lot for users to keep in their heads.

I don't think the end users would need to understand the rule, mostly a proper builder should hide them neatly anyway.

As for the author of builder, I feel that build... functions are compact enough that you can consider them separately without a concern for the rule.

@Paul_Cantrell

The non-support for for and switch , and buildBlock(_:_:_:_:_:_:_:_:_:_:) , aren’t just missing future features; they’re indications of how much this proposal doesn’t compose well with existing language

I do agree, I did say similar thing a while ago.

@Lantua

Iterative Method is nice with control flow.

[Les_Pruszynski]

@Max_Desiatov

Interestingly enough, recently React went away from that with the introduction of React Hooks , as compared to the higher-order components pattern that was widely used previously.

They decided to use hooks because React's performance is horrible. Hooks were supposed to overcome some of these problems but created even more. This is a horrible move further exposed by Svelte's 3 incredible performance.

[a.b]

An argument in favor of a @builder(HTMLBuilder) approach rather than a @HTMLBuilder approach is the introduction of a similar syntactic construct down the line, e.g. @schmilder(HTMLSchmilder).

As things stand now, there would be no syntactic distinction between the cases.

[GetSwifty]

@Lantua

This was brought up very early on in the pitch. The current design technically doesn't create new type of closure, but rather allow for builder to transform the closure inside.

Right, that's my problem. There's no real syntactic indication that's what's happening and it's difficult to infer how that transform will take place. It's just Magic!

[griotspeak (TJ Usiyan)]

@Paul_Cantrell

The non-support for for and switch , and buildBlock(_:_:_:_:_:_:_:_:_:_:) , aren’t just missing future features; they’re indications of how much this proposal doesn’t compose well with existing language features.

This doesn't really make sense to me as an inference. "Doesn't compose well with…" for and switch support seems like a decision to start 'simple' and this feature just highlights the need for variadic generics.

@Paul_Cantrell

It seems like users are going to repeatedly hit frustrating situations where code that works fine outside a function builder doesn’t work inside one.

We have that situation now because code is context sensitive. Unqualified methods, types, captured variables… If you copied something from or into a package manifest, things get confusing. Seems like exactly the same situation.

[Paul_Cantrell]

First, an aside, just to be clear: I’m not trying to sink the proposal here; I do tend to like it. This is just a concern I’d like to explore, trying to weigh the pros and cons of the proposal’s approach.

An example of what I mean by “doesn’t compose well with existing language features” might help. This works:

// (1)
var body: some View {
    List(items) { item in
        Text(item.owner.name)
    }
}

If I understand the proposal correctly, this will also work:

// (2)
var body: some View {
    List(items) { item in
        let person = item.owner
        Text(person.name)
    }
}

…but this won’t, because it will try to transform print(…) just like any other expression, and print doesn’t return a View:

// (3)
var body: some View {
    List(items) { item in
        let person = item.owner
        print(person)
        Text(person.name)
    }
}

Putting myself in the shoes of an innocent user, the 1→2 transform doesn’t feel fundamentally different than 2→3, yet one works and one doesn’t. The eDSL is sort of real Swift, and sort of not. This strikes me as a wellspring of confusion.

The proposal’s future directions section does mention implicitly ignoring void-returning funcs, which would make (3) work as is. But that’s only pushing the confusion back a step, playing whack-a-mole with common use cases. Now while (3) works, this equally innocent-looking code is a problem:

// (4)
var body: some View {
    List(items) { item in
        let person = item.owner
        for toe in person.toes {
            print(person.toe)
        }
        Text(person.name)
    }
}

Yet (assuming that future direction is implemented) this works:

// (5)
var body: some View {
    List(items) { item in
        let person = item.owner
        printToes(person)
        Text(person.name)
    }
}

func printToes(_ person: Person) {
    for toe in person.toes {
        print(person.toe)
    }
}

This all sure seems like it will be confusing to users. You mention SwiftPM manifests, but here the eDSL is embedded right in the middle of normal Swift code. Say what you will about JSX (and I could say thing or two), but XML doesn’t look remotely like Javascript, and it’s thus really clear which one you’re writing where.

All of this is not a necessarily a showstopper. I just don’t love the smell of it. I wonder if there are ways to reduce the mental friction.

[Joe_Groff]

@griotspeak

This doesn't really make sense to me as an inference. "Doesn't compose well with…" for and switch support seems like a decision to start 'simple' and this feature just highlights the need for variadic generics.

Yeah, it shouldn't be a huge stretch to extend this to for and switch. The approach John's devised for mapping ifs into Eithers separates the concerns of conditional control flow from the concerns of the DSL, and so similar transformations ought to be possible for switch, catch, guard, and non-loop control flow constructs in the fullness of time. for might be something function builders want to hook into more deeply; if you look at SwiftUI, the ForEach component seems like a natural fit for for syntax.