Function builders

After having a bit more time to consider the specifics of this proposal, I have some more comments:

  • Firstly, I think the name "function builders" is confusing - this isn't really about building functions, but rather about special functions that are more amenable to the builder pattern. builder functions, if you will.

  • Secondly, I think the examples we have seen are overly simplistic (e.g. Apple's SwiftUI examples). In reality, your Views will not only define their layout hierarchy; there may be methods dealing with user interaction, animations and drawing, and all sorts of other code. What does it look like when some parts of your code behave differently to others (this is the "embedded" part)? Or when certain control-flow statements are allowed in some contexts but not in other, identical-looking contexts?

    By its nature, a DSL is not exactly Swift and doesn't follow the regular language rules. When I look at the examples, there is just a lot of unwritten context which is vitally important to somebody reading the code. I'm thinking of an overworked programmer, hacking away late at night and whose view-code is a lot more complex than the simple examples we have seen. They will have to be switching contexts (or brain modes) regularly.

  • Lastly, going back to the problem of making functions that are more amenable to the "builder pattern", it seems to me that we want a builder-function which spits-out children to the builder and then resumes, until the entire tree is built. So basically, we want coroutines:

    var body: some View {
      VStack {
        guard let user = self.user else { yield LoginButton() }
        yield HStack {
          if let pic = user.profilePicture { yield Image(pic) }
          yield Text(user.name)
        }
      }
    }
    

    This turns the implicit thing (the unused results of expressions) in to an explicit thing. It would seem to remove the limitations imposed by not having variadic generics (because each builder receives its children one-at-a-time), remove the need for the builder to handle control flow and make the whole thing fit more nicely in the context of actual Swift.

    Of course, it does introduce a bit of punctuation. We need to consider where we draw the line with this feature: if we wanted a DSL that was exactly like, say, HTML, that already exists - it's called HTML! Why not just embed real HTML in to your Swift code? There is some philosophical debate about what exactly an eDSL is (some would say that any sufficiently complex API is an eDSL), but IMO, the whole point of embedding it is that it should be reasonably like the host language.

    This thing is a little bit like Swift, but it's different enough that I think it will be confusing and difficult to use in the real world.

20 Likes

I’ve been contemplating about the mechanism for transforming the Expression.
So far 2 (2.5?) mechanisms has popped up;

  • Collecting method that collects all Components at the end of closure (currently the proposed design), and
  • Iterative method that keeps the current state of builder (either via builder instance, or static function’s return type) and change that state as new object emits.

Iterative method using builder instance doesn’t catch much of my attention since it lacks the type checking that the other 2 methods offer, which to me is a big plus.

So I would like to share my thought in comparing these two

  1. Iterative Method is nice with control flow. It works almost naturally with looping. We could add buildBeginLoop, buildEndLoop for loop enclosure, and buildNone for block that generates no Expression. This also play pretty well into return, break, continue so long as the branching paths has the same Type.

    Collecting method seems to struggle somewhat with looping, we can either have buildLoop(_: [Component]) -> Component for dynamic loop, but I’m not sure How I feel about for/while being something akin to an expression. We could also flatten the result within the block for loop that we can unroll.

  2. Collecting method compose different structures nicely. The exemplar buildFunction Generally is of the form buildFunction(_: Component...) -> ReturnValue, but it can take other signature (IIUC; at least this is not being corrected where mentioned). Say, you want structure A to be of either the form B C D, or B E F then you can declare

    buildFunction(_: B, _: C, _: D) -> X
    buildFunction(_: B, _: E, _: F) -> X
    

    If Iterative method is to provide this level of type checking, it’d be much harder (tedious?) since author must keep track of the state machine oneself.

    buildFunction(partial: PartialResult0, _: B) -> PR_B
    buildFunction(partial: PR_B, _: C) -> PR_BC
    buildFunction(partial: PR_BC, _: D) -> PR_BCD
    ...
    

    This shouldn’t be much of a problem if the struct has simple grammar (as I mentioned before, both methods permits type order of any regular language).
    Depending on the scope of eDSL of this feature, it may or may not be a big hurdle for Iterative method.

@skagedal suggested elsewhere (maybe jokingly? But I think it's interesting) that we could use yield-style approach, but with a operator, so it looks like

func body() -> some View {
  • div {
    • p { "hello" }
    • p { "world" }
  }
}

and I would like as well some annotation on the function itself, like it's necessary for mutating at the moment. e.g. builder func body() -> some View { ... }

2 Likes

Quick summary of the pitch’s motivation:

  • Swift array literals have a problem with constrained heterogeneous types
  • Also we’d like to allow local variables and conditions in our list-like constructs

If we assume that Sufficiently Generalized Existentials could solve the first part, we’re left with a seemingly obvious question: why can’t you allow local variables and conditions inside collection literals? Is it because ALGOL didn’t?

Such a feature would obviously have a wider set of use cases than ones where a DSL is reasonable (the pitch describes the tradeoffs in such a decision very well). It would also remove a class of thorny design decisions: this list-like thing might sometimes have conditional members, so should I take a collection here, or a builder, or maybe both?

I recognize that SwiftUI wants a solution which reflects the full generated structure at the type level in order to do build-time structural optimizations. However, this use case is not emphasized in the pitch, and with no indication of how great an advantage it will give, it cannot reasonably be evaluated as an argument for a more complex, less orthogonal feature.


Given that a change in direction is unlikely, I do have some comments on the pitched solution:

  • I think the namespace issue is a bigger issue than John does. SwiftUI ducks it by using constructors as it terms, but using functions as in the HTTP example is very reasonable, yet littering the global namespace with functions violates the API Design Guidelines.
    I have some amount of sympathy for those labouring under Apple’s product cycle, but I would like us to see this as a known defect that should be addressed soon. (I was going to suggest a design, but it’s basically what ferranpujolcamins said).

  • I’m worried about confusion that might arise when people accidentally add return statements to closures that are intended to be builders. I’ve already seen what seems to be an example, although I’m told it’s not.
    In general, I worry about the difficulty of generating good diagnostics for builder errors. I’m disinclined to accept “we’ll fix it later” here, given that type inference errors in Swift are still so bad that TI is more of a time sink than a benefit.

  • Several commenters have suggested introducing builders with a specific keyword or annotation.
    The pitch brushes this off by saying that if you do this, you must be explicit about all nested builders and their types, and this is too great a syntactic burden. This is a false dilemma; if no annotation is OK, then an annotation with limited information must also be at least as OK.
    A build keyword at the top level would be enough to tell human readers that a special thing is happening. Not specifying which builder is in use is just a case of implicit type context, which is not a new concept in Swift.

18 Likes

The idea of coroutines is interesting. To make it more lightweight, it could be combined it with a block notation such as {: }, that would yield all the values declared inside.

The example would become:

func body() -> some View {:
  div {:
    p {: "hello" }
    p {: "world" }
  }
}

It is still magical but more general.

11 Likes

Edit: I've moved some ideas into a new thread: @builderOperator attribute

3 Likes

I also have an example that can produce deeply nested HTML structures with the expanded version of this syntax. Works just fine. If anyone would like me to expand on this, let me know.

1 Like

Forgot to say: as others have pointed out, Kotlin’s receiver functions can be used to very similar effect, and deserve some Alternatives Considered love – especially as they have already been pitched for Swift.

18 Likes

Just thinking out loud, I guess, but an alternative approach could perhaps be (mis-)using default arguments:

func div(makeChildren: (builder) -> Void) -> HTMLNode { ... }

func p(builder: Builder = implicit, ...) { ... }

div { builder in
  implicit builder

  if useChapterTitles {
    h1("1. Loomings.")
  }
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

Where implicit is a new keyword, and arguments defaulting to implicit only has a default if the innermost scope has an implicit declaration.

This would make it clear that the rest of the current scope includes magic :slight_smile:

To me the @functionBuilder attribute is like user defined operators. It's just even more hidden and magic. I think it would make Swift code more readable if there was a clear marker that the following code redefines the usual rules.

7 Likes

I am going to agree with this. If I am reading code it is not immediately obvious that I am reading a function builder. In the case of UI code of course that’s not the case but if this feature gets used for other cases then it will be confusing. I guess the culprit here is the actual trailing closure. I am thinking about the SwiftUI case. I much rather have a method call bodyBuilder than a property just name body. A bodyBuilder method would have to be marked with the function builder annotation for example.

1 Like

Let me say first that nothing is defined in the global scope; these things are at module scope, and it's a measure of the fact that we have a weak system for importing modules without making all the names available for unqualified lookup that we keep talking about it that way.

I think if we had the lookup mechanism described when we were designing SwiftUI, there's a good chance we'd have tried to use it. It's hard to know how that experiment would have played out, though. It really depends on the details and how they interact with real usage.

11 Likes

Just saw this on reddit from somebody attending WWDC:

But most importantly, SwiftUI changes the paradigm from MVC to just MV. Had a talk with an Apple Engineer about that yesterday and he confirmed. Logic goes into the view. States go into the view. The view is inside the view. Everything is in the view.

This reinforces my concern above about these builders looking too much like invalid/bad Swift code in real large-scale applications, even though it might look fine in a tutorial app or code snippet on a slide.

I know this pitch isn't about SwiftUI specifically, but it's also a model for any large framework that might want to use this feature.

3 Likes

This is something that React was doing for a while in and it wasn't too big of a problem. This confusion probably could have been avoided in SwiftUI if it used a name like Component instead of View, which is what React did. Not all of the components have to contain any "view"-ish code, they can only encapsulate behavior and get embedded or wrap other views to add/compose those behaviors. Interestingly enough, recently React went away from that with the introduction of React Hooks, as compared to the higher-order components pattern that was widely used previously.

On the other hand, in Swift with generics it's much easier to build non-view logic into different types not conforming to View. E.g.

protocol Behavior {}

struct ViewStuff<B: Behavior>: View {
// ...
}
3 Likes

I'm not too concerned with the architectural side - of course, people who care and have experience developing and maintaining large, complex applications will encapsulate their business logic in to isolated, testable components that can be easily composed. But lots of people don't have the experience to inform them about why that's important or how to do it well.

My concern is mostly about readability. In React's case, HTML looks very different from JS. This is just Swift with implicit special behaviour.

3 Likes

I, for one, am very glad the proposal decided to not have any special syntax or other things to call out the builder-ness of closures at use site. I think it makes the DSLs designed with it much more elegant and compelling.

I also don't think this will be much of an issue. Ruby/Kotlin have the same approach of not differentiating blocks called with instance_exec/receiver closures at use site. They behave even more weirdly, since self/this inside the block is different from self/this outside the block, which means no access to instance methods/properties. It's maybe a little confusing the first time you encounter it, but in my experience pretty much a non-issue afterwards, since APIs using this just look different in use than normal code.

Sure, you could design an API that doesn't look DSL-ish, and that may confuse people (at least those not looking at the parameter types of the functions you're calling), but you can abuse any language feature to create confusing things, so I don't think this is too much of an issue either.

13 Likes

Could someone elaborate more on the pros of having builder use static method, vs Type instance, builder instance, etc?

4 Likes

I've been thinking about this since I learned about SwiftUI as well, and I think I would disagree with their perspective - at least with my still very immature understanding of SwiftUI.

A view is still a representation of external and internal state, with actions that also modify internal and external state. (Internal state of the view affecting the way external state is presented, but which doesn't modify that state themselves)

For samples and small apps, you may be able to operate entirely with actions that manipulate internal and external state directly, sure. But I can't imagine a complex app won't break out the business logic of their data into a separate Model, possibly representing a subset of that state to the View. At that point you are far closer to MVVM.

2 Likes

As a user of Quick the Syntax feels super natural.

Right now in the beta it feels kind of clunky, but I think that's mainly due to code completion being almost non existent for a lot of it. Once that's better (or I learn it :innocent:) I think it will have a noticeably distinct experience compared writing typical (imperative) Swift.

That said, I do think it would be nice to have some sort of syntax/highlighting/etc to indicate that it's different.

3 Likes

Elaborating a bit more now that I feel mentally caught up - there are two main approaches for a DSL feature, both focused on reducing boilerplate to make the code 'feel' like specific language support for their domain.

The first approach is to alter the interpretation of code within a block by altering the default invocation of methods. This is very familiar for people who have Ruby experience - since ruby does late binding of types, the "binding" used to evaluate a block of code can be modified to have a different scope than context of the block. This was actually one of the very first things I pitched - days after SE started - in the form of overriding self within a closure.

More recently and more appropriately specified, this is the tact of the receiver closures pitch. You would pass in a HTML node object. For example, a system in this vein (not the proposed receivers closure syntax) might work as :

var html = HTML { 
  body {
   p("paragraph of text") 
  }

And would be interpreted as

HTML() { document in 
   document.body() { body in 
     body.p("paragraph of text")
   }
}

With @dynamicCallable, it is easy to imagine such a system being extended to allow building of say arbitrary xml or html documents with custom tags as well.

The second approach (which I have less experience with, but which function builders adheres to) is more of a code transformation, like generator functions or coroutines. This could be a visitor, a builder, or an accumulator, but the result is that method resolution of your code doesn't change - the execution context does.

This approach imposes some limitations on the HTML example - body and pre have to be resolvable not just within the DSL, but outside as well. This may mean:

  • use of the DSL is limited to a type where a protocol, extension, or superclass declares these
  • these are declared as functions at DSL-providing module scope
  • you instead use P and Body as types, again at DSL-providing module scope. This at least prevents function/method/property name collision

Something like a custom XML builder would require either a more verbose function/method, or a type specific to each tag. While this is a big overhead for a XML builder, this is one of the benefits of this sort of system with SwiftUI - you wouldn't want to have to extend some View type with factory methods for each view you create in your app just to have them usable in a receiver closure-style DSL.

So obviously each approach has its pros and cons for different kinds of DSLs.

ViewBuilder also shows an additional limitation - the result of the DSL in Swift has a built concrete type. This is why the builder function uses static methods, and why other pitches for this sort of approach - like an accumulator or state monad - would wind up being very complex. These also wind up affecting the types of DSLs that could be made with this sort of approach, although in more nuanced ways.


I suppose I'm most curious about whether there is room in the swift language for both style of approaches, or if SwiftUI has pushed us firmly into the 'function builder' style only.

9 Likes

So I just spent the evening making a basic XML DSL (in Xcode 11) and here's my takeaway from a user perspective.

Experience:
The thing I spent most of the night struggling with turned out to be the connection between (from the proposal)

static func buildBlock(_ children: Component...) -> Component
and
func body(@HTMLBuilder makeChildren: () -> [HTML]) -> HTMLNode

I couldn't for the life of me figure out how to have multiple elements in the same scope.

After starting with a couple different examples and plenty of trial and error, turns out I needed something like:

//Apparently you have to build it even when it's the same inferred Type
static func buildBlock(_ children: Element...) -> [Element] {
    return children
}

Maybe once the feature's finished and there are meaningful exception messages (which weren't very helpful :disappointed:) some of that difficulty will be alleviated.

Thoughts: Again, coming at this from a user perspective as I'm not familiar enough with language building to know what is/isn't possible

I at least want the addition of some built in extensions e.g.

static func buildBlock<T: Expression>(_ children: T...) -> [T] {
    return children
}

which would have taken care of my issue.

I do feel there's a bit too much magic going on with this connection with little to no indication of how it's happening. Specifically with @HTMLBuilder makeChildren: () -> [HTML] I would like some indication of what method in HTMLBuilder would be triggered so it's at least possible to infer the expected code-path.

Especially since I can see wanting to extend @functionBuilders to work with my own Types, it would be helpful to have some visible (or all least clear semantic) connection. Maybe it can be done with tooling, but that seems like a cop out in this situation.

Ideas: Not sure if it's currently possible, but adding the Type like @HTMLBuilder([HTML]) would be nice. Maybe even:

func body( makeChildren: (@HTMLBuilder([HTML])) -> [HTML]) -> HTMLNode
// or 
func body( makeChildren: @HTMLBuilder( ([HTML]) -> [HTML] )) -> HTMLNode
//still callable with makeChildren()

My guess is this would increase the number of body overloads required, but generics and protocols have generally been the way that type of issue is handled and IMO it would drastically increase clarity.

4 Likes