Function builders

I'm still processing the pitch, but I just want to say that I've wanted a feature like this for a while, and I'm really excited about it!

5 Likes

This is phenomenal work! It manages to introduce a principled way of creating new DSLs such that Swift can have something like JSX without it being an entirely separate language. SwiftPM manifests could probably have (still could?) benefitted from this feature.

Question: What was the thought behind supporting special treatment of do blocks and buildDo? Clearly, the authors have some use case in mind but I don't know that I understand.

4 Likes

I’m not sure how important that is. I think the idea was that do naturally provides a kind of grouping, which might be relevant for some DSLs, but on balance I’m not sure expressing grouping through do is very fluent compared to calling some sort of group function provided with the DSL.

1 Like

This is a lot to take in, but I have one comment and one question while I digest it.

Comment: I found the Detailed design section to be difficult to understand. I find myself referring back to proposals somewhat regularly, so I'd love to see some more work here to make it easy to understand.

Question: How does this affect composition?

i.e. let's take this code:

div {
  if useChapterTitles {
    h1(chapter + "1. Loomings.")
  }
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

A natural thing might be to pull out a function that takes 2 strings to create 2 paragraphs:

div {
  if useChapterTitles {
    h1(chapter + "1. Loomings.")
  }
  pp("Call me Ishmael. Some years ago", "There is now your insular city")
}

// Okay, this is a bit contrived. But it hopefully gets the point across.
func pp(_: String, _: String) -> [HTML] { … }

Is that possible? I don't see where an array of HTMLs can be added in this way.

What if I wanted to use map instead?

div {
  if useChapterTitles {
    h1(chapter.title)
  }

  chapter.paragraphs.map(p)
}
11 Likes

The HTMLBuilder design in the proposal is really just for exposition, but you're right that the design as given has a problem with composition, and I agree that it would be good to allow multiple elements to be expanded in the parent. Our design choice as the author of HTMLBuilder is between two options:

  • If a statement should be able to just produce a collection of HTML, then we can allow that by just giving an overload of buildExpression that takes an [HTML] (or maybe generically any Collection where Element == HTML).
  • Alternatively, we can require that to be more explicit by, say, defining a function like func include(_ elements: [HTML]) and having it return some private-to-the-DSL type that will be recognized by buildExpression and then unwrapped.
2 Likes

(Moderator note: with the author's permission, I've moved this post here as it poses only technical questions.)

i read through the rationale in your draft proposal, and it just doesn't feel like the rationale explains the choice of solution.

The first problem is that there's a lot of punctuation here: commas, parentheses, and brackets. This is a pretty superficial problem, and it's probably not a showstopper by itself, but it is something that it'd be nice to avoid, because it does distract a little from the content.

i don't think this counts as a problem. maybe half a problem?

The second problem is that, because we're using array literals for the children, the type-checker is going to require the elements to have a homogeneous type. That's fine for our HTML example, but it's limiting in general, because some trees are more generic and would benefit from propagating the types of the children into the type of the node. For example, SwiftUI uses this for various optimizations within the view hierarchy.

the proposed solution doesn't address this problem. without variadic generics, the elements still need to have homogenous type. if the workaround is this, i don't want it.

The biggest problem, though, is that it's awkward to change the structure of this hierarchy. That's fine if our hierarchy is just the static contents of Moby Dick , but in reality, we're probably generating HTML that should vary significantly based on dynamic information.

the most straightforward answer to this is conditional element inclusion, not function building. it would be a step further from the original compile-time inclusion idea, but you could use it outside of a function building API, so it would benefit the whole language.

12 Likes

I second the question about buildDo. The only use in the proposal is in the Future Directions section. I presume there's maybe some current use in SwiftUI, but it would be helpful to have an example or explanation in the proposal for why it exists and buildBlock isn't always sufficient.

3 Likes

What would need to change if/when we added variadic generics? If this feature is ‘simply’ augmented by variadic generics, I don’t see much of a problem

6 Likes

My first thoughts are:

  • @HTMLBuilder in
    "func div(@HTMLBuilder makeChildren: …) -> … {…}"
    I think it should be @builder(HTMLBuilder)

  • build.* as static functions?
    If they were instance functions and an instance was created at the start and modified throughout, conditionals, loops, and break etc. would naturally work.
    It would also be more similar to SE-0228.

7 Likes

100% agree with your arguments, if no variadics, or some kind of ruby's blocks + metaprogramming this will harm the lang itself with people trying to replicated this thing: Apple Developer Documentation will be awful, IMHO. I think rather than introducing DSLs based on something so horrendous as the buildBlock(_:_:_:_:_:_:_:_:_:_:) func, we should relay in something like subscripts, or even Nick Lockwood's layout approach: GitHub - nicklockwood/layout: A declarative UI framework for iOS.

1 Like

I had a similar though but prefer the syntax @builder(HTML) since it working with a closure of: () -> [HTML]

1 Like

Well, as shown HTMLBuilder is the 'builder' type, so @builder(HTMLBuilder) is the minimal change.
But I agree that it would be better if HTML were it's own 'builder', with @builder(HTML).

2 Likes

Ah I missed that, I haven't read the actual draft pitch yet... But either way I like the @builder(...) style syntax for this.

I'll read the whole draft now :)

Edit: Having read the proposal I now like the @HTMLBuilder syntax having more understanding of what is actually going on with this pitch.

1 Like

It’s not a long-term problem

otoh most long-term problems start with that kind of thinking

4 Likes

Remember that we're not actually designing HTMLBuilder.

Can you explain why you prefer @builder(HTMLBuilder) here? Just general resistance to custom attributes, or do you feel that custom attributes are a poor match for this?

1 Like

There are several reasons I didn't use SwiftUI as the example in the proposal. The type-preservation that SwiftUI needs adds quite a lot of complexity both to the exposition and to the implementation. Most DSLs will not need it and will be just fine with type erasure either via an enum or a protocol type; in either case, ordinary variadic arguments on buildBlock will be fine.

I do not think it makes sense to commit SwiftUI to an inferior language solution because of an implementation detail that we know exactly how to address in the medium term.

11 Likes

Can you add to the proposal the reasoning to use buildEither(...) instead of pushing for if/switch expressions? That part of the proposal trying to explain buildEither(...) was hard for me to follow.

2 Likes

Sure. I understand this is subtle. The answer is, again, type preservation: an if/switch expression has to require all of its cases to return the same type, but if the DSL wants the result type to reflect the structure of its component types, it's obviously unreasonable to require all the cases to have identical structure. buildEither is effectively injecting into one side or the other of an Either type without actually requiring such a type to exist; if it does exist (as it does in SwiftUI), it can carry the structure of the alternatives in its type arguments. I'll revise the proposal to make that clear.

5 Likes

Overall, I like this feature; I don't think it feels too magical and can see it being very powerful, especially once variadic generics are in place.

I do have one concern, though. The proposal says of having annotations for the top level call site:

  • Requiring an annotation on the outermost function and then propagating it to lexically-nested closures avoids “attribute overload” but is quite limiting on the DSL. For one, there may be compelling reasons for the DSL to use non-DSL functions within its body; for example, a UI hierarchy might want to directly define handler functions for various events. This suggests that there would have to be a way of turning off the lexical propagation (which would be error-prone in much the same way as non-propagation would be). Second, a DSL might actually consist of multiple similar function builder types used in different positions; this sort of implementation detail should not be allowed to become a serious usability problem for clients.

While I can see how this would apply with e.g. an @HTMLBuilder attribute, I feel as though having some signifier to specify that a function-builder context is being entered would still be useful.

Has having a generic e.g. build keyword (similar to try) been considered, to signify that a function-builder context is being entered? Within the scope of the context, build would be implicitly prefixed to each expression; it would have no effect on functions/closures that do not take a @functionBuilder type (similar to how try has no effect on functions that do not throw other than emitting a warning).

That would transform the example code into:

build div {
  if useChapterTitles {
    h1(chapter + "1. Loomings.")
  }
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

I feel that the (presumably) syntax-highlighted build keyword would aid clarity in mixed-paradigm codebases.

8 Likes

Hi John,

This is a super interesting proposal. I am so happy to see a direction that takes eDSL's seriously - this is a very exciting opportunity for Swift, one that could solve a huge range of issues and problems. It could even potentially get us to the golden world where all "statements" are just standard library features with a default statement rewrite rule in the standard prelude / stdlib.

Stepping back from that beautiful world, the quote above highlights an interesting point. I'm still trying to understand the feature itself (is there a patch implementing the feature available somewhere?), but it seems like there are two things going on here:

  1. You're providing a very general statement rewriting facility, where a syntactic transformation is changing the statement AST into a series of function calls.

  2. You're transforming if/switch "into a single result value that becomes the return value of the current function"

Have you considered decomposing these into two separate and orthogonal -- but composable features? One that allows if/switch to return values (this is something that has been highly requested for a long time) and one that allows "function builders" to transform the AST into overloadable function calls (e.g. along the lines of tf autograph)?

The composition of these two features seem like they'd be far more powerful and also simpler than the feature as proposed. I'm very interested in allowing this to eventually feature creep towards defining CUDA kernels and other accelerator functions for machine learning applications right within Swift (i.e., true DSLs, not just a thing that rewrites ignored top level expressions).

-Chris

23 Likes