Function builders

[gwendal.roue (Gwendal Roué)]

@trs

Kotlin uses another builder pattern to model eDSLs like the HTML example (https://kotlinlang.org/docs/reference/type-safe-builders.html). Strangely they have not even been mentioned in the draft proposal.
The idea there is that div for example is a simple function with a closure parameter that is called on a receiver being supplied by div . The other elements h1 , p etc. are then methods of the receiver object which solves gathering the results quite nicely.

Well spotted. This has been recently discussed here in this thread: Receiver Closures

[trs (Thorsten Seitz)]

Thanks! I missed that discussion thread.

I do share the worry expressed there by @jrose that it is difficult to see what's going on. But doesn't the exact same question arise for the proposed function builders?

[davbeck (David Beck)]

My first reaction to this wasn't positive at all. But the more I think about it, the more it makes sense... if it's implemented well. I could easily see this going awry and either the compiler getting confused or unintended side effects.

[ebg]

Have you considered requiring commas or between expressions?

without:

div {
  let useChapterTitles = true
  if useChapterTitles {
    h1(chapter + "1. Loomings.")
    h1("title")
  }
  p {
    "Call me Ishmael. Some years ago"
  }
  p {
    "There is now your insular city"
  }
}

with commas:

div {
  let useChapterTitles = true
  if useChapterTitles {
    h1(chapter + "1. Loomings."),
    h1("title")
  },
  p {
    "Call me Ishmael. Some years ago"
  },
  p {
    "There is now your insular city"
  }
}

Seems a little clearer coming from other Swift code as to what is going on, and where the division is between expressions, particularly when local variables are introduced.

[AlexanderM (Alexander Momchilov)]

I have previous been toying with an idea that might be useful here: the ability for a function to produce warnings/errors at its call site. This would be useful for anything that wraps a #warning or #error.

/// A function that produces a stub value, to allow you to stub out code, but remember to come back to it.
@propogatesWarningToCallSite
func todo<T>(_ value: T, _ message: StaticString = nil) -> T {
    #warning("TODO: \(message)")
    return value
}

let inflationRate = todo(1.02, "Retrieve live inflation rate from web API") 

Which would produce a warning on the calcite:

TODO: Retrieve live inflation rate from web API

Something like this would be perfect for validating the DSLs, but it would require compile-time execution, in order to determine whether warnings/errors should be thrown or not.

[gwendal.roue (Gwendal Roué)]

@AlexanderM

Something like this would be perfect for validating the DSLs, but it would require compile-time execution, in order to determine whether warnings/errors should be thrown or not.

Yes. I did not suggest we add compile-time validation, because we're not ready for it at all.

[Tino]

That's odd: The proposal about comma elision (https://forums.swift.org/t/se-0257-eliding-commas-from-multiline-expression-lists/) got quite a lot of backlash, whereas this proposal is about eliding much more than only commas, yet seems to have no strong opposition

[Mordil (Nathan Harris)]

@Tino

That's odd: The proposal about comma elision (https://forums.swift.org/t/se-0257-eliding-commas-from-multiline-expression-lists/) got quite a lot of backlash, whereas this proposal is about eliding much more than only commas, yet seems to have no strong opposition

The impact on the language, source compatibility, premise, and scope are entirely different, even if it's doing the same "mechanics".

Comma elision is about the entire language's grammar and affects source compatibility for seemingly a minor gain - while function builders is limited to a closure with a custom annotation that provides a language feature that people can use invisibility with no runtime requirements nor concern about source compatibility. The capabilities we gain from the language feature (while doing "magic" and the same elision mechanics that were rejected in the comma proposal) unlocks more capabilities as we've seen with SwiftUI.

The comma proposal was more akin to the removal of requiring ;, and by nature more of a stylistic choice, rather than a deliberate API and architecture design unlocking feature of the language. I can name 2 use cases I want to use function builders for in my code today - but I didn't see any use cases for being able to elide commas in my code.

[Avi]

Comma elision was about making the "call site" (i.e. where the commas would have gone) look nicer. It was a change to the syntax that was both confusing and only subjectively nicer. This proposal doesn't change any Swift syntax for the user of the builder.

[Lily_Ballard]

This thread is really long and I apologize if this has already been addressed, I skimmed it and didn’t see answers:

One alternative I don’t see in the proposal is thread-local storage. This DSL looks to me to be very similar to something I’d expect to be doable by leveraging thread-local storage. The biggest difference is assigning a value to a variable suppresses it being used as a builder argument (I think), since a TLS approach can’t detect that (and presumably, using that variable multiple times later would then duplicate the element, which TLS won’t do either). Besides that, you could reproduce this DSL by having the builder set up TLS, each component add itself to TLS, and then each component modifier remove the original value from TLS and add the new value (assuming the component itself isn’t simply a reference type that’s mutated in-place).

Besides that, I also didn’t see an answer to how optional chaining works in this DSL. If I say something like foo?.bar(), my hope would be that this behaves identically to if let foo = foo { foo.bar() }, but I’m afraid that what actually happens is the builder is invoked with an Optional component.

[John_McCall]

map doesn’t need special support from the DSL feature. You just need buildBlock or buildExpression to be able to take collections.

I feel like you could fairly easy enforce the structural constraints you want on blocks through the signature of buildBlock. If you want a block to contain exactly four values, or to require the first value to be a specific type, just define buildBlock to only accept that; the diagnostics won’t be ideal at first, but that’s something we certainly ought to be able to improve on until we get to something like “error: function builder ‘blah’ requires exactly four elements in a block”. Remember that nothing about this feature requires different closures in the conceptually-same DSL to actually use the exact same builder type.

[John_McCall]

I’m not sure how TLS replaces the DSL without some amount of either boilerplate or implicit code insertion at each statement. And I didn’t get into this in the proposal, but TLS is a great example of the way that writing eDSLs without language support tends to force bad compromises on libraries; collecting values into TLS is pretty much strictly worse than collecting them into local storage, both for composability (since you need to worry about recursive use of the builder) and performance.

Optional chaining is interesting; I think you could certainly make an argument that we should put the building “inside the chain”, although you could also make an argument that that’s excessively magical.

[Lily_Ballard]

TLS would require boilerplate in each component, yes, although the standard libraries could provide helpers to abstract that away.

I’m not saying the TLS approach is better, I’m just surprised it’s not discussed as an alternative in the proposal.

[John_McCall]

I can add a discussion of that, sure.

[Lily_Ballard]

@John_McCall

Optional chaining is interesting; I think you could certainly make an argument that we should put the building “inside the chain”, although you could also make an argument that that’s excessively magical.

I do think optional chaining should be part of the builder magic, as it’s already special syntax.

Also, if it’s not part of the builder syntax, then I’m a bit concerned that something like foo?.methodThatReturnsVoid() would try to pass Optional.some(()) to the builder, which is almost certainly not desired.

[John_McCall]

That’s definitely something we can consider; I’d be curious in hearing what other people think about it.

The Void problem is something we don’t have a solution for generally, even ignoring optional chaining; I’d like to figure something out, but I don’t want to promise we can solve that in this release.

[pschuh (Parker Schuh)]

Another thing that is very related to TLS is adding support for annotating that a function takes an implicit context parameter (That gets forwarded to function calls that are annotated with the same type). This would give you something that acts the same as TLS but doesn't have as much performance overhead and is more type-safe. Because this implicit context now has a type that is known to the function, you could also use it as a vehicle over time for customizing DSL function behavior depending on what functions are defined on the context type.

[gwendal.roue (Gwendal Roué)]

With all due respect for your time and schedule, your answer does not quite address the concerns I expressed: validations that can not be expressed statically. I'm not quite sure addressing them would require a drastic rewrite of the pitch.

Edit: as a matter of fact, I didn't even dare talking about static validation, because it would be so crippled and it would have been a lost of time of all readers. I really thought runtime validation would have been easier to consider for the pitch authors, and a true improvement.

[John_McCall]

I'm sure there are constraints that would need to be enforced dynamically instead of statically, but I'm not sure you've described one, and besides, I'm not sure that a dynamic error arising at the end of a block is as bad for the dynamic-checking story as you think. But it's rather core to the design of this feature that this is an ad hoc protocol with the builder, so we could certainly find a way to allow a builder to opt in to a slightly different scheme which would be called one-by-one for the components or even carry arbitrary local state. That would be additive, so even if we didn't have it in the first version, it wouldn't preclude ever doing it.

[AlexanderM (Alexander Momchilov)]

@pschuh

is adding support for annotating that a function takes an implicit context parameter

Passing a context as a parameter to a static function would be terrible design in Swift. This is what instances are for, and that implicit context parameter is exactly what self is.

I suggested that these static functions be removed, in favour of instance methods that are called on the builder instance, where one builder instance is construct per entire DSL block.

See my previous comment:

@AlexanderM

I also do not like that this proposal uses static functions for everything, it just seems needlessly restrictive. If the thinking here is that it's intended to prevent stateful behaviour, as some kind of counter-measure against abuse. I think that's futile. If people wanted stateful behaviour, they could just use thread local storage. It's just icky.

I think it's better to initialize new instances of a function builder for every "outermost block", and to call methods on that instance. This gives access to a shared state, who knows what kind of cool uses people could come up with.