Function builders

Indeed, implementation is trivial; my question is just whether it is / should be a part of the proposal.

Apologies if this has been considered above already (this is a very long thread and I am still working my way through it)

Have we considered just keeping a stack of some common currency type (for a particular builder), and then passing each statement to a build function which transforms it to that common currency type and places it on the stack. If we have each build function take the stack inout, then optimizations should still be possible.

This should have several advantages:

  1. You should be able to use normal control flow (skipped expressions just won't have a chance to alter the stack)
  2. You don't run into the same 10 children are ok, but 11 are not problem (without needing variadic generics)
  3. You should be able to type check each expression individually (based on whether there is a matching build function for that expression)
  4. Build functions can make optimizations as they go by looking at the stack
  5. It still allows arbitrary nesting
1 Like

I should mention this is the approach I took in my own DSL, and it works really well for my needs. It even allows you to do very strong runtime error handling (It is a parser that can parse around the error and return meaningful human readable error messages along with a range pointing out where the issues are)

I also find the coroutine approach interesting. Perhaps instead of using {: }, we could have an @autoyielding annotation (at the declaration site instead of call site) that would cause it to automatically yield in appropriate contexts.

That said, I still prefer the stack based approach, as it has a good power to complexity ratio in my experience. Basically we would be bringing the power of something like Forth or Factor into our DSLs...

There are a lot of comments in this thread that discuss alternatives that are not capable of capturing the detailed type information this proposal is capable of capturing. It’s fine to prefer these approaches. They work very well in many contexts! But I think it’s important to address the elephant in the room. What would have SwiftUI do? How would you modify the design of SwiftUI such that it is compatible with that kind of DSL support?

3 Likes

I would argue that the Controller is now able to be broken up into lots of reusable pieces (i.e. Combine) instead of being a massive cluster that has to be rewritten for each app.

I had written my own version of this a couple of years ago (I called them Chains), and have been happily using that pattern in most of my code. Was working on open sourcing those, but with Combine doing it better, there is no need. lol.

It should still be able to capture the same type information when needed (see post above), however some of the optimizations might be pushed to runtime (until we get compile-time functions). SwiftUI would look the same at the use-site, but have a slightly different approach for builder declarations (defining functions taking individual lines and only a single build-block). The advantage is that you could have things like variable assignment and control-flow statements natively, and you wouldn't have the 10 item limit. It could also be done in a reasonably short timeframe.

Which post? Are you able to provide examples? I would like to see a sketch of how the feature would be used by a library like SwiftUI so I can better understand how it is capable of meeting the needs of a library like that.

If you mean this post I don’t see how a “common currency type” for the builder could meet the needs of SwiftUI. It’s typing requirements are much more sophisticated than anything I can imagine meeting that description. But maybe I’m missing something.

I suppose it depends on exactly what optimizations it needs, but remember that that common currency type could be a protocol, class, or enum that can easily bucket things based on various optimization opportunities... and since it is internal to the builder type, it can easily be changed in the future when new opportunities arise.

More importantly, a stack lets you lay down hint/control tokens that can affect the way the other components are used/read/etc...

Basically, you can kind of think of it kind of like a tokenizer, and then the build block becomes a parser of those tokens. Very powerful. Should have enough power for SwiftUI (but I could be wrong because I haven't seen the internals).

But doesn't currency type type-erase the underlying expression?
So the type information is lost.

I think it would be helpful for everyone involved in this discussion if folks involved in SwiftUI would be willing to comment further on how preserving type information is important to the design and implementation. @dabrahams is there any way that would be possible?

5 Likes

There is still full type information in the functions which transform it to that currency type, so many optimizations could be performed there. There are also a lot of tricks that you can do with the stack to allow recovery of type information (e.g. all of the statements after this token have the same type), which can allow cross-statement optimization.

I am making an assumption here that most optimizations can be placed in buckets of functionality (e.g. group these things in metal) and shouldn't need full type info beyond which optimization bucket is appropriate.

Yes, fully agreed!

Subscripts can capture type information and it would be arguably simpler to comprehend. It provides good distinctions between eDSL codes with native codes and is just standard Swift syntax. It doesn’t create a context around the arguments but it can be manually added through immediately called blocks.

@anandabits @John_McCall

I also feel like I need to clarify one point that I didn’t make clear before: when I say Stack, I do NOT mean array. With an actual stack it is possible to carry forward type information or to erase it as desired.

Let me give a concrete example where I use this to propagate the type info from the top item of the stack (it should be possible to do the same thing with the entire stack of type info with a related design).

I have a drawing framework that has the concept of a “DrawContext”. This is a pair of protocols that provide access to a CGContext as well as a number of values used in things like animation. The stack nodes are implemented as a bunch of types which adhere to the protocols, and which know how to wrap another node.

There are a series of functions which make the wrapping process much cleaner, and they all return the actual type created as opposed to type erasing to the protocol. In some cases, those functions (as well as initializers) are overloaded so they can take advantage of that type information (e.g. concatenation of transforms into a single layer)

If I want a type erased version, I simply store in a variable with the protocol type. The end result is that I have an efficient functional way to represent a drawing environment (including transforms, clipping, etc...).

It should be possible to capture the whole stack’s type info by making nodes where the type is dependent on what they are wrapping. It should also allow partial type erasure, etc...

The end result should be powerful enough for whatever SwiftUI needs, but that complexity can disappear for simple use cases like HTML Builder.

Does that make more sense?
(I sometimes forget that others don’t have access to the context of my ideas)

1 Like

With the current builder functions, it executes builder functions as it exits scopes. That's a depth-first post-order traversal (but just of the blocks that are executed), but only because we don't yet support any looping control flow.

That's true. We could certainly ignore builder functions that aren't defined in the builder's module, although that seems like the sort of extension that people will end up complaining about.

I think I'm leaning towards the same conclusion: there's really no reason not to use instance methods for building, since clients like SwiftUI that don't need state tracking can of course just use an empty type. It would require every builder type to provide a static startFunction() method (or maybe an init?) that creates the builder instance, but that seems like a fairly modest imposition.

Builders like SwiftUI that want to propagate typed values for each block can define finishBlock methods that then cause the results from the block to be collected into locals (thus requiring buildConditional and so on) so they can be passed finishBlock later, while builders that just want to internally collect values as they come can just define buildExpression to collect the value.

This would capture most of the expressible behavior of receiver closures except for the ability to initialize the builder context dynamically by passing it in as a parameter. I don't know how important that is in practice to its use cases in Kotlin, and the implicit contextual data dependency seems pretty unfortunate in some ways.

12 Likes

If we can teach the builder to ignore Void-typed expressions, we can teach it to ignore uninhabited types like Never as well, and that seems reasonable.

I don't think we can reasonably do things like have the builder ignore results that appear in unreachable code.

4 Likes

SwiftUI wants to propagate information about component types into the type of the built result, not just propagate information about component types temporarily into the builder instead of erasing at an earlier stage. At a very low level, this allows the in-memory representation of a stack of, say, an A, a B, and a C to be essentially an (A, B, C) tuple instead of having to store type information and potentially boxing at each level, which is important for the memory performance of the system given its heavy use of type-recursion and decorators; but more importantly it provides a guarantee that the type structure won't change (except at specific places that request it with AnyView), which makes structural analysis and value-comparison far more efficient.

At least, that's how I understand it; I am a language developer, not a SwiftUI expert.

12 Likes

Here is some very simple code to show it is possible to store all type information:

protocol Stack {
    func topValue<V>(of type:V.Type) -> V?
    var popErased:Stack? {get}
}

extension Stack {
    func push<V>(_ value:V) -> Node<Self,V> {
        return Node(parent: self, value: value)
    }
}

struct Root<T>:Stack {
    let topValue:T
    
    init(value:T) {
        self.topValue = value
    }
    
    func topValue<V>(of type: V.Type) -> V? {
        return topValue as? V
    }
    
    var popErased:Stack? {
        return nil
    }
}

struct Node<P:Stack,T>:Stack {
    let pop:P
    let topValue:T
    
    init(parent:P, value:T) {
        self.pop = parent
        self.topValue = value
    }
    
    func topValue<V>(of type: V.Type) -> V? {
        return topValue as? V
    }
    
    var popErased:Stack? {
        return pop
    }
}

let nodeA = Root(value: "One")   // Type: Root<String>
let nodeB = nodeA.push(2)        // Type: Node<Int, Root<String>>
let rootAgain = nodeB.pop        // Type: Root<String>
let val = nodeB.topValue         // Type: Int
let rootVal = rootAgain.topValue // Type: String
let erased:Stack = nodeB         // Type: Stack
1 Like

Sure, I don't disagree that it's possible to simulate some effects of variadic generics through your own generic pair types. The Either approach used in this proposal in fact simulates a variadic sum type by folding to a tree of pairwise sums. I think this would be much more awkward for some DSL clients, but I don't know if there's any reason we couldn't use this for SwiftUI.

1 Like