[Pitch] Result builder scoped unqualified lookup

Hello Swift community,

I have a pitch for you about adding new name lookup rules for result builders. Please share any thoughts or questions about this pitch!

Special thanks to @xedin and @hborla for meaningful feedback and design discussions!


Introduction

Result builders provide a foundation for defining declarative DSLs - domain-specific languages offering a customized syntax for working in a specific domain, such as, generating diagrams or text processing. Complex DSL APIs that leverage result builders have encountered issues with design scalability and type-checking performance, introducing a critical challenge to be solved. Extending result builders to support scoped, unqualified name lookup within their bodies, i.e. scoped name spacing for builder-specific types, will enable new API patterns that significantly reduce type-checking complexity while also improving call-site aesthetics.

Previous Swift evolution discussion: swift-evolution/0289-result-builders.md at main · apple/swift-evolution · GitHub

Motivation

Swift libraries can use result builders to define domain-specific languages, which are especially useful for declarative APIs such as regex builders in the Swift Standard Library and view builders in SwiftUI.

Result-builder-based DSLs, as the name suggests, operate on a limited, domain-specific set of inputs, typically defined as types conforming to a shared protocol. For example, @RegexComponentBuilder composes elements conforming to the RegexComponent protocol, whereas SwiftUI’s @ViewBuilder composes elements conforming to the View protocol. SwiftUI in particular makes extensive use of this pattern of pairing result builders with protocols, not just for views, but also (not limited to):

  • Scenes (@SceneBuilder/Scene)
  • Commands (@CommandsBuilder/Commands)
  • Toolbars (@ToolbarContentBuilder/ToolbarContent)
  • Table columns (@TableColumnBuilder/TableColumnContent)
  • Table rows (@TableRowBuilder/TableRowContent)

DSL components do not typically conform to more than one DSL protocol, forming distinct API families with unique component names. In this SwiftUI example, both the List and Text types conform to the View protocol, but not to any of the other DSL protocols listed above, and so are exclusive to the @ViewBuilder DSL:

public struct List<Content: View>: View { ... }
public struct Text: View { ... }

@ViewBuilder var example: some View {
    List {
        Text("Apple")
        Text("Orange")
        Text("Banana")
    }
}

DSL types are also typically declared as top-level types, like List and Text above, optimizing for succinctness and clarity at the point of use. However, this causes different DSLs to declare their component types within the same namespace, increasing the risk of type name collisions.

Alternatives Approaches Considered

Explicit DSL Namespacing

One approach for avoiding DSL type name collisions is to manually namespace types using DSL-specific prefixes or suffixes. SwiftUI’s Table API uses the Table* prefix for TableColumn and TableRow, since Column and Row are too generic on their own and would likely collide with other DSLs or libraries:

Table {
    TableColumn("With 15% tip") { ... }
    TableColumn("With 20% tip") { ... }
    TableColumn("With 25% tip") { ... }
} rows: {
    TableRow(Purchase(price:20))
    TableRow(Purchase(price:50))
    TableRow(Purchase(price:75))
}

The obvious downside of this approach is the verbosity and boilerplate of writing the Table* prefix on every line. It’s always clear at the call site that we’re building a table, so the prefixes are redundant.

The RegexBuilder module in the Standard Library was created in part due to namespacing concerns. The Regex type is currently available in the Standard Library without additional import statements. However, the regex builder DSL types are hidden behind the RegexBuilder module to not pollute the top-level namespace. This is because Regex components like One, OneOrMore and Capture had to be defined as top-level types whereas they only make sense within a regex builder DSL block.

Sharing DSL Components

Another approach for avoiding name collisions is to define a single component that is shared across multiple DSLs. This works best when the shared component serves a consistent purpose within each DSL.

There are several examples of shared DSL components in SwiftUI, including ForEach, Group, and Section. For example, you can define sections of views in a List:

List {
    Section("Dogs") {
        Text("Bulldog")
        Text("Beagle")
        Text("Poodle")
    }
    Section("Cats") {
        Text("Bengal")
        Text("Sphynx")
        Text("Siamese")
    }
}

You can also use Section to organize rows in a Table:

Table {
    TableColumn("Breed") { ... }
    TableColumn("Species") { ... }
    TableColumn("Description") { ... }
} rows: {
    Section("Dogs") {
        TableRow(Pet.bulldog)
        TableRow(Pet.beagle)
        TableRow(Pet.poodle)
    }
    Section("Cats") {
        TableRow(Pet.bengal)
        TableRow(Pet.sphynx)
        TableRow(Pet.siamese)
    }
}

This approach avoids the need for DSL-specific prefixes, e.g. no TableSection type, and helps developers transfer their skills from one domain to another, instead of having to relearn how to implement the same concept in each DSL.

Reusing components across DSLs requires a top-level type that conforms to multiple DSL protocols. Because Section is a container type, it uses conditional conformances based on its content type, with each extension defining an equivalent initializer that uses the correct result builder associated with the protocol:

// Note: This example is not consistent with SwiftUI's public API, using 
// a simplified declaration for illustrative purposes.

public struct Section<Content> {}

extension Section: View where Content: View {
    public init(_ title: String? = nil, @ViewBuilder content: () -> Content)
}

extension Section: TableRowContent where Content: TableRowContent {
    public init(_ title: String? = nil, @TableRowBuilder content: () -> Content)
}

While there are clear benefits to this pattern mentioned above, there are also several significant weaknesses:

  • Sharing types also means sharing ABI and storage. Some DSLs may share a concept, but require different generic type parameters or different stored properties.
  • A new DSL defined in a separate module may also not be able to extend an existing shared type. For example, RegexBuilder and SwiftUI are independent libraries, so it doesn’t make much sense for both to share a Group type if both DSLs wanted a Group component.
  • Since sharing types is not always an option, name collisions are still possible. Sharing types encourages abstraction, favoring lowest-common-denominator names that are even more likely to collide. For example, if RegexBuilder wanted to add its own Group type, clients that import both SwiftUI and RegexBuilder would need to prefix every use of Group with either SwiftUI. or RegexBuilder..
  • The pattern shown above of overloading interfaces across conditional conformances can severely impact type-checking performance.

The negative impact on type-checking performance, in particular, imposes an effective cap on the scalability of this pattern.

Type-Checking Performance for Shared DSL Components

Consider the Section declaration example from above. At the call site, each extension’s initializer uses the same trailing-closure syntax:

Section {
    SomeContent()
}

Since there is no other available hint, the only way to resolve the type of this Section instance is to first resolve the type of SomeContent. This bottom-up type inference means that the type checker must collect and keep track of all the possible outcomes at each level in the expression tree of DSL content, until it resolves the types of the leaf content and can then work its way back up. Because the language allows result builders to be arbitrarily composed, all overloads of the Section initializer must be attempted, and overload resolution can only fail once it reaches a leaf component that does not meet the requirements of the given result builder.

In other cases, DSL types must be resolved collectively or top-down. For example, the real TableRowContent protocol has a TableRowValue associated type, and @TableRowBuilder is generic on a row value that all of its rows must share. This means that in reality, Section's conditional conformance to TableRowContent has several more constraints than shown above:

// Note: This example is not consistent with SwiftUI's public API, using a
// simplified declaration for illustrative purposes.

extension Section: TableRowContent where Content: TableRowContent {
    public typealias TableRowValue = Content.TableRowValue

    public init<V>(
        _ title: String? = nil,
        @TableRowBuilder<V> content: () -> Content
    ) where TableRowValue == V
}

These collections of generic constraints can form webs of semi-circular dependencies: a table section’s type depends on its content’s type, but the content’s type depends on the builder’s type, and the builder’s type must be consistent with the section’s type!

The type checker is often able to figure it out, at the cost of increased checking time and memory use, but in some cases fails and must emit an “unable to type-check expression in reasonable time” error. Clients can always work around these errors locally by providing more explicit type information, or by breaking their code into smaller expressions, but this is often a frustrating, trial-and-error-driven process.

Further, while Section currently conforms to only two protocols, other lower-level components like ForEach and Group should theoretically be available in most or all current and future SwiftUI DSLs; Group currently conforms to eight DSL protocols, following the same conditional conformance pattern shown above.

Some of these performance issues could be addressed by abandoning shared DSL types and adopting the verbose prefix-based approach instead, e.g. TableSection, TableForEach, TableGroup, etc. However, an ideal solution would support DSL namespacing with unqualified lookup, as well as scalable type-checking performance.

Proposed solution

This proposal introduces new unqualified name lookup rules that allow unqualified names used inside result builder bodies to find declarations inside the result builder type.

This approach also allows DSL authors to combine builder-scoped names and explicit DSL prefixes on global types to enable concise, scoped component names while still sharing an implementation. For example, TableRowBuilder and TableColumnBuilder can enable the concise component names Row and Column in their respective DSLs by introducing a scoped typealias to the long-form name:

@resultBuilder 
struct TableColumnBuilder {
  typealias Column = TableColumn 
  ... 
}

@resultBuilder 
struct TableRowBuilder {
  typealias Row = TableRow 
  ... 
}

Table {
   Column("With 15% tip") { ... }
   Column("With 20% tip") { ... }
   Column("With 25% tip") { ... }
 } rows: {
    Section("Dogs") {
        Row(Pet.bulldog)
        Row(Pet.beagle)
        Row(Pet.poodle)
    }
    Section("Cats") {
        Row(Pet.bengal)
        Row(Pet.sphynx)
        Row(Pet.siamese)
    }
 }

This approach also allows DSLs to declare concise components that do not make sense to introduce at the top-level. In the below example, the @HTMLBuilder type can create names for DSL components without polluting the global namespace with these declarations that are unlikely to be used in other contexts. In the result builder body, at the use-site for the unqualified name div, the compiler will search the result builder type scope as if this were a qualified lookup, e.g. HTMLBuilder.div . By restricting unqualified names to look within the result builder context, type-checking time will be scaled down significantly because scoped DSL components are only discoverable within the DSL body (without explicit qualification), including in editing tools like code completion.

protocol HTML { ... }

@resultBuilder
struct HTMLBuilder {
  typealias Component = any HTML
  static func buildBlock(_ component: Component...) -> [Component] { ... }
  
  // Standard HTMLBuilder components
  static func body(@HTMLBuilder _ children: () -> Component) -> Component { ... }
  static func div(@HTMLBuilder _ children: () -> Component) -> Component { ... }
  static func p(@HTMLBuilder _ children: () -> Component) -> Component { ... }
  static func h1(_ text: String) -> Component { ... }
}

@HTMLBuilder
var body: [HTML] {
  div {
    h1("Chatper 1. Loomings.")
    p {
      "Call me Ishmael. Some years ago"
    }
    p {
      "There is now your insular city"
    }
  }
}

Detailed design

Declaring API that can be found via unqualified name lookup in result builders is done by writing the declaration in the scope of the result builder type.

@resultBuilder
struct Builder {
  static func buildBlock(_ values: Any...) { ... }
  
  // ScopedValue can be found by unqualified lookup inside
  // @Builder bodies.
  struct ScopedValue { ... }
}

Declarations in extensions of the result builder type can also be found via unqualified lookup in a result builder context:

extension Builder {
  // AnotherValue can be found by unqualified lookup inside
  // @Builder bodies.
  struct AnotherValue { ... }
}

Any declaration that can be nested in a type and accessed with qualified lookup on the result builder type, e.g. Builder.ScopedValue, can be found using the unqualified name in result builder context, including:

  • Types
  • Type aliases
  • Static functions
  • Static properties

For an unqualified name written in a result builder, name lookup will first look inside the result builder type, e.g. as if the programmer had written Builder.name. If a declaration of that component name is found, that result is used and other lexical lookup results will not be considered. If a declaration of that component name is not found, lookup will fall back to lexical lookup. This approach is similar to shadowing; even in cases where a type is declared in the global scope and then again in a result builder type, the innermost scope will take precedence over the outer scope. Note, the outer lookup results will not be considered if picking the shadowed declaration fails to type check:

// Standard shadowing

struct S: Equatable {}

let globalS = S()

struct Parent {
  struct S {}

  func shadow() -> Bool {
    S() == globalS // error: Binary operator '==' cannot be applied to operands of type 'Parent.S' and 'S'
  }
}

// Result builder shadowing

protocol Component {}

struct Value: Component {}

@resultBuilder
struct Builder {
  static func buildBlock<C: Component>(_ components: C...) -> [C] {
    return components
  }

  struct Value {}
}

@Builder
var body: [some Component] {
  Value() // error: Static method 'buildBlock' requires that 'Builder.Value' conform to 'Component'
}

In cases where the inner result fails to type check, the outer result can still be used by writing a qualified name, e.g. a global declaration Value can be qualified with the module name MyModule as MyModule.Value.

Source compatibility

This is technically a source breaking change. If a declaration already nested inside a result builder type has the same name as another declaration that can be used inside a body with that result builder applied, existing code using that name inside the result builder body will find the nested declaration:

@resultBuilder
struct Builder {
  static func buildBlock(_ values: Any...) { ... }
  
  struct Value { ... }
}

struct Value { ... }

@Builder
var body: Any {
  // Lookup of 'Value' in '@Builder' context will change from the 
  // global 'Value' to 'Builder.Value'
  Value() 
}

This could be mitigated by including lexical lookup results in an overload set with the inner results found inside the result builder type. This would alleviate source breakage in the case where the builder-scoped declaration fails to type check when used in the result builder, but name lookup results will still silently change if using the scoped declaration is well-typed.

This lookup behavior is consistent with the current behavior of shadowing in the language; considering outer lexical lookup results can be considered generally as a future direction.

Effect on ABI stability

This change has no impact on ABI stability.

Effect on API resilience

This feature does not add any new API resilience rules. Adding new types or functions in result builder types (with appropriate availability) is a resilient change. Moving existing global types used in result builders inside the result builder is ABI breaking.

48 Likes

This is an interesting approach that solves the stated problems quite nicely. If I understand correctly, one could think of it as if the symbols available in the result builder function scope are the same as in the scope of an extension to the result builder type:

@ShapeBuilder
func square() -> some Shape {
    // ShapeBuilder.Rectangle
    Rectangle(width: 100, height: 100)
}

extension ShapeBuilder {
    static func square() -> some Shape {
         // ShapeBuilder.Rectangle
        Rectangle(width: 100, height: 100)
    }
}

That leads to one question: would the buildXXX methods provided by the builder also be available unqualified (suggested by auto complete etc.) in the scope or somehow hidden? It could be unexpected for builder-related implementation details to be available and suggested by autocomplete (and possibly be preferred over e.g. my own buildBlock method for some other purpose in an outer scope (such as a method decl in a SwiftUI view struct).

struct ReportView: View {
    
    var body: some View {
        build|
        //   ^ autocomplete
        //         buildBlock(…)
        //         buildEither(…)
        //         buildFinalResult(…)
        //         …
        //         buildReport()
    }

    buildReport() -> some View {
    // …
    }
}

[Edit: added autocomplete example]


An alternative, more extensible approach that I could see is if rather than being specific to the members of the result builder type, the result builder could have a Namespace (pseudo?) associated type sort of thing, defaulted to itself.

In essence:

// Imaginary pseudo-protocol
protocol ResultBuilder {
    associatedtype Namespace = Self
}

This way the default behavior would be exactly as prescribed, but would give the option of specifying a different namespace container using either a nested type or a typealias:


@resultBuilder
struct PublisherBuilder {
    typealias Namespace = Publishers
}

This would allow the result builder author to keep a separation between the result builder’s public interface and the namespace (e.g. an enum) given to the scope if they so wished. For example, this would allow the author to use a custom namespace that is more natural for uses of the type outside of the result builder, without needing to use typealiases to ‘import’ every single type into the result builder’s namespace.

10 Likes

Yes please! I’ve been wanting exactly this for exactly this use case ever since result builders were introduced.

10 Likes

This looks like a step forward. I also like the idea of explicitly declaring a namespace type suggested by @codename.

One problem with result builders is that it is bothersome to look up all possible inputs / input types allowed inside the builder. What I would like is a way to utilize autocomplete to get a listing of all possible inputs to the builder. Maybe this could look something like this:

@HTMLBuilder
var body: [HTML] {
  .div {
    .h1("Chatper 1. Loomings.")
    .p {
      "Call me Ishmael. Some years ago"
    }
    .p {
      "There is now your insular city"
    }
  }
}

The idea is that if I started typing ., the completions body, div, p and h1 would be suggested.

This would also be more in line with how static member lookup works elsewhere in the language. Not as aesthetically pleasing, but more consistent and more discoverable.

The unfortunate challenge with static member syntax in result builders is that expressions are intended to be declared consecutively, in which case the . is interpreted as a chained method on the previous expression rather than a following static method on e.g. the builder type, forcing you to use semi-colons to get the intended behavior.
From your example (added comments and semicolon):

.div {
    .h1("Chapter 1. Loomings."); // -> HTML
    .p { // interpreted as instance method on HTML type without semicolon
      "Call me Ishmael. Some years ago"
    }
    // …
}

From a discoverability perspective (with autocomplete) nothing beats dot notation, but I don’t think it’s a great solution with result builders.

8 Likes

I didn‘t think of that. Thanks for the clarification. Maybe there is another solution to allow autocompletion…

3 Likes

A neat effect of this pitch (or equally my suggestion) is that documentation-wise, there would be a namespace in which you could find all types made available (with unqualified access) in the result builder scope, which I think improves discoverability to some extent.


One of my concerns with using the result builder type itself as that namespace is that while I expect this to be a useful feature in the documentation structure, I don’t think a *Builder type is going to be anyone’s first place to look for what symbols are available to use. With my suggestion, even were someone to look at the result builder they would conveniently be directed to the namespace type via the Namespace (or what have you) typealias declaration.

Using the result builder type as the namespace for the scope will mean many small libraries could largely become contained within the builder type, which I view more as an oft-hidden (at the use site) utility type (similar to a string interpolation type) than the centerpiece of an API (even if a powerful one).

It is true as pitched that for types (but not functions), the symbol available in the scope could simply be a typealias to the actual type rather than actually beating the types, but I think that would still be the norm as typealiases do not carry all the documentation of the original type and it would be less easily maintainable (creating a typealias for every type that is introduced in a library so it is available in a result builder’s scope).

To expound on my previously mentioned concern, for certain libraries while types may be primarily intended to be used in a result builder, they could also be used in other ways (akin to React without JSX), in which case it would be awkward IMO to be namespaced within the builder type:

root.render(HTMLBuilder.p {
    "Some paragraph"
})

// vs

root.render(HTML.p {
    "Some paragraph"
})

In fact, for certain DSLs (such as this example), the best namespace could be an original type like HTML, depending on whether they’re intended to be used outside the DSL or not.

I think this is a reasonable idea but the source compat break worries me quite a bit. By being a flagship feature of SwiftUI result builders are used a lot.

2 Likes

Would it be possible we make this feature explicitly opt-in by an attribute like @resultBuilderLookupable? This will also reduce the source-breaking worries from @jrose since existing codes keep unaffected.

1 Like

I agree the current behavior and workaround here are not ideal. It really is a shame implicit member syntax doesn't work in this position. :slightly_frowning_face:

I'm thinking about this pitch with regards to Doug's post from the if/switch expressions review about replacing the magic of result builders with more general source-generation features:

On its face, it seems like this would dig us deeper down the hole of 'magic' for supporting an ergonomic experience for result builders, and its not immediately clear how we'd dig ourselves back out if we wanted to achieve the goal of making result builders be 'just' a macro feature.

So I have a couple different thoughts on the pitch as-is:

  • Are there existing language language features that we could improve to work around the issues presented here? Implicit member syntax almost works, is there any way we could tweak parsing rules to get it fully working, or not?
  • If not, is there a more general language feature looming behind this extension of result builders? This feels very similar to how Kotlin lets you rebind this within a closure scope so that lookup can find names outside the strict lexical context. We don't necessarily need to make that part of this pitch, but it would be nice to know that we're not introducing a feature just for result builders that will make it significantly more difficult to generalize them later.
4 Likes

How about opting in by defining the Namespace typealias?


Directly translating this into Swift, this would look like self.h1("Chapter 1. Loomings."). Altenatively, how about something like

$builder.h1("Chapter 1. Loomings.")

Or an even shorter version:

$.h1("Chapter 1. Loomings.")

This could be a feature orthogonal to result builders introduced via a separate attribute.


Thinking about it: Even if we would settle on the pitched spelling of h1("Chapter 1. Loomings."), it would make sense to make this a feature separate from result builders that can be used in other contexts without result builders.

1 Like

Maybe this could look something like this:

@resultBuilder
@namespaceProvider // To declare a namespace provider type
struct HTMLBuilder {
  typealias Component = any HTML
  typealias namespace = Self // This declares the actual namespace
  static func buildBlock(_ component: Component...) -> [Component] { ... }
  
  ...
}

It could be used just as pitched:

@HTMLBuilder
var body: [HTML] {
  h1("Chapter 1. Loomings.") // Alternatively: $.h1("Chapter 1. Loomings.")
  p {
    "Call me Ishmael. Some years ago"
  }
}

Types that are not result builders could use this as well:

@namespaceProvider
struct HTMLNamespace {
  typealias namespace = HTMLNames
}

enum HTMLNames {
  static func h1(_ text: String) -> HTML { ... }
}

@HTMLNamespace
var body: [HTML] {
  return [
    h1("Chapter 1. Loomings.") // Alternatively: $.h1("Chapter 1. Loomings.")
  ]
}
2 Likes

I prefer this to an attribute. Also, even with this approach, users can combine multiple namespaces, by creating protocols for each one and then sharing them.
E.g.

protocol HTMLDiv {}
extension HTMLDiv {
  static func div(@HTMLBuilder _ builder: () -> HTMLValue) -> HTMLValue { … }
} 

protocol HTMLHeaders {}
extension HTMLHeaders {
  static func h1(_ text: String) -> HTMLValue { … }
  // h2, h3, etc.
} 

@resultBuilder enum CustomHTMLBuilder {
  enum Namespace: HTMLDiv, HTMLHeaders {}
}

This reminds me of the SwiftPredicates pitch, where one of the main reasons for using macros is that operator overloading is too slow. Maybe with scoped unqualified lookup, the macro could be replaced with standard, well understood features for adding custom operators.

How will users opt out of name lookup in the result-builder namespace? Also would a type in the namespace be prioritized over a module name, e.g. ThisModule.div or Swift.OneOrMore?

+1 on this, i think this is a very elegant idea.

Or drop the $ maybe?

{ builder in 
    builder.h1("Chapter 1. Loomings.")
}

Isn't this just a tooling issue? Why can't Xcode or your IDE of choice autosuggest all possible builder elements inside a builder's context? (i.e. by pressing the escape key)

4 Likes

I don’t think this fulfills the goals of the original pitch which alludes to whole DSLs (such as SwiftUI or at least new DSLs) adopting namespacing in this way, with the goal of still being as fluent and concise as possible. If e.g. every view declaration within a ViewBuilder function mandatorily started with such a prefix token, that would be worse than just using prefixed names IMO (e.g. TableColumn vs $builder.Column).

This would be great. Currently, the relevant suggestions are drowned in a sea of global definitions…

as i understand it, the purpose of the $ prefix is to prevent source breakage from introducing new symbols to the lookup scope of the result builder body. in my opinion, $builder is not needed, we should just have:

{
    $h1("Chapter 1. Loomings.")
}