Pitch: Scoped Functions

Pitch: Scoped Functions

Scoped Functions are functions that enhance the leading dot syntax inside their body, so that it gives short-hand access to the members of a specific value. Such functions exist so that API designers and their consumers can collaborate towards a way to write focused code where an agreed implicit and obvious context does not have to be spelled out.

For the posterity, the initial pitch is below.

Initial pitch

Introduction

Scoped Functions are functions that enhance the leading dot syntax inside their body, so that it gives short-hand access to the members of a specific value. Such functions exist so that API designers and their consumers can collaborate towards a way to write focused code where an agreed implicit and obvious context does not have to be spelled out.

Motivation

This pitch considers that developers often want to use short-hand identifier resolution in particular contexts, and api designers often want to make it possible.

The Swift language already provides much help in this regard. We have the implicit self, the current (and evolving) rules for the leading dot syntax, the scoping rules for accessing nested types, etc.

But developers always want more:

In all those examples, developers express a desire to enhance identifier resolution in a lexically scoped context: a value, or the body of a closure, where both are arguments of a function.

Considering:

  • The braces of a closure body are a well delimited lexical scope.
  • A function call is a strong semantic signal that is able to define an implicit and obvious context that both api designers and consumers can agree on.
  • Naked identifier resolution in Swift is already pretty busy with local variables, self, type names, module names, etc, so we do not propose modifying this.
  • The leading dot syntax is a beloved short-hand syntax of Swift.

We modify the rules that govern leading dot syntax inside the body of a particular closure argument. When the closure is an autoclosure, the leading dot syntax is modified for the value that feeds this autoclosure.

This gives:

// The build(_:configure:) function accepts a closure 
// scoped on its first argument:
let label = build(UILabel()) {
    .text = "Hello scoped functions!"
    .font = .preferredFont(forTextStyle: .body)
}

// The `filter(_:)` and `order(_:)` functions accept an 
// autoclosure scoped on the generic type of the request
// (here, `Player`).
// `Player` collaborates, and has defined two
// `Player.score` and `Player.team` static constants.
let players = try Player.all()
    .filter(.team == "Red")
    .order(.score.desc)
    .limit(10)
    .fetchAll(from: database)

// A mix and match between result builders and
// scoped functions. Each closure is scoped on an
// object that defines the relevant leading-dot apis 
// inside the body.
let html = HTML {
    .head { .title("Hello scoped functions!") }
    .body { ... }
}

// A revisit of SE-0299: the argument of toggleStyle(_:) is
// an autoclosure scoped on an ad-hoc enum type:
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle(.switch)

Without scoped functions, developers face various problems:

// The `label` variable is repeated.
let label = UILabel()
label.text = "Hello World!"
label.font = .preferredFont(forTextStyle: .body)

// The `Player` type is repeated.
let players = try Player.all()
    .filter(Player.team == "Red")
    .order(Player.score.desc)
    .limit(10)
    .fetchAll(from: database)

// Typesquatting: the `Head`, `Title`, `Body`
// identifiers can't be used for other purposes.
let html = HTML {
    Head { Title("Hello World!") }
    Body { ... }
}

// As before SE-0299
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle(SwitchToggleStyle())

Personal motivation: as the author of GRDB, which defines a type-safe API for building SQL queries, and relies of static properties defined on user-defined "record types", I'd love to simplify some user-land code:

// CURRENT GRDB CODE
struct Team: Codable, FetchableRecord {
    static let players = hasMany(Player.self)
    static let awards = hasMany(Award.self, through: players, using: Player.awards)
}
struct Player: Codable, FetchableRecord {
    static let awards = hasMany(Award.self)
}
struct Award: Codable, FetchableRecord {
}

// All red teams with their awarded players
let request = Team
  .filter(Team.Columns.name == "Red")
  .including(all: Team.players.having(Player.awards.count > 0))

// WITH SCOPED FUNCTIONS
struct Team: Codable, FetchableRecord {
    static let players = hasMany(Player.self)
    static let awards = hasMany(Award.self, through: players, using: .awards) // <-
}
struct Player: Codable, FetchableRecord {
    static let awards = hasMany(Award.self)
}
struct Award: Codable, FetchableRecord {
}

// All red teams with their awarded players
let request = Team
  .filter(.name == "Red")                             // <-
  .including(all: .players.having(.awards.count > 0)) // <-

Sorry if I provided a contrieved example. But contrieved examples are precisely those where unnecessary clutter obscures the intent the most.

Proposed solution

The syntax for defining the type of a function is extended. When the arguments list is prepended with a type T and a dot, as in T.(...) -> ..., then this is the type of a scoped function, and T is the scope type. Such a scoped function accepts an extra first argument, of type T, which is the scope value. Inside the body of the function, the scope value feeds the leading dot syntax. The function is said to be T-scoped.

For example:

let f: String.() -> Int // A
f = { return .count }   // B
f("Hello")              // C: prints "5"
  • In (A), f is defined as a String-scoped function.

  • In (B), the compiler is able to resolve .count as String.count.

  • In (C), the function is called with a particular String.

There is no syntax for defining a scoped function from scratch, as in func f T.(...) -> ... { ... }. Scoped functions are designed to be used as closure arguments, and not free functions or methods. The functions that accept scoped functions as arguments define the "agreed, implicit, and obvious context" that is key for this pitch.

We are able to address some of the above examples:

  1. The build(_:_:) function

    func build<T>(_ value: T, with transform: T.() throws -> ()) rethrows -> T {
        return transform(value)
    }
    
    let label = build(UILabel()) {
        .text = "Hello scoped functions!"
        .font = .preferredFont(forTextStyle: .body)
    }
    label.text // "Hello"
    

    Woot, we'll have to define how the compiler deals with leading dot syntax for .font, .preferredFont, and .body :-)

  2. The database query

    struct Request<T> {
        func filter(_ expression: @autoclosure T.Type.() -> Expression) -> Self { ... }
        func order(_ ordering: @autoclosure T.Type.() -> Ordering) -> Self { ... }
        func limit(_ limit: Int) -> Self { ... }
    }
    
    extension Player {
        static let team = Column(...)
        static let score = Column(...)
    }
    
    let request: Request<Player> = ...
    request.filter(.team == "Red").order(.score.desc)
    

    The @autoclosure qualifier has the compiler change the leading dot scoping for values.

Detailed design

TBD

Source compatibility

Scoped function are an additive feature that creates no source compatibility issue.

Effect on ABI stability

A scoped function, at runtime, is a plain function that accepts its scope as the first argument. Its a compiler-only feature, without any effect on the ABI.

Effect on API resilience

TBD

Alternatives considered

Use another sigil than . ? For example, ':

// The build(_:configure:) function accepts a closure 
// scoped on its first argument:
let label = build(UILabel()) {
    'text = "Hello scoped functions!"
    'font = .preferredFont(forTextStyle: .body)
}

// The `filter(_:)` and `order(_:)` functions accept an 
// autoclosure scoped on its receiver type.
// `Player` collaborates, and has defined two
// `Player.score` and `Player.team` static constants.
let players = try Player
    .filter('team == "Red")
    .order('score.desc)
    .limit(10)
    .fetchAll(from: database)

// A mix and match between result builders and
// scoped functions. Each closure is scoped on an
// object that defines the relevant leading-dot apis 
// inside the body.
let html = HTML {
    'head { 'title("Hello scoped functions!") }
    'body { ... }
}

// A revisit of SE-0299: the argument of toggleStyle(_:) is
// an autoclosure scoped on an ad-hoc enum type:
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle('switch)

The latest version of the pitch is at Pitch: Scoped Functions · GitHub

14 Likes

In that case, I'm not sure if scoped function should be a type per se, especially that it carries no additional semantic meaning — the impact is purely syntactical. The examples that you list are also particular to the "longer" kind of closures (i.e. those that are spelled more than the simple { $0 + 1 }, for example), so I believe it's the closure definition, not its type that should choose an argument to be scoped on. Especially that the initial choice of the argument to be scoped over may misalign with the user's wishes to rather make the function scoped over another argument of their choice (because it's not really possible to know for library authors what argument will be accessed the most and require the most syntactic sugar).

Which leads me to an alternative suggestion: provide a @scoped attribute that could be applied to one of the explicit arguments in the list:

forEach { @scoped item: String in
    print(.count)
}

— could we possibly explore that?

(I know that this doesn't really align with the GRDB examples you provide, but I also find them quite hard to parse because to my knowledge .filter wants a SQLExpressible, which is not a function type, doesn't it?) Oh, it's an autoclosure, I see. Given this, it really looks too magical, because an autoclosure is both such a niche case and feels to me more like an implementation detail. It's also quite hard to spot (even impossible, perhaps?) with current Xcode autocompletion.


Variadic arguments could be a valid counter-example to my objections though.

1 Like

What happens if .head { ... } has a body property?

3 Likes

I guess GitHub - devxoul/Then: ✨ Super sweet syntactic sugar for Swift initializers is another example that scoping is something people want

1 Like

I think @implicit is a better name for it, albeit longer, but I think it would be most useful if you could apply it on the declaration of the closure parameter in addition to it's definition.

For example, I think this should work:

func with<T>(_ initial: T, update: (@implicit inout T) -> Void) -> T {
    var value = initial
    update(&value)
    return value
}

let answer = with("42") {
    .append("!")
}

While also allowing your forEach example.

1 Like

You are correct. A scoped function, that I wrote T.(...) -> ... could also be spelled @scoped (T, ...) -> ..., or even (@scoped T, ...) -> ....

Arguably @scoped (T, ...) -> ... can be considered a superior alternative, since it follows existing precedent (@escaping, etc.)

I'll use this syntax from now on, and update the pitch.

The examples that you list are also particular to the "longer" kind of closures (i.e. those that are spelled more than the simple { $0 + 1 } , for example), so I believe it's the closure definition, not its type that should choose an argument to be scoped on. Especially that the initial choice of the argument to be scoped over may misalign with the user's wishes to rather make the function scoped over another argument of their choice (because it's not really possible to know for library authors what argument will be accessed the most and require the most syntactic sugar).

I understand, but it's not the intent of the pitch. There is no misalignment between the user's wishes and the api author, by definition. If there is one, then the API is badly designed.

I can try to give another definition of the purpose of scoped functions.

Currently, the leading dot syntax is driven by types: setFont(.body) works because the argument type has a body static member.

The implicit and obvious context that allows the .body snippet to be understood is provided by the signature of the function, via the type of its argument.

I propose that the power of function signatures is extended, by giving them another way to provide implicit and obvious context. This is scoped functions. They make the leading dot syntax possible in more situations. Including in the body of argument closures. Including in "autoclosured-values".

The "true master" of context is the function signature, not only types of arguments.

1 Like

What did you mean?

Thanks, @Nobody1707. I've updated the pitch with your example. I switched to @scoped instead of @implicit in order to be consistent with the title of the pitch, but this is not something very important at this stage.

This sounds like the same concept as Pascal's (or is it a Borland extension?) with statement.

With someObject do
  ...
end;

The compiler would prioritize lookups to the type of someObject, and any lookups of property or method references would check the type first, before moving on to the regular lookup rules.

This is more flexible than tying the syntax, and thus functionality, to a function declaration.

2 Likes

I was arguing that a feature affecting more or less the syntax only doesn't need to be reflected in the type system (that is, it could simply desugar to some more common representation), but then it occurred to me that functions with variadic parameters exist — and Int... in the parameter list is not interchangeable with [Int]:

func variadic(params: Int...) { }

let f: (Int...) -> () = variadic(params:) // OK
let badf: ([Int]) -> () = variadic(params:) // Error: Cannot convert value of type '(Int...) -> ()' to specified type '([Int]) -> ()'

// The opposite would apply to this function...
func array(params: [Int]) { }

— but maybe there's an ABI-level reason that I'm not aware of why Int... is distinct from [Int] more than on just syntactic level.

This is exact. But it goes beyond the with statement. Besides, the implicit context is chosen by the enclosing function (here the with function), not by the user.

It happens that the enclosing function injects a user value as the context. This is the case of with:

func with<T>(_ value: T, run: @scoped (T) -> Void) { run(value) }

with("Hello") {
     print(.count)        // prints "5"
     print(.lowercased()) // prints "hello"
}

But the enclosing function can choose to provide another context: see the database requests examples in the pitch.

Do you think this could impact this pitch?

I was thinking if actually helps the pitch by pointing at some precedent.


I now better understand your intent, but I still think that overall the solution as-proposed won't be equally robust in general case as compared to your examples.

What helps your argument in GRDB examples is that you're both the vendor of the API that asks for a closure and also the vendor of the types that will be passed into this closure — you have a reason to be very sure that the parameter you mark with @scoped one way or another will very be likely the one the user would want to acceess the most.

This won't hold in the general case though. Imagine I'm vending a generic function over T (point being that I don't know anything about T) that wants an argument of type (T, T) -> (), that is

func doSomethingGeneric<T>(_ closure: (T, T) -> ()) {
    // code...
    closure(val1, val2)
    // some more code...
}

— if I make the closure parameter scoped over, say, the first parameter, then it's really only 50/50 chance that I correctly guess that users would want to access the first argument more often than the second one and thus would like to have syntactic conveniences with that particular argument. This is different from the contextual type: the latter is unambiguously singular, there's no tradeoff being made when allowing users to make use of it. You can imagine, however, the problem becoming more and more severe as the number of parameters grows. So the only case where it could work is with function parameters that have one and only one parameter, as the contextual type would be singular in that case too, except in the return statement: what happens with @scoped (T) -> V?

wantsScoped() {
    return .value // is .value a member of `T` or a member of `V`? 
}

Moreover, the solution as-proposed is unnecessarily restrictive: it makes it impossible to "just take" a closure from somewhere else and "just pass" it into the function that wants a scoped function parameter; the latter would need to be explicitly definied at the call site — which would prevent me from, for example, doing this:

func passesFurther(_ f: (T) -> ()) {
    // code...
    wantsScoped(f) // error?
}

Autoclosures is an even stranger case to me: first and foremost, they are defined to be an attribute that is applied to "a parameter whose type is a function type that takes no arguments", so the autoclosure semantics would need to be extended at least. More important perhaps is that semantically, an autoclosure is an instrument to implement delayed evaluation without affecting the syntax of the call site — this is different (in my understanding) from the GRDB use case you want to enable, since I believe you're not really trying to delay the evaluation of a predicate, just enhance its syntax. I'm not saying here that it's not workable, it's just that to me you are really proposing two different features, and this one shouldn't rely on autoclosures.


Going back to my earlier argument: I believe that it's the user who should decide how they want to write the closure definition, not the API that takes the closure. To be more concrete, here's how I would rewrite one of your examples if I were concerned with the ergonomics of accessing the members of a UILabel:

// For the sake of my argument, I'll make `update` accept more parameters

enum ColorPalette: Toolbox { ... } // suppose we have this type in my module

// suppose this comes from another module
func build<T: Toolbox, U>(_ toolbox: T, _ initial: U, update: (T, U) -> ()) -> T {
    var value = initial
    update(toolbox, value)
    return value
}

// this way, I can myself decide if I want easier access to the members of `palette` or `label`
let label = build(myPalette, UILabel()) { palette, @scoped label in
    .text = "Hello scoped functions!"
    .textColor = palette.red
    .font = .preferredFont(forTextStyle: .body)
}
1 Like

Seems to me like something similar to implicit context? Looks more like a mini function builder (result builder)? Maybe even a Lens?

Not sure if this could be aligned to Task locals but for functions: swift-evolution/nnnn-task-locals.md at 6605ec8e5b7584397b964f9449f3a51c2d7147a5 · apple/swift-evolution · GitHub

looks cool. I don’t know if dot syntax is the way to go though. PowerShell uses $_ for this.

PowerShell using it seems like a good reason for Swift not to. :wink:

Overall, I agree with Gwendal's reasoning here: leading dot syntax for member access is well precedented in Swift. This is the first version of the feature which feels natural to me. My only concern is the spelling or shape of the feature's activation. Attributes are okay, but Swift is gaining so many I have to wonder if it's tenable to keep adding more.

1 Like

To be honest, I think that existing implicit member lookup rules, function builders, key paths, and some clever library design get us most of the way there in many cases, without introducing a whole new feature. For the HTML example we can get very close to the syntax you want, with the caveat that we can't really use implicit member syntax here anyway due to the ambiguity with chained member references from the previous line (ETA: you might be able to do something clever here by taking advantage of SE-0287 to define an instance level head function...):

@_functionBuilder
struct HTMLBuilder<ParentContext: HTMLContext> {
    static func buildExpression(_ context: ParentContext) -> HTML { context.html }
    static func buildExpression(_ html: HTML) -> HTML { html }
    static func buildBlock(_ htmls: HTML...) -> HTML {
        HTML(raw: htmls.reduce("", { $0 + $1.raw }))
    }
}

protocol HTMLContext {
    var html: HTML { get }
}

extension Never: HTMLContext {
    var html: HTML { fatalError() }
}

struct HTML {
    struct Root: HTMLContext {
        static func head(@HTMLBuilder<Head> _ builder: () -> HTML) -> Root {
            Root(html: HTML(raw: "<head>\(builder().raw)</head>"))
        }
        static func body(@HTMLBuilder<Never> _ builder: () -> HTML) -> Root {
            Root(html: HTML(raw: "<body>\(builder().raw)</body>"))
        }

        var html: HTML
    }
    struct Head: HTMLContext {
        static func title(_ str: String) -> Head {
            return Head(html: HTML(raw: "<title>\(str)</title>"))
        }

        var html: HTML
    }

    var raw: String

    init(@HTMLBuilder<Root> _ builder: () -> HTML) {
        self = builder()
    }

    init(raw: String) {
        self.raw = raw
    }
}

let html = HTML {
    .head { .title("Hello scoped functions!") }; // OK
    .body { .title("Hello from body") } // Error
}

The instance builder example I was able to accomplish using key paths and a custom operator:

import UIKit

func <=<T, U>(lhs: WritableKeyPath<T, U>, rhs: U) -> Assignment<T, U> {
     Assignment(keyPath: lhs, value: rhs)
}

struct Assignment<T, U> {
    var keyPath: WritableKeyPath<T, U>
    var value: U

    func apply(to instance: inout T) {
        instance[keyPath: keyPath] = value
    }
}

struct AnyAssignment<T> {
    private var _apply: (inout T) -> Void
    func apply(to instance: inout T) {
        self._apply(&instance)
    }

    init<U>(_ assignment: Assignment<T, U>) {
        self._apply = { assignment.apply(to: &$0) }
    }
}

@_functionBuilder
struct InstanceBuilder<T> {
    static func buildExpression<U>(_ assignment: Assignment<T, U>) -> AnyAssignment<T> {
        return AnyAssignment(assignment)
    }
    static func buildBlock(_ assignments: AnyAssignment<T>...) -> [AnyAssignment<T>] {
        return assignments
    }
}

func build<T>(_ instance: T, @InstanceBuilder<T> buildFunction: () -> [AnyAssignment<T>]) -> T {
    var instance = instance
    let assignments = buildFunction()
    assignments.forEach { $0.apply(to: &instance) }

    return instance
}

let label = build(UILabel()) {
    \.text <= "Hello world"
    \.font <= .systemFont(ofSize: 12.0)
}

Function builders are really powerful here, and with some effort on the part of the library there's a lot of magic you can enable.

To address the proposal directly, if this direction were pursued I would probably prefer a different sigil than the leading dot. Right now, deducing the meaning of an implicit member expression is highly local—the compiler (and the programmer) must be able to infer the base type from other context in the expression. Overloading the feature in this manner would mean that you would not be able to tell if the inference was happening because of a local type inference, or if the type was coming from the scoped function.

9 Likes

I agree. It is the responsibility of the API vendor to carefully choose the scope. We have a simple example with the with(value) { ... } function. This one is very local, very easy to understand.

The GRDB example is interesting, because it exemplifies how the scope can be different than a direct function argument. But it is still very carefully chosen:

In the code below, the request returned by Player.all() is generic on Player. Its filter(_:) method has an autoclosure argument which is scoped on Player.Type. This allows the user to define the Player static members that are granted with leading dot syntax:

// Library code (simplified)
protocol Table {
    static func all() -> Request<Self>
}
struct Request<T: Table> {
    func filter(_ expression: @autoclosure @scoped (T.Type) -> Expression) { ... }
}

// User code
struct Player: Table {
    static let name = Column("name")
}

let request = Player.all().filter(.name == "foo")
//            ^...................~~~~~

See how it happens that the API vendor knows where the user is most likely to define the identifiers that deserve the leading-dot syntax. In the case of GRDB, it is natural that a type defines its own columns as static members. The documentation will guide the user to this natural sweet spot.

Some users willl prefer to define columns in some other places, and it's 100% ok. But they may end up preferring static properties for more streamlined code:

struct Player: Table {
    enum Column: String { case name }
}

let request = Player.all().filter(.Column.name == "foo")

This won't hold in the general case though. Imagine I'm vending a generic function over T (point being that I don't know anything about T ) that wants an argument of type (T, T) -> () [...] if I make the closure parameter scoped over, say, the first parameter, then it's really only 50/50 chance that I correctly guess that users would want to access the first argument more often than the second one and thus would like to have syntactic conveniences with that particular argument.

Then let the user scope on his preferred one?

func doSomethingGeneric<T>(_ closure: (T, T) -> ()) { ... }

doSomethingGeneric { a, b in
    with(a) {        // if the user wants to scope, she scopes
        print(.foo)  // a.foo
        print(b.foo) // b.foo
    }
}

Autoclosures [...] are defined to be an attribute that is applied to "a parameter whose type is a function type that takes no arguments ", so the autoclosure semantics would need to be extended at least.

This is exact. GRDB needs scoped autoclosures. I'm aware those won't be easy to deal with (both in the compiler, and with language lawyers).

Here I have to count on the will to improve the language. Scoped auto closures are a boon that would help GRDB, but not only. They could have solved the SE-0299 Problem:

// In SwiftUI
extension Toggle {
    func toggleStyle(_ style: @autoclosure @scoped (ToggleStyles.Type) -> ToggleStyle)
}
enum ToggleStyles {
    static let switch: ToggleStyle = SwitchToggleStyle()
    ...
}

// In applications
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle(SwitchToggleStyle()) // Still OK
  .toggleStyle(.switch)             // New!

I will look for more examples.

Going back to my earlier argument: I believe that it's the user who should decide how they want to write the closure definition, not the API that takes the closure. To be more concrete, here's how I would rewrite one of your examples [...]

I think this pitch, with the help of the with function, addresses your need.

By the way, your messages are very helpful, thank you!

1 Like

I admit I'd be happy if we could remove as much clutter as possible.

I do not suggest that removing everything is on the table. It looks too much like the redefinition of self:

with(label) {
    text = "Hello" // just no
}

The leading dot is well known:

with(label) {
    .text = "Hello" // really nice
}

The current state of swift is:

with(label) {
    $0.text = "Hello" // possible today
}

So $_ would not bring much benefit.

And I don't want GRDB to look like this:

// Meh
Player.filter($_.score >= 1000)

// The statu quo is better:
Player.filter(Player.score >= 1000)

// Scoped functions are like a dream:
Player.filter(.score >= 1000)

I wholeheartedly concur. Scoped function are only ok when both the API vendor and the developer agree on an implicit and obvious context. When this condition is not met, scoped functions are not the correct tool. But when this agreement is established, the leading dot is just natural.

Thank you!

My only concern is the spelling or shape of the feature's activation. Attributes are okay, but Swift is gaining so many I have to wonder if it's tenable to keep adding more.

The initial pitch was using an unusual syntax, inspired by Kotlin:

let f: T.() -> Void

The . between the scope type and the argument list was kind of a reminder that a T would come at the source of . inside the body.

But it looks really... "new", and when I was suggested to use an attribute, I immediately switched because there are much more important qualities in this pitch that I want to promote. This gives:

// Back to basics
let f: @scoped (T) -> Void

It has exactly the same meaning, but looks less strange.

I find it difficult, from my position, to be bold enough and address your concern about the proliferation of attributes with a radical syntax. I count on this pitch, or others, to refine such details.

Terms of Service

Privacy Policy

Cookie Policy