Pitch: Scoped Functions

Evolution Pitches
1

Pitch: Scoped Functions

Scoped Functions are functions that enhance the leading dot syntax inside their body, so that it gives short-hand access to the members of a specific value. Such functions exist so that API designers and their consumers can collaborate towards a way to write focused code where an agreed implicit and obvious context does not have to be spelled out.

For the posterity, the initial pitch is below.

Initial pitch

Introduction

Scoped Functions are functions that enhance the leading dot syntax inside their body, so that it gives short-hand access to the members of a specific value. Such functions exist so that API designers and their consumers can collaborate towards a way to write focused code where an agreed implicit and obvious context does not have to be spelled out.

Motivation

This pitch considers that developers often want to use short-hand identifier resolution in particular contexts, and api designers often want to make it possible.

The Swift language already provides much help in this regard. We have the implicit self, the current (and evolving) rules for the leading dot syntax, the scoping rules for accessing nested types, etc.

But developers always want more:

In all those examples, developers express a desire to enhance identifier resolution in a lexically scoped context: a value, or the body of a closure, where both are arguments of a function.

Considering:

  • The braces of a closure body are a well delimited lexical scope.
  • A function call is a strong semantic signal that is able to define an implicit and obvious context that both api designers and consumers can agree on.
  • Naked identifier resolution in Swift is already pretty busy with local variables, self, type names, module names, etc, so we do not propose modifying this.
  • The leading dot syntax is a beloved short-hand syntax of Swift.

We modify the rules that govern leading dot syntax inside the body of a particular closure argument. When the closure is an autoclosure, the leading dot syntax is modified for the value that feeds this autoclosure.

This gives:

// The build(_:configure:) function accepts a closure 
// scoped on its first argument:
let label = build(UILabel()) {
    .text = "Hello scoped functions!"
    .font = .preferredFont(forTextStyle: .body)
}

// The `filter(_:)` and `order(_:)` functions accept an 
// autoclosure scoped on the generic type of the request
// (here, `Player`).
// `Player` collaborates, and has defined two
// `Player.score` and `Player.team` static constants.
let players = try Player.all()
    .filter(.team == "Red")
    .order(.score.desc)
    .limit(10)
    .fetchAll(from: database)

// A mix and match between result builders and
// scoped functions. Each closure is scoped on an
// object that defines the relevant leading-dot apis 
// inside the body.
let html = HTML {
    .head { .title("Hello scoped functions!") }
    .body { ... }
}

// A revisit of SE-0299: the argument of toggleStyle(_:) is
// an autoclosure scoped on an ad-hoc enum type:
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle(.switch)

Without scoped functions, developers face various problems:

// The `label` variable is repeated.
let label = UILabel()
label.text = "Hello World!"
label.font = .preferredFont(forTextStyle: .body)

// The `Player` type is repeated.
let players = try Player.all()
    .filter(Player.team == "Red")
    .order(Player.score.desc)
    .limit(10)
    .fetchAll(from: database)

// Typesquatting: the `Head`, `Title`, `Body`
// identifiers can't be used for other purposes.
let html = HTML {
    Head { Title("Hello World!") }
    Body { ... }
}

// As before SE-0299
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle(SwitchToggleStyle())

Personal motivation: as the author of GRDB, which defines a type-safe API for building SQL queries, and relies of static properties defined on user-defined "record types", I'd love to simplify some user-land code:

// CURRENT GRDB CODE
struct Team: Codable, FetchableRecord {
    static let players = hasMany(Player.self)
    static let awards = hasMany(Award.self, through: players, using: Player.awards)
}
struct Player: Codable, FetchableRecord {
    static let awards = hasMany(Award.self)
}
struct Award: Codable, FetchableRecord {
}

// All red teams with their awarded players
let request = Team
  .filter(Team.Columns.name == "Red")
  .including(all: Team.players.having(Player.awards.count > 0))

// WITH SCOPED FUNCTIONS
struct Team: Codable, FetchableRecord {
    static let players = hasMany(Player.self)
    static let awards = hasMany(Award.self, through: players, using: .awards) // <-
}
struct Player: Codable, FetchableRecord {
    static let awards = hasMany(Award.self)
}
struct Award: Codable, FetchableRecord {
}

// All red teams with their awarded players
let request = Team
  .filter(.name == "Red")                             // <-
  .including(all: .players.having(.awards.count > 0)) // <-

Sorry if I provided a contrieved example. But contrieved examples are precisely those where unnecessary clutter obscures the intent the most.

Proposed solution

The syntax for defining the type of a function is extended. When the arguments list is prepended with a type T and a dot, as in T.(...) -> ..., then this is the type of a scoped function, and T is the scope type. Such a scoped function accepts an extra first argument, of type T, which is the scope value. Inside the body of the function, the scope value feeds the leading dot syntax. The function is said to be T-scoped.

For example:

let f: String.() -> Int // A
f = { return .count }   // B
f("Hello")              // C: prints "5"

  • In (A), f is defined as a String-scoped function.

  • In (B), the compiler is able to resolve .count as String.count.

  • In (C), the function is called with a particular String.

There is no syntax for defining a scoped function from scratch, as in func f T.(...) -> ... { ... }. Scoped functions are designed to be used as closure arguments, and not free functions or methods. The functions that accept scoped functions as arguments define the "agreed, implicit, and obvious context" that is key for this pitch.

We are able to address some of the above examples:

  1. The build(_:_:) function

    func build<T>(_ value: T, with transform: T.() throws -> ()) rethrows -> T {
    return transform(value)
}

let label = build(UILabel()) {
    .text = "Hello scoped functions!"
    .font = .preferredFont(forTextStyle: .body)
}
label.text // "Hello"

    Woot, we'll have to define how the compiler deals with leading dot syntax for .font, .preferredFont, and .body :-)

  2. The database query

    struct Request<T> {
    func filter(_ expression: @autoclosure T.Type.() -> Expression) -> Self { ... }
    func order(_ ordering: @autoclosure T.Type.() -> Ordering) -> Self { ... }
    func limit(_ limit: Int) -> Self { ... }
}

extension Player {
    static let team = Column(...)
    static let score = Column(...)
}

let request: Request<Player> = ...
request.filter(.team == "Red").order(.score.desc)

    The @autoclosure qualifier has the compiler change the leading dot scoping for values.

Detailed design

TBD

Source compatibility

Scoped function are an additive feature that creates no source compatibility issue.

Effect on ABI stability

A scoped function, at runtime, is a plain function that accepts its scope as the first argument. Its a compiler-only feature, without any effect on the ABI.

Effect on API resilience

TBD

Alternatives considered

Use another sigil than . ? For example, ':

// The build(_:configure:) function accepts a closure 
// scoped on its first argument:
let label = build(UILabel()) {
    'text = "Hello scoped functions!"
    'font = .preferredFont(forTextStyle: .body)
}

// The `filter(_:)` and `order(_:)` functions accept an 
// autoclosure scoped on its receiver type.
// `Player` collaborates, and has defined two
// `Player.score` and `Player.team` static constants.
let players = try Player
    .filter('team == "Red")
    .order('score.desc)
    .limit(10)
    .fetchAll(from: database)

// A mix and match between result builders and
// scoped functions. Each closure is scoped on an
// object that defines the relevant leading-dot apis 
// inside the body.
let html = HTML {
    'head { 'title("Hello scoped functions!") }
    'body { ... }
}

// A revisit of SE-0299: the argument of toggleStyle(_:) is
// an autoclosure scoped on an ad-hoc enum type:
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle('switch)

The latest version of the pitch is at Pitch: Scoped Functions · GitHub

14 Likes
3

What happens if .head { ... } has a body property?

3 Likes
4

I guess GitHub - devxoul/Then: ✨ Super sweet syntactic sugar for Swift initializers is another example that scoping is something people want

1 Like
5

I think @implicit is a better name for it, albeit longer, but I think it would be most useful if you could apply it on the declaration of the closure parameter in addition to it's definition.

For example, I think this should work:

func with<T>(_ initial: T, update: (@implicit inout T) -> Void) -> T {
    var value = initial
    update(&value)
    return value
}

let answer = with("42") {
    .append("!")
}

While also allowing your forEach example.

1 Like
6

You are correct. A scoped function, that I wrote T.(...) -> ... could also be spelled @scoped (T, ...) -> ..., or even (@scoped T, ...) -> ....

Arguably @scoped (T, ...) -> ... can be considered a superior alternative, since it follows existing precedent (@escaping, etc.)

I'll use this syntax from now on, and update the pitch.

The examples that you list are also particular to the "longer" kind of closures (i.e. those that are spelled more than the simple { $0 + 1 } , for example), so I believe it's the closure definition, not its type that should choose an argument to be scoped on. Especially that the initial choice of the argument to be scoped over may misalign with the user's wishes to rather make the function scoped over another argument of their choice (because it's not really possible to know for library authors what argument will be accessed the most and require the most syntactic sugar).

I understand, but it's not the intent of the pitch. There is no misalignment between the user's wishes and the api author, by definition. If there is one, then the API is badly designed.

I can try to give another definition of the purpose of scoped functions.

Currently, the leading dot syntax is driven by types: setFont(.body) works because the argument type has a body static member.

The implicit and obvious context that allows the .body snippet to be understood is provided by the signature of the function, via the type of its argument.

I propose that the power of function signatures is extended, by giving them another way to provide implicit and obvious context. This is scoped functions. They make the leading dot syntax possible in more situations. Including in the body of argument closures. Including in "autoclosured-values".

The "true master" of context is the function signature, not only types of arguments.

1 Like
7

What did you mean?

8

Thanks, @Nobody1707. I've updated the pitch with your example. I switched to @scoped instead of @implicit in order to be consistent with the title of the pitch, but this is not something very important at this stage.

9

This sounds like the same concept as Pascal's (or is it a Borland extension?) with statement.

With someObject do
  ...
end;

The compiler would prioritize lookups to the type of someObject, and any lookups of property or method references would check the type first, before moving on to the regular lookup rules.

This is more flexible than tying the syntax, and thus functionality, to a function declaration.

2 Likes
11

This is exact. But it goes beyond the with statement. Besides, the implicit context is chosen by the enclosing function (here the with function), not by the user.

It happens that the enclosing function injects a user value as the context. This is the case of with:

func with<T>(_ value: T, run: @scoped (T) -> Void) { run(value) }

with("Hello") {
     print(.count)        // prints "5"
     print(.lowercased()) // prints "hello"
}

But the enclosing function can choose to provide another context: see the database requests examples in the pitch.

12

Do you think this could impact this pitch?

14

Seems to me like something similar to implicit context? Looks more like a mini function builder (result builder)? Maybe even a Lens?

Not sure if this could be aligned to Task locals but for functions: swift-evolution/nnnn-task-locals.md at 6605ec8e5b7584397b964f9449f3a51c2d7147a5 · apple/swift-evolution · GitHub

looks cool. I don’t know if dot syntax is the way to go though. PowerShell uses $_ for this.

15

PowerShell using it seems like a good reason for Swift not to. :wink:

Overall, I agree with Gwendal's reasoning here: leading dot syntax for member access is well precedented in Swift. This is the first version of the feature which feels natural to me. My only concern is the spelling or shape of the feature's activation. Attributes are okay, but Swift is gaining so many I have to wonder if it's tenable to keep adding more.

1 Like
16

To be honest, I think that existing implicit member lookup rules, function builders, key paths, and some clever library design get us most of the way there in many cases, without introducing a whole new feature. For the HTML example we can get very close to the syntax you want, with the caveat that we can't really use implicit member syntax here anyway due to the ambiguity with chained member references from the previous line (ETA: you might be able to do something clever here by taking advantage of SE-0287 to define an instance level head function...):

@_functionBuilder
struct HTMLBuilder<ParentContext: HTMLContext> {
    static func buildExpression(_ context: ParentContext) -> HTML { context.html }
    static func buildExpression(_ html: HTML) -> HTML { html }
    static func buildBlock(_ htmls: HTML...) -> HTML {
        HTML(raw: htmls.reduce("", { $0 + $1.raw }))
    }
}

protocol HTMLContext {
    var html: HTML { get }
}

extension Never: HTMLContext {
    var html: HTML { fatalError() }
}

struct HTML {
    struct Root: HTMLContext {
        static func head(@HTMLBuilder<Head> _ builder: () -> HTML) -> Root {
            Root(html: HTML(raw: "<head>\(builder().raw)</head>"))
        }
        static func body(@HTMLBuilder<Never> _ builder: () -> HTML) -> Root {
            Root(html: HTML(raw: "<body>\(builder().raw)</body>"))
        }

        var html: HTML
    }
    struct Head: HTMLContext {
        static func title(_ str: String) -> Head {
            return Head(html: HTML(raw: "<title>\(str)</title>"))
        }

        var html: HTML
    }

    var raw: String

    init(@HTMLBuilder<Root> _ builder: () -> HTML) {
        self = builder()
    }

    init(raw: String) {
        self.raw = raw
    }
}

let html = HTML {
    .head { .title("Hello scoped functions!") }; // OK
    .body { .title("Hello from body") } // Error
}

The instance builder example I was able to accomplish using key paths and a custom operator:

import UIKit

func <=<T, U>(lhs: WritableKeyPath<T, U>, rhs: U) -> Assignment<T, U> {
     Assignment(keyPath: lhs, value: rhs)
}

struct Assignment<T, U> {
    var keyPath: WritableKeyPath<T, U>
    var value: U

    func apply(to instance: inout T) {
        instance[keyPath: keyPath] = value
    }
}

struct AnyAssignment<T> {
    private var _apply: (inout T) -> Void
    func apply(to instance: inout T) {
        self._apply(&instance)
    }

    init<U>(_ assignment: Assignment<T, U>) {
        self._apply = { assignment.apply(to: &$0) }
    }
}

@_functionBuilder
struct InstanceBuilder<T> {
    static func buildExpression<U>(_ assignment: Assignment<T, U>) -> AnyAssignment<T> {
        return AnyAssignment(assignment)
    }
    static func buildBlock(_ assignments: AnyAssignment<T>...) -> [AnyAssignment<T>] {
        return assignments
    }
}

func build<T>(_ instance: T, @InstanceBuilder<T> buildFunction: () -> [AnyAssignment<T>]) -> T {
    var instance = instance
    let assignments = buildFunction()
    assignments.forEach { $0.apply(to: &instance) }

    return instance
}

let label = build(UILabel()) {
    \.text <= "Hello world"
    \.font <= .systemFont(ofSize: 12.0)
}

Function builders are really powerful here, and with some effort on the part of the library there's a lot of magic you can enable.

To address the proposal directly, if this direction were pursued I would probably prefer a different sigil than the leading dot. Right now, deducing the meaning of an implicit member expression is highly local—the compiler (and the programmer) must be able to infer the base type from other context in the expression. Overloading the feature in this manner would mean that you would not be able to tell if the inference was happening because of a local type inference, or if the type was coming from the scoped function.

9 Likes
17

I agree. It is the responsibility of the API vendor to carefully choose the scope. We have a simple example with the with(value) { ... } function. This one is very local, very easy to understand.

The GRDB example is interesting, because it exemplifies how the scope can be different than a direct function argument. But it is still very carefully chosen:

In the code below, the request returned by Player.all() is generic on Player. Its filter(_:) method has an autoclosure argument which is scoped on Player.Type. This allows the user to define the Player static members that are granted with leading dot syntax:

// Library code (simplified)
protocol Table {
    static func all() -> Request<Self>
}
struct Request<T: Table> {
    func filter(_ expression: @autoclosure @scoped (T.Type) -> Expression) { ... }
}

// User code
struct Player: Table {
    static let name = Column("name")
}

let request = Player.all().filter(.name == "foo")
//            ^...................~~~~~

See how it happens that the API vendor knows where the user is most likely to define the identifiers that deserve the leading-dot syntax. In the case of GRDB, it is natural that a type defines its own columns as static members. The documentation will guide the user to this natural sweet spot.

Some users willl prefer to define columns in some other places, and it's 100% ok. But they may end up preferring static properties for more streamlined code:

struct Player: Table {
    enum Column: String { case name }
}

let request = Player.all().filter(.Column.name == "foo")

This won't hold in the general case though. Imagine I'm vending a generic function over T (point being that I don't know anything about T ) that wants an argument of type (T, T) -> () [...] if I make the closure parameter scoped over, say, the first parameter, then it's really only 50/50 chance that I correctly guess that users would want to access the first argument more often than the second one and thus would like to have syntactic conveniences with that particular argument.

Then let the user scope on his preferred one?

func doSomethingGeneric<T>(_ closure: (T, T) -> ()) { ... }

doSomethingGeneric { a, b in
    with(a) {        // if the user wants to scope, she scopes
        print(.foo)  // a.foo
        print(b.foo) // b.foo
    }
}

Autoclosures [...] are defined to be an attribute that is applied to "a parameter whose type is a function type that takes no arguments ", so the autoclosure semantics would need to be extended at least.

This is exact. GRDB needs scoped autoclosures. I'm aware those won't be easy to deal with (both in the compiler, and with language lawyers).

Here I have to count on the will to improve the language. Scoped auto closures are a boon that would help GRDB, but not only. They could have solved the SE-0299 Problem:

// In SwiftUI
extension Toggle {
    func toggleStyle(_ style: @autoclosure @scoped (ToggleStyles.Type) -> ToggleStyle)
}
enum ToggleStyles {
    static let switch: ToggleStyle = SwitchToggleStyle()
    ...
}

// In applications
Toggle("Wi-Fi", isOn: $isWiFiEnabled)
  .toggleStyle(SwitchToggleStyle()) // Still OK
  .toggleStyle(.switch)             // New!

I will look for more examples.

Going back to my earlier argument: I believe that it's the user who should decide how they want to write the closure definition, not the API that takes the closure. To be more concrete, here's how I would rewrite one of your examples [...]

I think this pitch, with the help of the with function, addresses your need.

By the way, your messages are very helpful, thank you!

1 Like
18

I admit I'd be happy if we could remove as much clutter as possible.

I do not suggest that removing everything is on the table. It looks too much like the redefinition of self:

with(label) {
    text = "Hello" // just no
}

The leading dot is well known:

with(label) {
    .text = "Hello" // really nice
}

The current state of swift is:

with(label) {
    $0.text = "Hello" // possible today
}

So $_ would not bring much benefit.

And I don't want GRDB to look like this:

// Meh
Player.filter($_.score >= 1000)

// The statu quo is better:
Player.filter(Player.score >= 1000)

// Scoped functions are like a dream:
Player.filter(.score >= 1000)
19

I wholeheartedly concur. Scoped function are only ok when both the API vendor and the developer agree on an implicit and obvious context. When this condition is not met, scoped functions are not the correct tool. But when this agreement is established, the leading dot is just natural.

20

Thank you!

My only concern is the spelling or shape of the feature's activation. Attributes are okay, but Swift is gaining so many I have to wonder if it's tenable to keep adding more.

The initial pitch was using an unusual syntax, inspired by Kotlin:

let f: T.() -> Void

The . between the scope type and the argument list was kind of a reminder that a T would come at the source of . inside the body.

But it looks really... "new", and when I was suggested to use an attribute, I immediately switched because there are much more important qualities in this pitch that I want to promote. This gives:

// Back to basics
let f: @scoped (T) -> Void

It has exactly the same meaning, but looks less strange.

I find it difficult, from my position, to be bold enough and address your concern about the proliferation of attributes with a radical syntax. I count on this pitch, or others, to refine such details.

21

Reminded me of Kotlin's apply :

and I really like the idea overall.

2 Likes
22

While I'm not fond of the dot syntax, I work enough with Kotlin to know the limit and risk of redefining this/self to omit the dot. It shadows outer self properties with the scoped object properties, and can be confusing. (and also makes it harder to reference shadowed outer self properties).

So, I think this is a right balance.

1 Like
23

Oh no, now I want this :pleading_face:

1 Like