Pitch: Scoped Functions

Evolution Pitches
24

I thought this might be worth to inform that some of virtual features used with higher order functions in the pitch are already (kind of) possible using KeyPath, though you have to make methods manually.

func ==<T, V: Equatable>(lhs: KeyPath<T, V>, rhs: V) -> (T) -> Bool {
    return { $0[keyPath: lhs] == rhs }
}

Makes below possible:

let fullLengthArticles = articles.filter(\.category == .fullLength)

Explained here:

7 Likes
25

Awesome. Yeah, this is exactly the kind of thing I was talking about with my claim that a lot of these problems are already largely solvable with some clever library design and existing features.

2 Likes
26

As already stated in the future directions alternatives considered of SE-0289

leading dot syntax isn't suitable. In the specific case of the HTML builder example, today you can do something like the following (even without result builders if you don't need control flow statements)

extension HTML {
  struct Root {
    func head(content: () -> Head) -> Root { ... }
    func body(content: () -> Body) -> Root { ... }

    static func head(content: () -> Head) -> Root {
      Root().head(content)
    }
    static func body(content: () -> Body) -> Root {
      Root().body(content)
    }
  }
}

let html = HTML {
  .head { ... }  // uses static func head
  .body { ... }  // uses func body
}

You basically provide all methods as static functions too in order to create the first statement. But then formatters would indent one level deeper any subsequent leading dot statement and with no context there would be no way for them to prevent that. So you will always end up with this whenever you press enter or re-format your code:

let html = HTML {
  .head { ... }
    .body { ... }
}
3 Likes
27

I agree with the above posters that a lot of the proposed use cases for scoped functions can be implemented (with different spelling) using existing features. I haven’t used Kotlin enough to understand the tradeoffs of scoped functions there, but scoped functions don’t seem to offer much advantage over explicitly passing a parameter to the closure and accessing members on that.

To modify @gwendal.roue’s earlier example:

// not @scoped
func with<T>(_ value: T, run: (T) -> Void) { run(value) }

with("Hello") {
    // call members on `$0`, or give the parameter a name
    print($0.count)        // prints "5"
    print($0.lowercased()) // prints "hello"
}

$0.foo is arguably not as pretty as .foo or foo, but:

  • it uses existing Swift features (nothing new to learn)
  • there’s no confusion about where foo is declared (is it local? global? self? scoped?)
  • you don’t have to look at the signature of with to know its closure is @scoped (so it’s easier for future readers to understand)
6 Likes
28

This is an absolute genius use of a Function Builder. Initially, I was really supportive of this pitch, but maybe we actually don't need it that much...

I have played around a bit with your code and it's possible to build quite a good library with that. If someone is interested, I have a gist with a bigger example over here. Its main limitations are now that we have to use those ugly semicolons to separate the function calls (we would also have to deal with that issue, in case we accepted this pitch), that Xcode isn't that helpful in suggesting the functions inside of the function builder closures and also that we don't have (as the generics manifesto calls them) Generic value parameters because I really don't like my Indent0 through Indent5 types, but that is another topic...

2 Likes
29

See @xAlien95’s approach above that avoids the semicolon issue, at the cost of some less-than-ideal auto-indent behavior in Xcode

30

Oh yeah, I completely overlooked that. Unfortunately it doesn't work with my string literals but otherwise such a library could work very well.

31

Agreed. The HTML example was added because it is "trendy", but it does not hold much water. I'll remove it from the pitch.

1 Like
32

Yes, key paths are very powerful. But:

  • Both $0 and key paths create visual noise that this pitch aims at removing:

    let label = build(UILabel()) {
    \.text <= "Hello world"
    \.font <= .systemFont(ofSize: 12.0)
}

let label = build(UILabel()) {
    $0.text = "Hello world"
    $0.font = .systemFont(ofSize: 12.0)
}

// Please
let label = build(UILabel()) {
    .text = "Hello world"
    .font = .systemFont(ofSize: 12.0)
}

  • It is not always correct that key paths are a suitable tool.

    In the database query example, they look quite OK at first sight:

    // Vapor's Fluent
Galaxy.query(on: conn).filter(\.name == "Milky Way")

    The problem with key paths is that when one wants to be able to write a query based on some column, the model type must expose this column as a property. This prevents model type from hiding database implementation details. Some models would prefer to only expose a coordinate: CLLocationCoordinate2D property and keep their latitude and longitude private. Some models would prefer to expose a price: Decimal property, and keep their database storage private: a priceCents integer column allows SQLite to compute exact sums, for example. Yet, queries on individual coordinates and price must be expressible.

    I argue that being able to decouple the columns that can feed the query language, from the properties, is desirable. This is how model types can become the one and only layer that interfaces the database and the rest of the app.

    That's why I prefer:

    // GRDB + Scoped functions
Galaxy.filter(.name == "Milky Way")

    For this to be possible, I'd like to tell the compiler where to look for name (here, in the static properties of Galaxy). That's one of the practical goals of the pitch.

6 Likes
33

Agreed.
Typical instance creation on the fly would be one of them:

let label: UILabel = {
    let label: UILabel = .init()
    label.text = "Hello world"
    label.font = .systemFont(ofSize: 12.0)
    return label
}()

let label: UILabel = {
    $0.text = "Hello world"
    $0.font = .systemFont(ofSize: 12.0)
    return $0
}(UILabel())

lazy var label: UILabel = makeLabel()

private func makeLabel() -> UILabel {
    let label: UILabel = .init()
    label.text = "Hello world"
    label.font = .systemFont(ofSize: 12.0)
    return label
}

And everytime I write one of them, I feel it's noisy. It's good if we could write simpler.
To be honest, I always wanted to write them much more simpler. The last one is acceptable though.

Yet, I'm not sure if proposed

build(_:_:)

would fit into Swift standard (although I understand this is just showing one possible use case).
I mean, this kind of global function feels a bit... hacky.

Also, I agree it's nice if this kind of expression is accepted at compiler level.

Galaxy.filter(.name == "Milky Way")

Despite this is accepted by compiler:

Galaxies.filter(\.isEmpty)

This is not:

Galaxies.filter(!isEmpty)
// nor
Galaxies.filter(!\.isEmpty)

which feels awkward. I understand the former is kind of just by-product and if you understand why or how it works, there is no wonder though.

(There's user-end workaround though):

prefix func !<T>(keyPath: KeyPath<T, Bool>) -> (T) -> Bool {
    return { !$0[keyPath: keyPath] }
}

enables:

Galaxies.filter(!\.isEmpty)

I think one point is if this kind of expression should be implemented at compiler level or not.

...

However, if you need to use leading dot, what would you think prefix operators would work with @scope?

1 Like
34

I thought a bit about that and then I had the idea, that we should be able to write such an API in current Swift already by using dynamic member lookup. I built this little example but unfortunately it doesn't work:

@dynamicMemberLookup
struct Comparison<T, U> where U: Comparable {
    private let lhs: KeyPath<T, U>
    private let rhs: U!
    private let op: ((U, U) -> Bool)! // Could also be an enum or something like that, which could be converted into an SQL query comparison...

    static subscript(dynamicMember keyPath: KeyPath<T, U>) -> Comparison<T, U> {
        return .init(lhs: keyPath, rhs: nil, op: nil)
    }

    static func ==(lhs: Self, rhs: U) -> Self {
        .init(lhs: lhs.lhs, rhs: rhs, op: ==)
    }

    static func >(lhs: Self, rhs: U) -> Self {
        .init(lhs: lhs.lhs, rhs: rhs, op: >)
    }

    func apply(to value: T) -> Bool {
        self.op(value[keyPath: self.lhs], self.rhs)
    }
}

struct Galaxy {
    let name: String
    let numberOfStars: Int

    static func getAll() -> [Galaxy] {
        // ...
    }

    static func filter<T>(_ isIncluded: Comparison<Galaxy, T>) -> [Galaxy] {
        let galaxies = getAll()
        return galaxies.filter(isIncluded.apply(to:))
    }
}

let stringComparison = Comparison<Galaxy, String>.name == "Milky Way" // Works
let intComparison = Comparison<Galaxy, Int>.numberOfStars == 100_000_000_000 // Works

let stringComparison2: Comparison<Galaxy, String> = .name == "Milky Way" // Doesn't work
let intComparison2: Comparison<Galaxy, Int> = .numberOfStars == 100_000_000_000 // Doesn't work


let filteredGalaxies = Galaxy.filter(Comparison.name == "Milky Way") // Works

let filteredGalaxies2 = Galaxy.filter(.name == "Milky Way") // Doesn't work

Apparently implicit member lookup is currently not working for dynamic members. I don't know if that is just a bug or an oversight, or if there really is a limitation why that cannot work, but if it could work, it would be a better way than bringing scoped functions into Swift AND introducing auto closures which take an argument (which would have to be a completely new proposal and I really doubt that it would get accepted...).
Basically I'm saying that we should look into why this implicit member lookup isn't working. @Jumhyn could you say something about that? AFAIK you worked on the last implicit member lookup proposal.

EDIT:

I checked it without dynamic member lookup... it does indeed work, if we add this extension:

extension Comparison where T == Galaxy {
    static var name: Comparison<Galaxy, String> {
        .init(lhs: \.name, rhs: nil, op: nil)
    }

    static var numberOfStars: Comparison<Galaxy, Int> {
        .init(lhs: \.numberOfStars, rhs: nil, op: nil)
    }
}

But obviously that is not the way to go. We need the implicit member lookup to work with dynamic members as well, if we want a nice API...

2 Likes
35

This proposal is centered around the syntax of how scoped function could look like, but it doesn’t go into why they should exist in the first place. I would rather take the angle of defining it first, have a design centered around the definition and syntax will naturally follow.

If we simplify closures for a moment as unnamed functions, let’s take a step back and explore what scoped functions in general mean: functions that are defined in the scope a given type. We already have that language feature, functions in any extension are basically scoped functions, where self refers to the instance of the extended type.

I expect scoped closures to behave similarly, in parity with extension functions. When you refer to a property or a function in a scoped closure, the compiler should search these on ‘self’, the type you defined your scoped closure on. (The natural syntax derived from this doesn’t even need the proposed leading dot, as self should refer to the scoped type directly)

In reality, closures are much more capable than simple functions, they capture context, they can escape, etc. So the real questions is how do you reach things out of your scope, how do you redefine self conveniently, so that it won’t become awkward and ambiguous to the reader, how do you not accidentally shadow properties on self in the scoped closure, and how do you refer to a different scope.
I believe we should focus on this question rather than the actual syntax.

(I’m really glad we’re having this conversation again, apply/with comes up every once in a while, which suggests we all think of some really powerful concept we just currently can’t express in Swift.)

4 Likes
36

Agreed. Such functions make it possible to talk about the feature, but they are not part of the pitch.

They work, without any extra methods, operator, or whatever. We're dealing with regular values, with their regular types, which happen to profit from a shortcut syntax. In the below snippet, !, ==, >=, &&, etc. are already provided by GRDB on all expressions:

Galaxy.filter(.isEmpty)
Galaxy.filter(!.isEmpty)
Galaxy.filter(.name == "Milky Way")
Galaxy.filter(.radius >= 50000 && .starCount >= 1.0e11)

// Code above is interpreted as:
Galaxy.filter(Galaxy.isEmpty)
Galaxy.filter(!Galaxy.isEmpty)
Galaxy.filter(Galaxy.name == "Milky Way")
Galaxy.filter(Galaxy.radius >= 50000 && Galaxy.starCount >= 1.0e11)

// Replacing static properties with their values:
Galaxy.filter(Column("isEmpty"))
Galaxy.filter(!Column("isEmpty"))
Galaxy.filter(Column("name") == "Milky Way")
Galaxy.filter(Column("radius") >= 50000 && Column("starCount") >= 1.0e11)

Scoped functions would allow the first group to compile out of the box, without any extra support code.

1 Like
37

You're right: the pitch still has a weak introduction and motivation. I'm happy the conversation makes it possible to refine and focus it.

I would now answer your question this way:

The pitch aims at changing the way we look at this code snippet:

func f(_ x: Int) { ... }
f(.max) // Implicit Int

Currently, we see a type, Int, be granted with a short-hand syntax. There is an implicit and obvious context that allows the reader of the code to understand what .max means.

Now, the way we understand the code is the same, whether we say:

  1. .max means Int.max because the argument of f is an Int.
  2. .max means Int.max because f accepts an Int argument.

I thus propose that we shift from (1) to (2), and see the origin of the short-hand syntax in the function, instead of the type of its argument.

It is the function that has the intent, the meaning, the documentation, and drives the context. Not the types of its arguments.

Type-based leading dot syntax is, for functions, a mean to make this context implicit at call site.

From this new point of view, it is natural to desire to free functions from the types of their arguments when they want to provide short-hand syntax. And this is what scoped functions achieve. Functions become able to freely decide the implicit context. This makes it possible to design more fluent apis.

The rest of your post seems to suggest that we redefine self inside the body of a closure. I did not follow this path, because I did not want to have to deal with all the shadowing challenges this would create (and that you foresee as well):

let text = "foo"
let label = UILabel()
with(label) {
    text = text // 😭
}

The leading dot syntax avoids this shadowing.

38

Having a shadowed self is better than having a shadowed .text, because in the former case you can always refer to the underlying scope instance explicitly if needed.

// proposed leading dot syntax
with(label1) {
  with(label2) {
    .text  // no way to refer to the label1 scope instance
           // or to the label2 scope instance
  }
}

// custom 'self' scope
with(label1) {
  with(label2) { [scope1 = self] in
    self       // refers to with(label2)'s scope
    self.text  // refers to with(label2)'s scope.text
    text       // refers to with(label2)'s scope.text
    scope1     // refers to with(label1)'s scope
  }
}
4 Likes
39

Perhaps—In many situations what you call "visual noise" I consider an important indicator that something 'special' is happening that makes the code much easier to read and understand.

Consider: in theory, Swift 1.0 could have had implicit member syntax which didn't use the leading dot at all, so that

func f(_: Int) {}

f(max)

would work as f(.max) does today, with a rule akin to:

If the compiler can infer the contextual type of an unqualified identifier, then name lookup will search for results in that type's static members before starting to look in the local scope.

IMO, this would have been a confusing way to design the feature. It would conflate the qualified and unqualified lookup systems in a way that would make it difficult for programmers to reason about how a particular name would be resolved just by looking at the code. I.e., qualified lookup is 'different enough' from unqualified lookup that we want to make sure the difference between the two is always visible in the source.

Similarly, "scoped functions" are, IMO different enough from both qualified and unqualified lookup (sort of a hybrid between the two) that they would deserve their own signifier. I also don't believe that this scoping should be allowed in autoclosure'd expressions since it would make the introduction of a new scope entirely transparent to the library user.

7 Likes
40

For inspiration: Kotlin solves the same scoping problem by redefining this (self) and they assign compiler generated scope labels that allow access to outer scopes if needed. This expressions | Kotlin

41

At first I was happy to see this, but when I follow the link I see it's under the "Alternatives Considered" section, not the "Future Directions" section. So maybe it has been ruled out? I hope not.

I think there are two separate uses for this feature. One is like Kotlin's with/apply - a quick shorthand to avoid repeating the name of an object. The second use is for more powerful DSLs. SwiftUI is an example.

I've used Kotlin a lot, and while I like the way they use this for DSLs, I think... maybe... the Swift function builders have more potential. I'd like to see them developed further, and see this scoping ability added there (to Swift function builders).

I'm a -1 on the leading dot. :) To me it's confusing that it mixes with the existing meaning.

3 Likes
42

A related thought: I would be very interested in seeing an approach like the function builders extended to expressions, so that a library could receive an expression from the caller unevaluated, in AST form.

1 Like
43

I have wanted this for so long. I have tried to wrap my mind around how it would work for quite a while. Especially since I HATE how Builder functions work by requiring almost the exclusive use of functions that return the object to manipulate objects. It is very un-Swifty.

This would turn

{
Body()
.setInnerText("<div/>")
.setFont(Font())
}

Into:

{
Body() {.innerText = "<div>"; .font = Font(); .whatEver}
}

Correct? If so, bravo. You fixed one of the ugliest parts of SwiftUI/Builder functions.

1 Like