Receiver Closures

Receiver Closures

Hello,

I would like to get some feedback on a pitch idea.

Idea

allow specifying the receiver of a closure. (what a closure is being called on)

This would modify how self behaves inside a closure body.

Kotlin already has something similar Higher-Order Functions and Lambdas - Kotlin Programming Language and imo its really useful.

This is somewhat possible in current Swift using self rebinding.

let closure: (String) -> Void = { `self` in
	print(self)
}

closure("string") // prints: "string"

However referencing self always has do be done explicitly.

struct Person { var name: String }

var bruce = Person(name: "bruce")

var name = "not bruce"

let closure: (Person) -> Void = { `self` in
	print("self.name = \(self.name)")
	print("name = \(name)") // references outer variable name
}
closure(bruce) 
// prints: "self.name = bruce"
// prints: "name = not bruce"

With this proposal it would be possible to define a ReceiverClosure
inside the closure implicit (end explicit) self refers to the receiver (the thing the function is being called on).

// assuming there is a way to define this
typealias ReceiverClosure<Receiver, Args, ReturnType>

// and assuming a way to call it
func call<Receiver, Args, ReturnType>(
			_ closure: ReceiverClosure<Receiver, Args, ReturnType>
			on receiver: Receiver,
			arguments: Args
)
// the example before would behave differently

struct Person { var name: String }

var bruce = Person(name: "bruce")

var name = "not bruce"

let printNames: ReceiverClosure<Person, Void, Void> 
	= { // it is no longer necessary  to bind self 
		print("self.name = \(self.name)")
		print("name = \(name)") // references inner variable name
	}

// asummes call function exists
call(closure, on: bruce, arguments: ()) 
// prints: "self.name = bruce"
// prints: "name = bruce"

// behaviour inside closure body should be same as:
private extension Person {
	func printName() {
		print("self.name = \(self.name)")
		print("name = \(name)")
	}
}

UseCases

Simplifying weak self in closures


func weakCall <Receiver: AnyObject>(
	on: Receiver,
	_ closure: @escaping ReceiverClosure<Receiver, Void, Void>
) -> () -> Void {
	return { [weak value] in
		guard let strongValue = value else {
			return
		}
		
		call(closure, on: strongValue, arguments: ()) 
	}
}


class Computer {
	var computedValue: Int?

	func done() {}
	func computeNumber(_ completion: @escaping (Int) -> Void) 

	func start() {
		computeNumber(weakCall(receiver: self) { number in
			// can directly refer to self without unwrapping from weak self
			self.done()
			// implicit self also works
			computedValue = number
		})
	}
}

Nicer DSL Syntax

With the current Swift a DSL for building HTML could be defined like
this:


typealias Builder<T> = (T) -> Void

class HtmlElement {}

final class HTML: HtmlElement {
    var head = Head()
    var body = Body()
}

func html(_ build: Builder<HTML>) -> HTML {
    var result = HTML()
    build(result)
    return result
}

final class Body: HtmlElement {
    var text: String?
    var children: [HtmlElement] = []
    
    func div(_ build: Builder<Div>) {
        var div = Div()
        div.apply(build)
        children.append(div)
    }

}

final class Head: HtmlElement {
    var title: String?
}

final class Div: HtmlElement {
    var text: String?
}

extension HTML {
    func head(_ build: Builder<Head>) {
        build(head)
    }
    
    func body(_ build: Builder<Body>) {
        build(body)
    }
}

Usage:

let website = html {
    $0.head { $0.title = "Awesome Title" }
    $0.body {
      $0.div { $0.text = "div1" }
      $0.div { $0.text = "div2" }
    }
}

Having to always specify $0 really hurts readability.

By replacing the Builder closures with ReceiverClosures the same behaviour can acheived in a more concise way.

// before:
typealias Builder<T> = (T) -> Void

// after:
typealias Builder<T> = ReceiverClosure<T, Void, Void>

the example could be rewritten as:

let website = html {
    head { title = "Awesome Title" }
    body {
      div { text = "div1" }
      div { text = "div2" }
    }
}

How to define a ReceiverClosure

To work with Value semantics it should be made possible to define ReceiverClosure and MutatingReceiverClosure
a MutatingReceiverClosure behaves the same as a normal ReceiverClosure but in addition is also allowed to mutate self

typealias MutatingReceiverClosure<Receiver, Args, ReturnType>

// a way to call it
func apply(
	closure: MutatingReceiverClosure,
	to receiver: inout Receiver, 
	arguments: Args
) // generics ommited for brevity

Some Options to define those are:

Prefix function definition with ReceiverType


typealias ReceiverClosure<Receiver, Args, ReturnType>
	= Receiver.(Args) -> ReturnType 

// uses mutating keyword to create mutable variant
typealias MutatingReceiverClosure<Receiver, Args, ReturnType>
	= mutating Receiver.(Args) -> ReturnType 

// Example:

let printUppercased: String.() -> Void {
	print(uppercased()) 
}

let clear: mutating String.() -> Void { 
	self = "" 
}

This is also the a "Receiver Lambda" is defined in kotlin

Introduce receiver function parameter attribute


// new attribute `reveiver`
typealias ReceiverClosure<Receiver, Args, ReturnType>
	= (receiver Receiver, Args) -> ReturnType 
// only one parameter can be marked as `receiver`

typealias MutatingReceiverClosure<Receiver, Args, ReturnType>
	= (receiver inout Receiver, Args) -> ReturnType

Calling a ReceiverClosure

The previous examples avoided dealing with this problem by assuming the existence of a function callfunction which allows calling a receiver lambda.

A way to think about a ReceiverClosure is a function KeyPath
This could also be the syntax used for calling / creating a ReceiverClosure


struct Person {
	var name: String

	func introduce() {
		print("i am \(self.name)")
	}

	mutating func setName(_ newName: String) {	
		name = newName
	}
}


// could potentially be named:
// FunctionKeyPath<Base, Args, ReturnType>
// MutatingFunctionKeyPath<Base, Args, ReturnType>
 
let greet: ReceiverClosure<Person, Void, Void> 
 	= \Person.introduce 
let compare: MutatingReceiverClosure<Person, String, Void>
	=  \Person.setName


let person = Person(name: "tim")

person[keyPath: greet]()
// prints "i am tim"

person[keyPath: setName]("peter")  // compiler error cannot use mutating on immutable person

var mutablePerson = person

mutablePerson[keyPath: setName]("peter") // works 
mutablePerson[keyPath: greet]()
// prints "i am peter"

A potential way to call a ReceiverClosure on a receiver is via dot syntax


struct Person {
	var name: String

	func introduceSelf() {
		print("i am \(self.name)")
	}
	
}

let greetPerson: ReveiverClosure<Person, Person, Void> = {
		print("hi \($0.name)!")
		introduceSelf()
}

let han = Person(name: "han")
let luke = Person(name: "luke")

// method is not defined on Person but can still be called 
han.greetPerson(luke) 
// prints:
// hi luke!
// i am han

This could be used to make member functions "replaceable"

struct Person {
	var name: String

	var introduceSelf = ReceiverClosure<Self, Void, Void> = {
		print("i am \(self.name)")
	}

	// is equivalent to
	// func introduceSelf() {
	//		print("i am \(self.name)")
	// }
}

var luke = Person(name: "luke")

luke.introduceSelf() 
// prints:
// i am luke

luke.introduceSelf = {
	print("I AM \(name.upperCased())")
}

luke.introduceSelf() 
// prints:
// I AM LUKE

Please let me know what you think,
Simon

10 Likes

For the "simplifying weak self", the "each method is a func of type (Self) -> (<Args>) -> Result"-feature is something that imho should be mentioned in the pitch:

let computeDistance = Int.distance
let dist = computeDistance(1)(4) // 3

For the DSL syntax part, imho drawing is a good example:
Neither a global variable for the graphics context is nice, nor being forced to mention it in every call (and both should be familiar for many of us).

1 Like

This is one of those features that makes me envious of Kotlin. It's super nice to define nice higher order helpers (check Kotlin apply, with, let... and specially to use in DSLs. The typical "Object().then {}" pattern would be much nicer with this.

There has been some discussion in another thread on improvements over capture lists, this may be something related to it that would be good to think as a whole.

I would be really in favour of this, I'm just not sure at this point what's the best implementation or syntax, but that's why we have smarter people in the forums ^^

This looks great, I'd love to see DSLs in Swift.

Bit of bikeshedding: I'd prefer a syntax more aligned to ‘real’ function types, along the lines of (@self Person, String) -> () or @receiver (Person, String) -> ().

I'm very supportive of this pitch, even if I don't know how it should be done. I would rationalize my support by saying that there exist some $0 that are only noise.

The HTML example in the OP is striking.

My own SQL DSL could profit from this as well:

// Improved SQL generation
try db.create(table: "player") {
    autoIncrementedPrimaryKey("id")
    column("name", .text).notNull()
    column("score", .integer).notNull()
}

try db.alter(table: "player") {
    addColumn("email", .text)
}

My own experience with such a feature in Ruby tells me that redefining self can create small inconveniences:

struct Client {
    var bar: ...
    func doStuff {
        let foo = ...
        enterDSL {
            // Can't use outer foo or self.bar here if
            // foo or bar have been redefined by the closure receiver.
        }
    }
}

Well, in order to avoid this conflicting situation, the library consumer has to rename his own conflicting identifiers. This means that the library designer has to be careful avoiding reserving DSL words that are likely to be application identifiers.

When put in the hands of a careful library designer, such a feature can shine.

@MutatingFunk
Using @self or maybe even self would be a nice way to specify the receiver, since it wouldn't introduce a new keyword and actually describes whats happening pretty well.

(self Receiver, Arg1, Arg2, ...) -> ReturnType

I am personally leaning towards the Receiver prefix syntax for some reasons:

Having the Receiver be part of the Argument list implies that it will be passed as a argument to the closure and i also would assume to have to call it that way.( receiverClosure(self, arg1, arg2))

With receiver prefix:

Receiver.(Arg1, Arg2, ...) -> ReturnType

  • This to me more clearly separates the receiver from the function arguments.
  • Its easier to write
  • It also avoids having to enforce correct usage of self / receiver attribute
(String, self String, self String) -> Void
//                    ^
//                    error: self/receiver attribute may only be used once
// maybe also         error: self/receiver attribute may only on first attribute

Also this way the Receiver feature could be introduces as a "upgrade" to normal closure.
This would assume that every closure is called on a Receiver but for normal closures the Receiver would be the Void type.

let closure: (String) -> Void 
// is equivalent to
let closure: Void.(String) -> Void

// Receiver type is present for every closure but defaults to Void

This would also follow the precedent of being allowed to omit Void in certain places.

2 Likes

@gwendal.roue
I think in your example it should actually be possible to use self.bar?
but it depends on how the enterDSL method is defined.

i am assuming something like

extension Client {
    func enterDSL(closure: ReceiverClosure<Client, Void, Void>) {
    }
}

using your example the behaviour could also be

struct Client {
    var bar = ...

    func doStuff {
        typealias OuterSelf = Self
        // OuterSelf == Client

        let outerSelf = self
        let foo = ...
        enterDSL { 
            typealias InnerSelf = Self 
           // InnerSelf is defined by ReceiverClosure
           // InnerSelf == Client

            print(self.bar)  // should work since
            print(bar)       // should also works

            print(foo) // could actually be allowed
            // works same like normal closure

            // which could allow referencing a self from a outer scope
            print(outerSelf)
        }
    }
}

I think allowing captures inside the closure would be preferable because the receiver closure, with the exception of redefining Self/ self, would still behave like a normal closure.
Which would avoid having to learn new rules for capturing when using receiver closures.

Additionally (just guessing) i think it would make it easier to implement.

My first reaction is to be pretty unhappy with this. Right now, implicit lookup does happen on the self parameter, but it follows rules of lexical scoping, i.e. "look where the code is written; the surrounding declarations tell you what it can access without qualification". This is implicitly changing that just because of how the closure was declared, something you can't really see at the call site. That said, it does seem like a way to inject context into a closure (the HTML builder), which has come up plenty.

I like the spellings that make this a normal closure with an attribute on the self parameter, but in practice it's not part of the closure's type, so I think the extra annotation, whatever it is, belongs on the parameter whose type is the closure. I do not think that (Self) -> (<Args>) -> Result is the right type for the closure because that doesn't support mutation, which I agree is a useful concept should we add this to the language.

4 Likes

FWIW Kotlin has ways to access “self” and even self from different nested scopes. It’s something that shows up a lot in kotlin code.

@jrose
Would that mean that introducing a way to differentiate calling a ReceiverClosure vs a normal closure would solve the lexical scoping problem?

If so some Alternatives for calling a ReceiverClosure could be

let receiver: String
let closure: ReceiverClosure<String, String, Void>

// use do keyword
receiver do closure("arg")

// indicate calling on a receiver with @
@receiver.closure("arg") 


It's not the call site of the closure I care about; it's the closure literal itself. That is, in

let website = html {
    head { title = "Awesome Title" }
    body {
      div { text = "div1" }
      div { text = "div2" }
    }
}

...there's no way to tell what head refers to without knowing that html takes a receiver-closure rather than a normal closure.

3 Likes

This topic has been previously discussed. It’s been a long time and I’m not sure what threads it’s in, but IIRC I pitched an idea along the lines of supporting syntax at the call site which would make the first argument an implicit `self in the body of the closure. This makes the behavior clear at the call site and doesn’t require a special declaration by the library in order to be used.

The syntax might look something like this:

let website = html @{
    head @{ title = "Awesome Title" }
    body @{
      div @{ text = "div1" }
      div @{ text = "div2" }
    }
}

The obvious downside is that it adds syntactic weight to the call site. But I think the benefit of clarity is probably more important. This feels similar to the reasons we mark throwing call sites with try, for example.

2 Likes

@Alejandro_Martinez, @anreitersimon, I think the pitch should be very explicit about scopes.

Specifically, I have three questions, embedded in the code snippet below:

struct Page {
    var title: String
    var text: String
    func makeHTML() -> HTML {
        let title = self.title.uppercased()
        return html {
            head { title = /* Q1: can I access title variable here? How? */ }
            body {
              p { text = /* Q2: can I access text property here? How? */ }
            }
            print(type(of: self)) // Q3: what does it print?
        }
    }
}

I also think that in the closure itself there should be some hint so that the reader knows what its contents are referring to.
One simple way would be to special-case a self parameter:

let website = html { self in
    head { $0.title("Awesome Title" }
    body { self in
        div { $0.text("div1") }
        div { $0.text("div2") }
    }
}

With such a parameter, the closure would be parsed just like a method of type(of:self).
And if the self in feels too heavy for your closure, you can still simply use $0, as shown above.
I.e., this closure would not have any special type, just a normal (HTML) -> () function.

1 Like

Yes I agree. Of course we should have a much more specific idea of what's needed before pursuing this. And as Jordan pointed out it won't be without challenges. And I'm no expert here I'm just pointing out that other languages have this and I see it used a lot in everyday development.

By the looks fo it in Kotlin it's like you need to call the closure as if it were a member of the receiving type:

fun html(init: HTML.() -> Unit): HTML {
    val html = HTML()  // create the receiver object
    html.init()        // pass the receiver object to the lambda
    return html
}

html {       // lambda with receiver begins here
    body()   // calling a method on the receiver object
}

but as you say that's only visible at the caller site, the callee still doesn't know it, which is the whole point of the feature from a human perspective, but I get that the compiler will need more. Kotlin seems to solve this by having a different type of function for them: Functions with receivers.

In Kotlin Function literals with receiver it's interesting to me how they describe it as

This behavior is similar to extension functions, which also allow you to access the members of the receiver object inside the body of the function.

Although in Swift I'm not sure implementation wise how similar this would be to extensions on types.

Looking and Matthew idea it seems that may solve Jordan concern? It doesn't look bad to me.

Of course is not as clean as not having any syntactical difference but as pointed out it may be more inline with swift.

As an alternative not use @, would dot syntax make any sense?

html.{
    head.{ title = "Awesome Title" }
}

--

In terms of accessing different scopes Kotlin allows to access all the different scopes when this closures are nested see Scope control. But looking at Gwendal questions I'm not sure how that helps tbh.

1 Like

If I understand that correctly I think it defeats the purpose of receiver closures. The receiver (the new self) and an argument ($0) don't have to be the same objects. See Context object: this or it (substitute this for self, and it for $0) , it describes when to use one or the other. Also, it seems that this design makes the caller decide when to use a receiver instead of the API designer, no?

I'm not saying we need to go overboard and have all Scope Functions because having so many is quite confusing, but it would be interesting to keep them in mind to ensure whatever design Swift comes up with can tackle those situations.

Sorry for not being able to add more concrete solutions to the pitch.

To be clear, I'm just worried about the human implications of this, not the compiler side. At least at the moment. I'll admit that it might just be a reaction to something new and different, though—there's a story about how when C++ templates were introduced they needed a whole separate keyword (template) to call out how something ~unusual~ was going on, but now many years later people complain that templates feel "heavyweight".

6 Likes

FWIW i have been using the kotlin receivers lambda for a while, and this is a problem that i haven't encountered often.
To be fair this is probably because of the way the IDE surfaces the fact that a Receiver Lambda is called.

// kotlin code

fun takesIntReceiver(lambda: Int.() -> Void ) {}

// when called
takesIntReceiver { //this: Int
//                  ^
//                  IDE displays this

}

Introducing this features will make it harder to read Swift code without IDE (like reviewing a PR) but i think it could be worth it.

I think a way to go about this could be to start off with requiring syntax making a call to a ReceiverClosure explicit.
And evaluate if it makes sense to make this definition optional after the feature is used.
Still allowing users who prefer readability over the concise syntax to use it.

I like the @ syntax proposed by @anandabits but would like extend it a bit:
With the currently discussed syntax its not possible to define a ReceiverClosure without relying on explicitly defining its type.

By adding the ReceiverType in addition to the @ this would also be possible

let closure = @Int{ (param: String) in 
   // do something
}
// compiler infers closure to be ReceiverClosure<Int, String, Void>

let closure = @Int{ in
   // do something
}
// compiler infers closure to be ReceiverClosure<Int, Void, Void>

If the compiler can infer what the Receiver of a closure is, the type could also be omitted.
Still (mostly) allowing the proposed syntax.

let closure: ReceiverClosure<Int, String, Void> = @{ param in }

I've been thinking about a feature like this quite a bit and I'd love to see this get picked up into Swift. I've used a bunch of Ruby DSLs built on BasicObject#instance_exec, which is basically the same thing.

One thing I'd like to use this for in particular is to build a with-function (variants of which have been discussed on here before), that is used like this:

let label = with(UILabel()) {
    backgroundColor = .green
    text = "hello there"
}

And while things like the html-dsl in the pitch are already kind of possible-ish to build (free functions, stash away the builder objects in a stack somewhere) this functionality would make them much easier and type-safe to build (see Ruby, where DSLs built on instance_exec are absolutely everywhere).


Changing what self means in the closure like this however is both confusing if you don't know of this feature (well, it was for me in Ruby) and somewhat limiting (I'd expect to see a bunch of let actualSelf = self before these closures to regain access to members of self).

Tacking something onto the closure literal kind of solves the first, but not the second, and looks pretty ugly imho, esp in DSLs that might be deeply nested and contain many nested closures.

An alternative might be to allow an annotation on one (or many, why not) parameters in a closure type, let's call it @implicitLookupable for now, that adds the argument to the list of things implicit member lookup is done on, so that both members of self and the parameter are available. If there are conflicts between members of self and members of the @implicitLookupable parameter, you can resolve by prefixing with self. or param..

I'm not sure whether and how this annotation should also allow you to omit the parameter in your closure literals. At least in the case where there are no other parameters this would be pretty important, I think.

2 Likes

FWIW, with the suggestion I posted, the with function you can already write would just work. You would write:

let label = with(UILabel()) @{
    backgroundColor = .green
    text = "hello there"
}

This has the advantage of making the implicit self in the closure clear at the point of use. It also works with all of our existing code that was not written with receiver closures in mind. Can anyone identify an advantage of annotating the parameters themselves instead of just modifying the closures at the point of use?