Compiler directive for current closure reference

Evolution Discussion
1

Dear all,

I apologise in advance if this has already been suggested (or maybe even implemented), but its very difficult to keep track of the swift evolution list :)

We already have compiler directives like #line, #function. I would like to suggest a new directive #closure (name preliminary) that expands to the reference to the closure, function or method in whose body the directive occurs. Motivation: this will allow one to conveniently use design patterns like these:

addHandler({
  guard stillRelevant() else {
          removeHandler(#closure)
  }
      
       doSomething()
})

I am sure there are also other cases where such directive would come in handy.

2

Closures don't have identity (i.e. you can't use === on them). ObjC blocks do, so if you really need it you can pass things around as @convention(objc_block), but closures don't. So I think you'd have to start there if you want to do anything like this.

Jordan

···

On Feb 22, 2016, at 10:20, Taras Zakharko via swift-evolution <swift-evolution@swift.org> wrote:

Dear all,

I apologise in advance if this has already been suggested (or maybe even implemented), but its very difficult to keep track of the swift evolution list :)

We already have compiler directives like #line, function. I would like to suggest a new directive closure (name preliminary) that expands to the reference to the closure, function or method in whose body the directive occurs. Motivation: this will allow one to conveniently use design patterns like these:

addHandler({
  guard stillRelevant() else {
         removeHandler(closure)
  }

      doSomething()
})

I am sure there are also other cases where such directive would come in handy.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

3

Closures don't have identity (i.e. you can't use === on them). ObjC blocks do, so if you really need it you can pass things around as @convention(objc_block), but closures don't. So I think you'd have to start there if you want to do anything like this.

Actually, I've wondered about this for a while.

Closures are pretty weird in that they're a reference type, but they don't have a stable identity. I asked about this once before Swift was open sourced (I think on Twitter), and I believe I was told that it was due to certain optimizations. What are these optimizations, and can we give closures stable identities despite them?

In a few minutes of research, my guess is that it has to do with thick vs. thin closures: thick closures carry a context object, while thin ones don't. <https://github.com/apple/swift/blob/master/docs/CallingConvention.rst#closures&gt; Is that the issue here? If so…

- Leaving aside thin closures for the moment, is a thick closure's context object unique enough to be used as its identity? Is function pointer + context unique enough, or does the function pointer change too?

- Now, for thin closures, obviously we don't want to allocate an entire unnecessary object to carry an empty context around. But what would the costs be if, instead of omitting the context pointer, we replaced it with a unique integer (presumably with the low bit set to avoid colliding with valid context pointers)? Perhaps keep a thread-local counter and combine it with the thread ID so we don't need any locking?

(As for motivation: besides just being kind of a weird asymmetry, there are certain classes of APIs which are made more convoluted by the lack of closure identity. Basically any API where you can register and unregister multiple handlers with a single object is affected: the "add" operation has to return some kind of token to be used for the "remove" operation, which may now have a different lifetime from the registration itself. It seems simpler to add and remove the closure itself, but the lack of identity prevents that.)

···

--
Brent Royal-Gordon
Architechies

4

Closures don't have identity (i.e. you can't use === on them). ObjC blocks do, so if you really need it you can pass things around as @convention(objc_block), but closures don't. So I think you'd have to start there if you want to do anything like this.

Actually, I've wondered about this for a while.

Closures are pretty weird in that they're a reference type, but they don't have a stable identity. I asked about this once before Swift was open sourced (I think on Twitter), and I believe I was told that it was due to certain optimizations. What are these optimizations, and can we give closures stable identities despite them?

I think it's more proper to think of closures as *immutable*—they're above either reference or value semantics. Closures can capture references to shared mutable state, either by closing over class references or `var` boxes, but that state is shared with code in the same lexical scope, not part of the closure itself. Swift will mess with the underlying identity of closures for a couple of reasons today:

- When functions are passed into contexts with different genericity, we may "reabstract" them, thunking to get a better calling convention for the destination's generic abstraction level. If we weren't allowed to reabstract, function objects would always need to use an inefficient indirect calling convention to be usable from generic contexts.
- We defer closure formation in some cases, particularly when using local `func` decls:

func foo(x: Int, asynchronously: Bool) {
  func bar() { print(x) }

  if !asynchronously {
    bar() // We don't form a closure on this branch
  } else {
    async(bar) // We do on this branch
  }
}

If every reference to `bar` needed to form an equivalent function object, it would be harder to do this.

Future potential optimizations include replacing a function literal that's equivalent with a function or method call with a direct reference to that function (e.g. { $0.method($1, $2) }), or stack-allocating closures to be lazily copied to the heap on escape, the same way ObjC blocks are explicitly heap-promoted with _Block_copy.

Even without these optimization considerations, relying on the identity of specific block objects is fraught with problems, and in practice there's usually a better discriminator available to you somewhere to key on. We could possibly support identity of function or method *declarations*, similar to selectors in ObjC, but those would have to be at best a subtype of all function values.

-Joe

···

On Feb 22, 2016, at 8:00 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

In a few minutes of research, my guess is that it has to do with thick vs. thin closures: thick closures carry a context object, while thin ones don't. <https://github.com/apple/swift/blob/master/docs/CallingConvention.rst#closures&gt; Is that the issue here? If so…

- Leaving aside thin closures for the moment, is a thick closure's context object unique enough to be used as its identity? Is function pointer + context unique enough, or does the function pointer change too?

- Now, for thin closures, obviously we don't want to allocate an entire unnecessary object to carry an empty context around. But what would the costs be if, instead of omitting the context pointer, we replaced it with a unique integer (presumably with the low bit set to avoid colliding with valid context pointers)? Perhaps keep a thread-local counter and combine it with the thread ID so we don't need any locking?

(As for motivation: besides just being kind of a weird asymmetry, there are certain classes of APIs which are made more convoluted by the lack of closure identity. Basically any API where you can register and unregister multiple handlers with a single object is affected: the "add" operation has to return some kind of token to be used for the "remove" operation, which may now have a different lifetime from the registration itself. It seems simpler to add and remove the closure itself, but the lack of identity prevents that.)

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

5

Oh, true, I have completely forgot about this litte detail. The truth is, I haven’t tried any pattern like this in Swift yet — it just occurred to my mind because I do use it quite often in a tool I work on now (Python).

I would also be curious to the reason why closures can’t be compared. I can imagine a bunch of optimisations which make it impossible (e.g. inlining closure body into another function), but I think this can be solved with some effort. Ideally, I’d like to see something like a Callable/Closure protocol that all closures (including functions and bound methods) adhere to, and which also allows identity comparison. This also opens up a formalism for argument and return type introspection as well as a way to reintroduce the tuple splat operation, e.g. by using an explicit .call() method of the Callable. Of course, details of such a proposal need to be fleshed up. Can someone do that? :)

— Taras

···

On 23 Feb 2016, at 05:00, Brent Royal-Gordon <brent@architechies.com> wrote:

Closures don't have identity (i.e. you can't use === on them). ObjC blocks do, so if you really need it you can pass things around as @convention(objc_block), but closures don't. So I think you'd have to start there if you want to do anything like this.

Actually, I've wondered about this for a while.

Closures are pretty weird in that they're a reference type, but they don't have a stable identity. I asked about this once before Swift was open sourced (I think on Twitter), and I believe I was told that it was due to certain optimizations. What are these optimizations, and can we give closures stable identities despite them?

In a few minutes of research, my guess is that it has to do with thick vs. thin closures: thick closures carry a context object, while thin ones don't. <https://github.com/apple/swift/blob/master/docs/CallingConvention.rst#closures&gt; Is that the issue here? If so…

- Leaving aside thin closures for the moment, is a thick closure's context object unique enough to be used as its identity? Is function pointer + context unique enough, or does the function pointer change too?

- Now, for thin closures, obviously we don't want to allocate an entire unnecessary object to carry an empty context around. But what would the costs be if, instead of omitting the context pointer, we replaced it with a unique integer (presumably with the low bit set to avoid colliding with valid context pointers)? Perhaps keep a thread-local counter and combine it with the thread ID so we don't need any locking?

(As for motivation: besides just being kind of a weird asymmetry, there are certain classes of APIs which are made more convoluted by the lack of closure identity. Basically any API where you can register and unregister multiple handlers with a single object is affected: the "add" operation has to return some kind of token to be used for the "remove" operation, which may now have a different lifetime from the registration itself. It seems simpler to add and remove the closure itself, but the lack of identity prevents that.)

--
Brent Royal-Gordon
Architechies

6

Even without these optimization considerations, relying on the identity of specific block objects is fraught with problems, and in practice there's usually a better discriminator available to you somewhere to key on.

I'm sorry, but I'm really having trouble understanding this point of view. Consider this:

  behavior var observable<Value where Self: class>: Value {
    typealias Observer = (observed: Self, oldValue: Value, newValue: Value) -> Void

    initialValue
    
    private var value: Value = initialValue
    
    // This would be better as a Set by making functions Hashable, but I understand
    // that compound types currently can't be conformed to protocols.
    private var observers: [ObjectIdentifier: Observer]
    
    func addObserver(observer: Observer) {
      observers[ObjectIdentifier(observer)] = observer
    }
    func removeObserver(observer: Observer) {
      observers[ObjectIdentifier(observer)] = nil
    }
    
    get { return value }
    set {
      let oldValue = value
      value = newValue
      for observer in observers.values {
        observer(observed: self, oldValue: oldValue, newValue: newValue)
      }
    }
  }

What is the better discriminator here? Sure, you could make Observer into a class that contains a closure, but how does that make anything better? It's an extra object to track and manage; it's an impediment to using trailing closure syntax with `addObserver`; it's more state you could accidentally share; it's additional memory and additional reference counting.

If there are implementation reasons not to support it, fine. But let's not pretend we're doing people a service by preventing this, like when we prevent enum extensions from adding cases to make code correctness more decidable, or that we're reflecting some fundamental semantic truth, like when we don't give value types identities because they get implicitly copied around and independently mutated.

There are perfectly sensible ways to define closure identity—a particular (source-level) function with a particular context, or a closure captured at a particular point in the program's execution—but we don't support the operation because we'd have to either spend extra memory on something rarely used, or give up very useful optimizations. There's no shame in that, and no need to try to turn it into a positive instead of something negative but necessary.

···

--
Brent Royal-Gordon
Architechies