Pitch: @autoreleasepool attribute for loops

When dealing with Objective-C code, especially older code that predates ARC, much of which can be found in standard Apple frameworks such as Foundation and AppKit, one is sure to generate a lot of autoreleased objects. In cases where a lot of autoreleased objects are generated inside a loop, this can lead to excessive memory usage unless one puts the body of the loop inside an autorelease pool. Traditionally, this would have been done in Objective-C like so:

for id foo in bar @autoreleasepool {
    …
}

The Swift equivalent of this would be:

for foo in bar {
    autoreleasepool {
        …
    }
}

However, due to Swift’s “autoreleasepool” function being implemented via a closure, this introduces a new scope, causing many forms of flow control not to work. Most problematically, the ‘break’ and ‘continue’ statements no longer function, since the control flow is no longer considered to be inside a loop, but there are other consequences too, such as not being able to early-return out of the method, or throw an error. The latter issue can be solved by writing a custom version of the “autoreleasepool” function that takes a throwing closure and rethrows it, but I can’t think of any workaround for the lack of “break” and “continue” with the current system other than breaking up the loop body into lots of smaller autoreleasepool statements.

I propose adding an @autoreleasepool attribute that could be added to loops, like so:

@autoreleasepool for foo in bar {
  ...
}

This would wrap the loop body inside an autorelease pool, without disabling the usual flow control statements.

What do you think?

Charles

I sympathize with the problem statement, but I wonder if we should instead
be considering ways of allowing control-flow statements (break, continue,
return) to work from inside @noescape closures.

Another possible solution would be to expose __pushAutoreleasePool() and
__popAutoreleasePool() as public API. This would also allow more careful
manual use of autorelease pools (for instance, draining the pool after
every 5 loop iterations, rather than every iteration).

Jacob

···

On Fri, Jan 8, 2016 at 5:32 PM, Charles Srstka via swift-evolution < swift-evolution@swift.org> wrote:

When dealing with Objective-C code, especially older code that predates
ARC, much of which can be found in standard Apple frameworks such as
Foundation and AppKit, one is sure to generate a lot of autoreleased
objects. In cases where a lot of autoreleased objects are generated inside
a loop, this can lead to excessive memory usage unless one puts the body of
the loop inside an autorelease pool. Traditionally, this would have been
done in Objective-C like so:

for id foo in bar @autoreleasepool {
    …
}

The Swift equivalent of this would be:

for foo in bar {
    autoreleasepool {
        …
    }
}

However, due to Swift’s “autoreleasepool” function being implemented via a
closure, this introduces a new scope, causing many forms of flow control
not to work. Most problematically, the ‘break’ and ‘continue’ statements no
longer function, since the control flow is no longer considered to be
inside a loop, but there are other consequences too, such as not being able
to early-return out of the method, or throw an error. The latter issue can
be solved by writing a custom version of the “autoreleasepool” function
that takes a throwing closure and rethrows it, but I can’t think of any
workaround for the lack of “break” and “continue” with the current system
other than breaking up the loop body into lots of smaller autoreleasepool
statements.

I propose adding an @autoreleasepool attribute that could be added to
loops, like so:

@autoreleasepool for foo in bar {
        ...
}

This would wrap the loop body inside an autorelease pool, without
disabling the usual flow control statements.

What do you think?

Charles

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Agreed, this seems like the right path forward.

-Chris

···

On Jan 8, 2016, at 10:00 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

I sympathize with the problem statement, but I wonder if we should instead be considering ways of allowing control-flow statements (break, continue, return) to work from inside @noescape closures.

@noescape is insufficient to allow that. You need more constraints on the closure's caller, perhaps even cooperation from the closure's caller depending on how the control flow is supposed to work.

Off the top of my head:
* The closure's caller can't call the closure more than once. Or if it does, it needs to participate in the control flow system somehow. (Example: what does `break` do in the middle of map().)
* Return values are complicated. You might need to limit the system to void returning closures only.
* Defining the closure in one place and handing it off in another is complicated. You might need to limit the system to closure literals only.

···

On Jan 8, 2016, at 10:00 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

I sympathize with the problem statement, but I wonder if we should instead be considering ways of allowing control-flow statements (break, continue, return) to work from inside @noescape closures.

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler

I’ve pondered proposing adding break and continue as results from a closure the same way return and ErrorType are today, and a modifier flag on function declarations similar to “throws” to indicate support.

The main complexity seemed to be in supporting labels (as well as when to not support them - labels would only make sense when the closure was declared inline.) Another thought was that it did not make sense for closure-level return, break, and continue to all be available at once

-DW

···

On Jan 9, 2016, at 1:21 PM, Greg Parker via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 8, 2016, at 10:00 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I sympathize with the problem statement, but I wonder if we should instead be considering ways of allowing control-flow statements (break, continue, return) to work from inside @noescape closures.

@noescape is insufficient to allow that. You need more constraints on the closure's caller, perhaps even cooperation from the closure's caller depending on how the control flow is supposed to work.

Off the top of my head:
* The closure's caller can't call the closure more than once. Or if it does, it needs to participate in the control flow system somehow. (Example: what does `break` do in the middle of map().)
* Return values are complicated. You might need to limit the system to void returning closures only.
* Defining the closure in one place and handing it off in another is complicated. You might need to limit the system to closure literals only.

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

+1
This would be helpful in several other cases that have come up on this list, e.g. it would allow creating new control statements with trailing closures which could offer expected behavior, e.g. a timesRepeat method instead of adding new looping syntax for that as discussed in another thread.
Furthermore it might allow "return" to be used for non-local return.

-Thorsten

···

Am 09.01.2016 um 19:47 schrieb Chris Lattner via swift-evolution <swift-evolution@swift.org>:

On Jan 8, 2016, at 10:00 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

I sympathize with the problem statement, but I wonder if we should instead be considering ways of allowing control-flow statements (break, continue, return) to work from inside @noescape closures.

Agreed, this seems like the right path forward.

Ruby has rules for how "return", "next" (= continue) and "break" behave in closures which match the expectations quite well. It allows next and break to return values, too.

-Thorsten

···

Am 09.01.2016 um 21:21 schrieb Greg Parker via swift-evolution <swift-evolution@swift.org>:

On Jan 8, 2016, at 10:00 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

I sympathize with the problem statement, but I wonder if we should instead be considering ways of allowing control-flow statements (break, continue, return) to work from inside @noescape closures.

@noescape is insufficient to allow that. You need more constraints on the closure's caller, perhaps even cooperation from the closure's caller depending on how the control flow is supposed to work.

Off the top of my head:
* The closure's caller can't call the closure more than once. Or if it does, it needs to participate in the control flow system somehow. (Example: what does `break` do in the middle of map().)
* Return values are complicated. You might need to limit the system to void returning closures only.
* Defining the closure in one place and handing it off in another is complicated. You might need to limit the system to closure literals only.

--
Greg Parker gparker@apple.com Runtime Wrangler

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I sympathize with the problem statement, but I wonder if we should instead be considering ways of allowing control-flow statements (break, continue, return) to work from inside @noescape closures.

Agreed, this seems like the right path forward.

+1
This would be helpful in several other cases that have come up on this list, e.g. it would allow creating new control statements with trailing closures which could offer expected behavior, e.g. a timesRepeat method instead of adding new looping syntax for that as discussed in another thread.
Furthermore it might allow "return" to be used for non-local return.

+1

···

On Jan 11, 2016, at 11:13 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:

Am 09.01.2016 um 19:47 schrieb Chris Lattner via swift-evolution <swift-evolution@swift.org>:

On Jan 8, 2016, at 10:00 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

-Thorsten
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Forgot to add this link describing the rules:

They are stated somewhere in Ruby's documentation as well but I don't have that link at hand right now.

-Thorsten

···

Am 11.01.2016 um 18:15 schrieb Thorsten Seitz via swift-evolution <swift-evolution@swift.org>:

Ruby has rules for how "return", "next" (= continue) and "break" behave in closures which match the expectations quite well. It allows next and break to return values, too.

-Thorsten

Am 09.01.2016 um 21:21 schrieb Greg Parker via swift-evolution <swift-evolution@swift.org>:

On Jan 8, 2016, at 10:00 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

I sympathize with the problem statement, but I wonder if we should instead be considering ways of allowing control-flow statements (break, continue, return) to work from inside @noescape closures.

@noescape is insufficient to allow that. You need more constraints on the closure's caller, perhaps even cooperation from the closure's caller depending on how the control flow is supposed to work.

Off the top of my head:
* The closure's caller can't call the closure more than once. Or if it does, it needs to participate in the control flow system somehow. (Example: what does `break` do in the middle of map().)
* Return values are complicated. You might need to limit the system to void returning closures only.
* Defining the closure in one place and handing it off in another is complicated. You might need to limit the system to closure literals only.

--
Greg Parker gparker@apple.com Runtime Wrangler

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution