Proposal: add `noescape` attribute to public API (particularly libdispatch)

jtbandes · December 5, 2015, 8:53pm

There are several functions in public API, such as dispatch_sync and
dispatch_apply, which do not allow their block parameters to escape the
function call.

However, they are not currently exposed to Swift as @noescape, meaning that
users are required to specify "self." or "[weak self]" in their closures.

The instances I can think of:

- dispatch_sync
- dispatch_barrier_sync
- dispatch_apply
- dispatch_block_perform (thanks to Pierre Habouzit for pointing this out)

I provided a preliminary implementation at
Add & use DISPATCH_NOESCAPE attribute macro by jtbandes · Pull Request #6 · apple/swift-corelibs-libdispatch · GitHub, but it needs
refinement. Tony Parker mentioned commented that there might be similar
changes to make for the swift-corelibs-foundation project.

Thoughts?

Question: does __attribute__((noescape)) have the same meaning when applied
to C function pointers? Namely, does Swift understand @noescape as it
applies to @convention(c)?

Jacob Bandes-Storch

Pierre_Habouzit · December 5, 2015, 9:39pm

There are several functions in public API, such as dispatch_sync and dispatch_apply, which do not allow their block parameters to escape the function call.

However, they are not currently exposed to Swift as @noescape, meaning that users are required to specify "self." or "[weak self]" in their closures.

The instances I can think of:

- dispatch_sync
- dispatch_barrier_sync
- dispatch_apply
- dispatch_block_perform (thanks to Pierre Habouzit for pointing this out)

I also pointed out dispatch_once(), which even it doesn’t make sense for swift where the globals already have the proper semantics, is still useful as the compiler can still implement various optimizations in how the block captures its variables (it is okay to use the things where they are on the stack as opposed to have to create a real state for the block).

···

On Dec 5, 2015, at 12:53 PM, Jacob Bandes-Storch via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

I provided a preliminary implementation at Add & use DISPATCH_NOESCAPE attribute macro by jtbandes · Pull Request #6 · apple/swift-corelibs-libdispatch · GitHub, but it needs refinement. Tony Parker mentioned commented that there might be similar changes to make for the swift-corelibs-foundation project.

Thoughts?

Question: does __attribute__((noescape)) have the same meaning when applied to C function pointers? Namely, does Swift understand @noescape as it applies to @convention(c)?

Jacob Bandes-Storch
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Tony_Parker · December 5, 2015, 9:40pm

Hi Jacob,

Thanks for bringing the discussion on this topic here.

One topic I wanted to discuss was finding a comprehensive list of API that needs the attribute. The next step after that is figuring out how to get this attribute to be consistent across Darwin and Swift open source.

One option was to use the API notes feature of the compiler to annotate these APIs as part of the import into Swift. Another is to get the C headers changed (on Darwin too) to add the attribute. We may want to do one approach in the short term and another in the long term.

- Tony

···

On Dec 5, 2015, at 12:53 PM, Jacob Bandes-Storch via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

There are several functions in public API, such as dispatch_sync and dispatch_apply, which do not allow their block parameters to escape the function call.

However, they are not currently exposed to Swift as @noescape, meaning that users are required to specify "self." or "[weak self]" in their closures.

The instances I can think of:

- dispatch_sync
- dispatch_barrier_sync
- dispatch_apply
- dispatch_block_perform (thanks to Pierre Habouzit for pointing this out)

I provided a preliminary implementation at Add & use DISPATCH_NOESCAPE attribute macro by jtbandes · Pull Request #6 · apple/swift-corelibs-libdispatch · GitHub, but it needs refinement. Tony Parker mentioned commented that there might be similar changes to make for the swift-corelibs-foundation project.

Thoughts?

Question: does __attribute__((noescape)) have the same meaning when applied to C function pointers? Namely, does Swift understand @noescape as it applies to @convention(c)?

Jacob Bandes-Storch
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

jtbandes · December 5, 2015, 9:44pm

One option was to use the API notes feature of the compiler to annotate
these APIs as part of the import into Swift. Another is to get the C
headers changed (on Darwin too) to add the attribute. We may want to do one
approach in the short term and another in the long term.

I've been operating under the assumption that __attribute__((noescape))
will eventually be deeply understood by clang, and allow the same elision
of "self" in Objective-C that @noescape does in Swift. (I filed
<rdar://19914650> for this some time ago.)

Thus, I think it makes the most sense to change the C headers.

However, if swift-corelibs-libdispatch isn't going to affect the Swift
versions that ship with Xcode in the near future, then I guess amending the
API notes is a good idea, so the @noescape versions can get into
developers' hands as soon as possible.

Jacob

jtbandes · December 5, 2015, 10:25pm

On this subject, here are some other APIs which could use the annotation.
Again, I think these should be annotated even in the C/Objective-C headers,
so they can benefit C/Objective-C callers as well.

- bsearch, heapsort, qsort, mergesort, psort, and their _b variants

"passing test" methods:
- -[{NSArray,NSOrderedSet} indexOfObjectPassingTest:]
- -[{NSArray,NSOrderedSet} indexesOfObjectsPassingTest:]
- -[{NSArray,NSOrderedSet} indexOfObjectsAtIndexes:passingTest:]
- -[{NSArray,NSOrderedSet} indexesOfObjectsAtIndexes:passingTest:]
- -[SCNNode childNodesPassingTest:]
- -[SCNSceneSource entriesPassingTest:]
- -[AVAudioUnitComponentManager componentsPassingTest:]
- -[NSIndexSet indexPassingTest:]
- -[NSIndexSet indexWithOptions:passingTest:]
- -[NSIndexSet indexesPassingTest:]
- -[NSIndexSet indexesWithOptions:passingTest:]
- -[NSIndexSet indexInRange:options:passingTest:]
- -[NSIndexSet indexesInRange:options:passingTest:]
- -[NSSet objectsPassingTest:]
- -[NSSet objectsWithOptions:passingTest:]
- -[NSDictionary keysOfEntriesPassingTest:]
- -[NSDictionary keysOfEntriesWithOptions:passingTest:]

"using comparator" methods:
- -[NSArray indexOfObject:inSortedRange:options:usingComparator:]
- -[NSArray sortedArrayUsingComparator:]
- -[NSArray sortedArrayWithOptions:usingComparator:]
- -[NSDictionary keysSortedByValueUsingComparator:]
- -[NSDictionary keysSortedByValueWithOptions:usingComparator:]
- -[NSMutableArray sortUsingComparator:]
- -[NSMutableArray sortWithOptions:usingComparator:]
- -[NSMutableOrderedSet sortWithOptions:usingComparator:]
- -[NSMutableOrderedSet sortRange:options:usingComparator:]

(and some of these have "usingFunction" variants)

"enumerate using block" methods:
- -[NSTableView enumerateAvailableRowViewsUsingBlock:]
- -[SKPhysicsWorld enumerateBodiesAtPoint:usingBlock:]
- -[SKPhysicsWorld enumerateBodiesInRect:usingBlock:]
- -[NSData enumerateByteRangesUsingBlock:]
- -[SCNNode enumerateChildNodesUsingBlock:]
- -[SCNNode enumerateChildNodesWithName:usingBlock:]
- -[NSIndexSet enumerateIndexesUsingBlock:]
- -[NSIndexSet enumerateIndexesWithOptions:usingBlock:]
- -[NSDictionary enumerateKeysAndObjectsUsingBlock:]
- -[NSDictionary enumerateKeysAndObjectsWithOptions:usingBlock:]
- -[NSString enumerateLinesUsingBlock:]
- -[NSString enumerateSubstringsInRange:options:usingBlock:]
- -[{NSArray,NSSet,NSOrderedSet} enumerateObjectsUsingBlock:]
- -[{NSArray,NSSet,NSOrderedSet} enumerateObjectsWithOptions:usingBlock:]
- -[NSIndexSet enumerateRangesUsingBlock:]
- -[NSIndexSet enumerateRangesWithOptions:usingBlock:]
- -[NSIndexSet enumerateRangesInRange:options:usingBlock:]
- -[MPMediaEntity enumerateValuesForProperties:usingBlock:]
- -[NSMetadataQuery enumerateResultsUsingBlock:]
- -[NSMetadataQuery enumerateResultsWithOptions:usingBlock:]

misc:
- -[NSDocument performSynchronousFileAccessUsingBlock:]

Wow, that's more than I expected to find! This is obviously beyond the
scope of just libdispatch, and I think a lot of these should probably
change upstream, in their original frameworks, internally (not something
that community members can really contribute to).

Arguably, the "enumerateXUsingBlock:" pattern should be bridged to Swift as
something SequenceType-ish which can be `for-in`ed.

Jacob Bandes-Storch

···

On Sat, Dec 5, 2015 at 1:40 PM, Tony Parker <anthony.parker@apple.com> wrote:

One topic I wanted to discuss was finding a comprehensive list of API that
needs the attribute.

Pierre_Habouzit · December 5, 2015, 9:50pm

I completely agree with that statement, and I think that this change should make it into Darwin and swift-corelibs-libdispatch both.

-Pierre

···

On Dec 5, 2015, at 1:44 PM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

One option was to use the API notes feature of the compiler to annotate these APIs as part of the import into Swift. Another is to get the C headers changed (on Darwin too) to add the attribute. We may want to do one approach in the short term and another in the long term.

I've been operating under the assumption that __attribute__((noescape)) will eventually be deeply understood by clang, and allow the same elision of "self" in Objective-C that @noescape does in Swift. (I filed <rdar://19914650> for this some time ago.)

Thus, I think it makes the most sense to change the C headers.

jtbandes · December 8, 2015, 8:02am

Alright, I've written up a proposal:

Feedback welcome!

Jacob

···

On Sat, Dec 5, 2015 at 2:25 PM, Jacob Bandes-Storch <jtbandes@gmail.com> wrote:

On Sat, Dec 5, 2015 at 1:40 PM, Tony Parker <anthony.parker@apple.com> > wrote:

One topic I wanted to discuss was finding a comprehensive list of API
that needs the attribute.

On this subject, here are some other APIs which could use the annotation.
Again, I think these should be annotated even in the C/Objective-C headers,
so they can benefit C/Objective-C callers as well.

- bsearch, heapsort, qsort, mergesort, psort, and their _b variants

"passing test" methods:
- -[{NSArray,NSOrderedSet} indexOfObjectPassingTest:]
- -[{NSArray,NSOrderedSet} indexesOfObjectsPassingTest:]
- -[{NSArray,NSOrderedSet} indexOfObjectsAtIndexes:passingTest:]
- -[{NSArray,NSOrderedSet} indexesOfObjectsAtIndexes:passingTest:]
- -[SCNNode childNodesPassingTest:]
- -[SCNSceneSource entriesPassingTest:]
- -[AVAudioUnitComponentManager componentsPassingTest:]
- -[NSIndexSet indexPassingTest:]
- -[NSIndexSet indexWithOptions:passingTest:]
- -[NSIndexSet indexesPassingTest:]
- -[NSIndexSet indexesWithOptions:passingTest:]
- -[NSIndexSet indexInRange:options:passingTest:]
- -[NSIndexSet indexesInRange:options:passingTest:]
- -[NSSet objectsPassingTest:]
- -[NSSet objectsWithOptions:passingTest:]
- -[NSDictionary keysOfEntriesPassingTest:]
- -[NSDictionary keysOfEntriesWithOptions:passingTest:]

"using comparator" methods:
- -[NSArray indexOfObject:inSortedRange:options:usingComparator:]
- -[NSArray sortedArrayUsingComparator:]
- -[NSArray sortedArrayWithOptions:usingComparator:]
- -[NSDictionary keysSortedByValueUsingComparator:]
- -[NSDictionary keysSortedByValueWithOptions:usingComparator:]
- -[NSMutableArray sortUsingComparator:]
- -[NSMutableArray sortWithOptions:usingComparator:]
- -[NSMutableOrderedSet sortWithOptions:usingComparator:]
- -[NSMutableOrderedSet sortRange:options:usingComparator:]

(and some of these have "usingFunction" variants)

"enumerate using block" methods:
- -[NSTableView enumerateAvailableRowViewsUsingBlock:]
- -[SKPhysicsWorld enumerateBodiesAtPoint:usingBlock:]
- -[SKPhysicsWorld enumerateBodiesInRect:usingBlock:]
- -[NSData enumerateByteRangesUsingBlock:]
- -[SCNNode enumerateChildNodesUsingBlock:]
- -[SCNNode enumerateChildNodesWithName:usingBlock:]
- -[NSIndexSet enumerateIndexesUsingBlock:]
- -[NSIndexSet enumerateIndexesWithOptions:usingBlock:]
- -[NSDictionary enumerateKeysAndObjectsUsingBlock:]
- -[NSDictionary enumerateKeysAndObjectsWithOptions:usingBlock:]
- -[NSString enumerateLinesUsingBlock:]
- -[NSString enumerateSubstringsInRange:options:usingBlock:]
- -[{NSArray,NSSet,NSOrderedSet} enumerateObjectsUsingBlock:]
- -[{NSArray,NSSet,NSOrderedSet} enumerateObjectsWithOptions:usingBlock:]
- -[NSIndexSet enumerateRangesUsingBlock:]
- -[NSIndexSet enumerateRangesWithOptions:usingBlock:]
- -[NSIndexSet enumerateRangesInRange:options:usingBlock:]
- -[MPMediaEntity enumerateValuesForProperties:usingBlock:]
- -[NSMetadataQuery enumerateResultsUsingBlock:]
- -[NSMetadataQuery enumerateResultsWithOptions:usingBlock:]

misc:
- -[NSDocument performSynchronousFileAccessUsingBlock:]

Wow, that's more than I expected to find! This is obviously beyond the
scope of just libdispatch, and I think a lot of these should probably
change upstream, in their original frameworks, internally (not something
that community members can really contribute to).

Arguably, the "enumerateXUsingBlock:" pattern should be bridged to Swift
as something SequenceType-ish which can be `for-in`ed.

Jacob Bandes-Storch

jtbandes · December 5, 2015, 9:56pm

Pardon my ignorance, but what is intended to be the difference between the
two? Will they diverge?

···

On Sat, Dec 5, 2015 at 1:50 PM, Pierre Habouzit <phabouzit@apple.com> wrote:

I completely agree with that statement, and I think that this change
should make it into Darwin and swift-corelibs-libdispatch both.

-Pierre

Philippe_Hausler · December 9, 2015, 6:22pm

To re-iterate my comment on github; I think this is a great idea.

There are a couple of suggestions for scope limiting; since this will require changes in the c and objective-c layers to support this attribute it would be good to limit this for now to just the core libraries (dispatch, XCTest and Foundation).

Most likely future optimizations in c and objective c code can be applied in addition to the positive effect to swift code.

···

On Dec 8, 2015, at 12:02 AM, Jacob Bandes-Storch via swift-evolution <swift-evolution@swift.org> wrote:

Alright, I've written up a proposal: Proposal to add `@noescape` in library APIs by jtbandes · Pull Request #40 · apple/swift-evolution · GitHub

Feedback welcome!

Jacob

On Sat, Dec 5, 2015 at 2:25 PM, Jacob Bandes-Storch <jtbandes@gmail.com <mailto:jtbandes@gmail.com>> wrote:
On Sat, Dec 5, 2015 at 1:40 PM, Tony Parker <anthony.parker@apple.com <mailto:anthony.parker@apple.com>> wrote:
One topic I wanted to discuss was finding a comprehensive list of API that needs the attribute.

On this subject, here are some other APIs which could use the annotation. Again, I think these should be annotated even in the C/Objective-C headers, so they can benefit C/Objective-C callers as well.

- bsearch, heapsort, qsort, mergesort, psort, and their _b variants

"passing test" methods:
- -[{NSArray,NSOrderedSet} indexOfObjectPassingTest:]
- -[{NSArray,NSOrderedSet} indexesOfObjectsPassingTest:]
- -[{NSArray,NSOrderedSet} indexOfObjectsAtIndexes:passingTest:]
- -[{NSArray,NSOrderedSet} indexesOfObjectsAtIndexes:passingTest:]
- -[SCNNode childNodesPassingTest:]
- -[SCNSceneSource entriesPassingTest:]
- -[AVAudioUnitComponentManager componentsPassingTest:]
- -[NSIndexSet indexPassingTest:]
- -[NSIndexSet indexWithOptions:passingTest:]
- -[NSIndexSet indexesPassingTest:]
- -[NSIndexSet indexesWithOptions:passingTest:]
- -[NSIndexSet indexInRange:options:passingTest:]
- -[NSIndexSet indexesInRange:options:passingTest:]
- -[NSSet objectsPassingTest:]
- -[NSSet objectsWithOptions:passingTest:]
- -[NSDictionary keysOfEntriesPassingTest:]
- -[NSDictionary keysOfEntriesWithOptions:passingTest:]

"using comparator" methods:
- -[NSArray indexOfObject:inSortedRange:options:usingComparator:]
- -[NSArray sortedArrayUsingComparator:]
- -[NSArray sortedArrayWithOptions:usingComparator:]
- -[NSDictionary keysSortedByValueUsingComparator:]
- -[NSDictionary keysSortedByValueWithOptions:usingComparator:]
- -[NSMutableArray sortUsingComparator:]
- -[NSMutableArray sortWithOptions:usingComparator:]
- -[NSMutableOrderedSet sortWithOptions:usingComparator:]
- -[NSMutableOrderedSet sortRange:options:usingComparator:]

(and some of these have "usingFunction" variants)

"enumerate using block" methods:
- -[NSTableView enumerateAvailableRowViewsUsingBlock:]
- -[SKPhysicsWorld enumerateBodiesAtPoint:usingBlock:]
- -[SKPhysicsWorld enumerateBodiesInRect:usingBlock:]
- -[NSData enumerateByteRangesUsingBlock:]
- -[SCNNode enumerateChildNodesUsingBlock:]
- -[SCNNode enumerateChildNodesWithName:usingBlock:]
- -[NSIndexSet enumerateIndexesUsingBlock:]
- -[NSIndexSet enumerateIndexesWithOptions:usingBlock:]
- -[NSDictionary enumerateKeysAndObjectsUsingBlock:]
- -[NSDictionary enumerateKeysAndObjectsWithOptions:usingBlock:]
- -[NSString enumerateLinesUsingBlock:]
- -[NSString enumerateSubstringsInRange:options:usingBlock:]
- -[{NSArray,NSSet,NSOrderedSet} enumerateObjectsUsingBlock:]
- -[{NSArray,NSSet,NSOrderedSet} enumerateObjectsWithOptions:usingBlock:]
- -[NSIndexSet enumerateRangesUsingBlock:]
- -[NSIndexSet enumerateRangesWithOptions:usingBlock:]
- -[NSIndexSet enumerateRangesInRange:options:usingBlock:]
- -[MPMediaEntity enumerateValuesForProperties:usingBlock:]
- -[NSMetadataQuery enumerateResultsUsingBlock:]
- -[NSMetadataQuery enumerateResultsWithOptions:usingBlock:]

misc:
- -[NSDocument performSynchronousFileAccessUsingBlock:]

Wow, that's more than I expected to find! This is obviously beyond the scope of just libdispatch, and I think a lot of these should probably change upstream, in their original frameworks, internally (not something that community members can really contribute to).

Arguably, the "enumerateXUsingBlock:" pattern should be bridged to Swift as something SequenceType-ish which can be `for-in`ed.

Jacob Bandes-Storch

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution