[Concurrency] Interoperability with Objective-C

[fbruneau (Florent Bruneau)]

@Douglas_Gregor

An Objective-C method is potentially an asynchronous completion-handler method if it meets the following requirements:

• The method has a completion handler parameter, which is an Objective-C block that will receive the "result" of the asynchronous computation. It must meet the following additional constraints:
  • It has a void result type.
  • It is called exactly once along all execution paths through the implementation.
  • If the method can deliver an error, one of the parameters of the block is of type NSError * that is not _Nonnull . A non-nil NSError * value indicates that an error occurred.
• The method itself has a void result type, because all results are delivered by the completion handler block.

A common pattern is to have a method that takes a completion handler, but also returns an NSProgress.

- (NSProgress *)doSomethingThatTakesALongTimeWithCompletionHandler:(void (^)(MyResult * _Nullable, NSError * _Nullable))completionHandler;

The progress can then be used to populate some UI while the operation is in progress. My understanding of the proposal is that this will not be a pattern support by the concurrency model?

[Ben_Cohen]

@rbishop

@Douglas_Gregor

Otherwise, it will be imported as non-optional.

This is problematic. Many existing completion handlers do not require that either the parameter or an error be supplied - it is perfectly valid to call the block with nil for both. We should not automatically introduce a nil-unwrapping crash so being conservative here seems appropriate.

A lot of the prior experience with annotating the nullability of APIs went into this decision – and it's definitely not clear cut so very much open for discussion.

I think it's a mischaracterization to call defaulting to optional in this case "conservative". A default of optional when true optionals are rare is essentially a technical debt generator. It encourages code to not be audited, and to just stick with the optional default, because that's "safe". But this leads to an anti-pattern, similar to the false benefit of optional array subscripting: users become used to having to bang stuff because it's never nil (or spend hours writing nil-handling code that will never be executed and so is untestable). So you still get crashes, you just get to offload moral responsibility for those crashes to your caller, which is not cool.

Note that when you have a mis-annotated Objective-C API, you can use the peephole feature to immediately assign the value to an optional:

// if evilMisannotatedFunc() -> SomeType returns nil, x will be nil
let x: SomeType? =  evilMisannotatedFunc()

This leads to another benefit of this default: when your callers discover that you have misannotated your function, and are working around it by using this technique while you fix the ticket they filed, then when you correct the misannotation it will not break their source. Conversely, if they aren't checking for nil, the source break is desirable: you are informing them that this is really optional and they need to account for that in their logic.

Whereas the other way around is just annoying. Switching to a non-optional value is guaranteed to break the caller's source without bringing any benefit (other than the caller getting to delete the code you shouldn't have made them write in the first place). And given the majority of callback results are not truly optional, this kind of source break is likely to be common amongst teams while adopting async in their code base.

[xwu (Xiaodi Wu)]

Some comments on the text:

---

@Douglas_Gregor

• If the method has a single parameter, and the suffix of the first selector piece is the phrase WithCompletion or WithCompletionHandler , the sole parameter is the completion handler parameter. The matching phrase will be removed from the base name of the function when it is imported.
• If the method has more than one parameter, the last parameter is the completion handler parameter if its selector piece or parameter name is completion , completionHandler , or withCompletionHandler .

Unless I'm mistaken, the current implementation also accepts as part of the second heuristic last selector pieces that have a suffix of WithCompletion or WithCompletionHandler. More on this later.

---

@Douglas_Gregor

This will be translated into the following async function:

@objc func generateCGImagesAsynchronously(
    forTimes requestedTimes: [NSValue]
) async throws -> (CMTime, CGImage, CMTime, AVAssetImageGenerator.Result)

This text is inaccurate, as it is immediately contradicted by the following statement:

@Douglas_Gregor

Additional rules are applied when translating Objective-C method name into a Swift name an async function:

• [ ... ]
• If the base name of the method ends with Asynchronously , that word is removed.

[Ben_Cohen]

@Ben_Cohen

A lot of the prior experience with annotating the nullability of APIs went into this decision

I realize this statement possibly comes across as an appeal to authority. To be clear, what I mean is that my prior experience with API annotation is what influences my preference for the default being this way around. But on top of that anecdata/bias, I think the rationale makes sense.

[xwu (Xiaodi Wu)]

@Douglas_Gregor

If the completion handler parameter name has the suffix "WithCompletionHandler" or "WithCompletion", remove the suffix from the parameter name and append the remaining name to the prior argument name.

Take one of the test examples in the relevant PR:

// Objective-C declarations
-(void)server:(NSString *)name restartWithCompletionHandler:(void (^)(void))block;
-(void)server:(NSString *)name atPriority:(double)priority restartWithCompletionHandler:(void (^)(void))block;

// Swift use site
await slowServer.serverRestart("localhost")
await slowServer.server("localhost", atPriorityRestart: 0.8)

It may make sense to add the information that precedes WithCompletion--in the rare circumstances that there is any such information--to the base name at all times rather than the prior argument name. While the ergonomics surrounding completion handlers push strongly for such parameters to be last, the action being performed with a completion handler is now the action that is being awaited, and await comes at the front of the expression:

   server:atPriority:restartWithCompletionHandler:
//                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ trailing closures are ergonomic

   await server.server("localhost", atPriorityRestart: 0.8)
// ^^^^^ await ... wait for it ...            ^^^^^^^ ... restart

---

If the Objective-C importer were to translate these names by first logically shuffling this information to the front, it seems like it'd be a consistently better result:

// Original
thing:frobnicateWithCompletionHandler:

// Rearrange to pull name information next to `await` use site
frobnicateWithCompletionHandler:thing:

// Eliminate "WithCompletionHandler"
frobnicateThing:

---

If this rule were to be applied, then the example APIs above would become:

// Swift use site with alternative importer rules:
await slowServer.restartServer("localhost")
await slowServer.restartServer("localhost", atPriority: 0.8)

[Douglas_Gregor]

@John_McCall

That’s very fair, I’m not actually sure how it ended up here. At some point it was going to be separately pitched.

Ah, this is my mistake, sorry all. I wrote the Objective-C interoperability document under the assumption that @asyncHandler would have a pitch proposal available at the same time. We don't have that document ready to go yet, so I've removed @asyncHandler inference from this proposal to keep it from being more of a distraction.

Doug

[Douglas_Gregor]

@fbruneau

A common pattern is to have a method that takes a completion handler, but also returns an NSProgress .

- (NSProgress *)doSomethingThatTakesALongTimeWithCompletionHandler:(void (^)(MyResult * _Nullable, NSError * _Nullable))completionHandler;

That's an interesting idea! I'll look into it, thank you.

Doug

[Douglas_Gregor]

@xwu

It may make sense to add the information that precedes WithCompletion --in the rare circumstances that there is any such information--to the base name at all times rather than the prior argument name.

I like your reasoning here. There were zero cases of this pattern occurring anywhere in the SDKs I translated, so I picked a rule somewhat at random. I've queued this up for the next proposal update.

Doug

[Douglas_Gregor]

@rbishop

Can we not infer that if the completion is preceded by a DispatchQueue or OperationQueue that those values are the executor or execution context for the completion?

To do this, we'd need to wrap up whatever executor we have as either a DispatchQueue or an OperationQueue, and then suppress the normal "hop back to the right executor" code. The compiler-generated "hop back to the right executor" code (which can be optimized away if the compiler realizes that it doesn't matter where the code executes before its next suspension point or return) is likely to be more efficient than wrapping up the executor in a DispatchQueue or OperationQueue.

Browsing through the instances of this across the SDKs (there are < 40 total, many of which are repeated across the 4 platforms), nearly all of them accept "nil". One way to approach this would be to drop the queue parameter from the async signature as seen in Swift, and have the Swift compiler pass "nil".

Doug

[smontgomery (Stuart Montgomery)]

• If the method has a single parameter, and the suffix of the first selector piece is the phrase WithCompletion or WithCompletionHandler, the sole parameter is the completion handler parameter. The matching phrase will be removed from the base name of the function when it is imported.
• If the method has more than one parameter, the last parameter is the completion handler parameter if its selector piece or parameter name is completion, completionHandler, or withCompletionHandler.

Has there been much thought about how the ObjC method name inference rules will work with protocols used with NSXPC APIs?

Methods in XPC protocols have async semantics and may include a completion handler block parameter. But if they do include a completion handler, its ObjC selector piece is usually suffixed reply:, WithReply:, or similar, without the word “completion”. I’m not sure if this is actually required by NSXPC, but the documentation encourages that convention and anecdotally it seems very common within Apple.

Should the inference rules in this proposal be expanded to also include block parameters whose labels have the suffix reply:? I suspect this naming convention would not show up in an audit of Apple’s public APIs since XPC protocols are typically wrapped by external-facing API, but Swift developers would likely encounter this within their own codebases which interact with XPC services or daemons.

[jrose (Jordan Rose)]

A thought on the implementation details:

@Douglas_Gregor

Overriding methods that have been translated into both completion-handler and async versions have a similar problem to protocol requirements: a Swift subclass can either override the completion-handler version or the async version, but not both. Objective-C callers will always call to the subclass version of the method, but Swift callers to the "other" signature will not unless the subclass's method is marked with @objc dynamic . Swift can infer that the async overrides of such methods are @objc dynamic to avoid this problem (because such async methods are new code). However, inferring @objc dynamic on existing completion-handler overrides can change the behavior of programs and break subclasses of the subclasses, so at best the compiler can warn about this situation.

I think this ought to be fine as long as we don't encounter this situation:

class Base: NSFoo {
  override func lookupName(completionHandler: (String) -> Void)
}

class Sub: Base {
  override func lookupName() async -> String {
  }
}

As long as only one requirement is overridden in the hierarchy, it doesn't matter which one is called (because calling the other will go through ObjC anyway). In this situation, the compiler could even implicitly override the completion handler version of the function to call the async version. You could have that work with the @objc dynamic rule, or just make it bidirectional.

---

I'd also like to throw my support behind "no overloading based on async" and just have the few conflicting cases stick "Async" or "Asynchronously" in the base name. It just makes things simpler all around, and I suspect it doesn't come up too much. (Checking my assumption against the Apple SDKs would be nice!)

---

I'm trying not to get too involved in this effort, but this set of proposals have all been very well-thought-out and I feel a lot better about adding these features to Swift because of it, as well as because of the community feedback. Kudos to everyone involved.

[benlings]

I’ve used a couple of other languages with async/await (JS and C#) and found there that the Async suffixes became pretty noisy. In the distant future, presumably most asynchronous APIs will use async and existing ones with completion handlers will be deprecated. When it gets to that point, I don’t think we’ll want to still be having to write/look at await fooAsync() everywhere - it should be await foo(). Whatever is done now should be done with this goal in mind.

[davedelong]

@benlings

I’ve used a couple of other languages with async/await (JS and C#) and found there that the Async suffixes became pretty noisy. In the distant future, presumably most asynchronous APIs will use async and existing ones with completion handlers will be deprecated. When it gets to that point, I don’t think we’ll want to still be having to write/look at await fooAsync() everywhere - it should be await foo() . Whatever is done now should be done with this goal in mind.

This makes me wonder (and I'm sorry if this has been proposed; it's hard to keep up with everything) if we should consider something like @asyncMembers and @nonasync, as analogies to @objcMembers and @nonobjc

[alex.vasenin (Alexander Vasenin)]

@Douglas_Gregor

Completion handlers must be called exactly once

I think we need to find a better example than AVAssetImageGenerator.generateCGImagesAsynchronously(forTimes:completionHandler:), because it calls its completion handler exactly once per each requested time.

The idea itself is great, BTW

[AnotherUser]

@alex.vasenin makes a good point. Does Swift do any static checks on the function it's translating before calling it async? I imagine it would be rather awkward if an Obj-C function called its completion handler more (or less) than once, breaking a fundamental assumption of async functions. Does it have access to the Obj-C implementation, or does it just assume that the creator followed the standard convention?

[Douglas_Gregor]

@benlings

I’ve used a couple of other languages with async/await (JS and C#) and found there that the Async suffixes became pretty noisy. In the distant future, presumably most asynchronous APIs will use async and existing ones with completion handlers will be deprecated. When it gets to that point, I don’t think we’ll want to still be having to write/look at await fooAsync() everywhere - it should be await foo() . Whatever is done now should be done with this goal in mind.

Yeah. I think the expectation is that overloading where you have one synchronous and one asynchronous operation with the same signature is fairly rare, and that one would only add the Async (or Asynchronously) suffix when you need to break the ambiguity. I did a quick experiment, and out of all of the macOS/iOS/watchOS/tvOS SDKs, only 60 async methods (out of ~2,200) were overloaded with synchronous counterparts. It's possible that these SDKs aren't representative of what the larger body of code looks like, but for my that number is low enough that I'm not so concerned about Async suffixes everywhere.

Doug

[Douglas_Gregor]

@alex.vasenin

I think we need to find a better example than AVAssetImageGenerator.generateCGImagesAsynchronously(forTimes:completionHandler:) , because it calls its completion handler exactly once per each requested time .

Hah! Serves me right for grabbing a convenient API without checking the documentation. Thank you, I'll pick another example with slightly more care.

@AnotherUser

Does Swift do any static checks on the function it's translating before calling it async ? I imagine it would be rather awkward if an Obj-C function called its completion handler more (or less) than once, breaking a fundamental assumption of async functions. Does it have access to the Obj-C implementation, or does it just assume that the creator followed the standard convention?

It does not have access to the ObjC implementation, so it has to make an assumption based on the convention. If that assumption is wrong, it's up to the ObjC declaration to state that it should not be imported as async with __attribute((swift_async(none))).

Doug

[Douglas_Gregor]

@smontgomery

Methods in XPC protocols have async semantics and may include a completion handler block parameter. But if they do include a completion handler, its ObjC selector piece is usually suffixed reply: , WithReply: , or similar, without the word “completion”. I’m not sure if this is actually required by NSXPC, but the documentation encourages that convention and anecdotally it seems very common within Apple.

Should the inference rules in this proposal be expanded to also include block parameters whose labels have the suffix reply: ?

That's a good idea; I hadn't seen this convention before.

@smontgomery

I suspect this naming convention would not show up in an audit of Apple’s public APIs since XPC protocols are typically wrapped by external-facing API, but Swift developers would likely encounter this within their own codebases which interact with XPC services or daemons.

It does show up a couple of dozen times in the public SDKs.

Doug

[Douglas_Gregor]

@Douglas_Gregor

@fbruneau

A common pattern is to have a method that takes a completion handler, but also returns an NSProgress .

- (NSProgress *)doSomethingThatTakesALongTimeWithCompletionHandler:(void (^)(MyResult * _Nullable, NSError * _Nullable))completionHandler;

That's an interesting idea! I'll look into it, thank you.

I looked into a bit further, and it's a bit tricky to make this work. The issue is that the translation is a two-way street: we could import an NSProgress-returning method as async and ignore the progress (for example), but we don't have a way to implement a corresponding @objc method in Swift and produce a reasonable NSProgress result for Objective-C.

Doug

[Michael_Ilseman]

This looks awesome! I am happy to see such a good interoperability story (and even happier to not worry about the details).

• If the method can deliver an error and a given parameter has the _Nullable_result nullability qualifier (see the section on Objective-C attributes below), it will be imported as optional. Otherwise, it will be imported as non-optional.

Taking a default view of a nil parameter signifying a failure is interesting and pretty aggressive!

@Ben_Cohen

A default of optional when true optionals are rare is essentially a technical debt generator.

@Ben_Cohen

when you correct the misannotation it will not break their source.
...
Switching to a non-optional value is guaranteed to break the caller's source
...
the majority of callback results are not truly optional

These are pretty good arguments. Do you have a rough feel of the ratio from the ~1k APIs in the iOS 14.0 SDK mentioned in the document?