Unavailable From Async Attribute
- Authors: Evan Wilde
- Status: Partially Implemented
- Implementation: Underscored @_unavailableFromAsync attribute, Attribute with optional message
- Discussion: Discussion: Unavailability from asynchronous contexts
I am ready to fully pitch the @unavailableFromAsync attribute that I had initially brought up in the discussion by the same name here: Discussion: Unavailability from asynchronous contexts. Thank you @beccadax for the detailed feedback on the initial discussion.
Edit 2: Replacing @unavailableFromAsync spelling with @available(noasync).
Introduction
The Swift concurrency model allows tasks to resume on different threads from the
one they were suspended on. For this reason, API that relies on thread-local
storage, locks, mutexes, and semaphores, should not be used across suspension
points.
func badAsyncFunc(_ mutex: UnsafeMutablePointer<pthread_mutex_t>, _ op : () async -> ()) async {
// ...
pthread_mutex_lock(mutex)
await op()
pthread_mutex_unlock(mutex) // Bad! May unlock on a different thread!
// ...
}
The example above exhibits undefined behaviour if badAsyncFunc resumes on a
different thread than the one it started on after running op since
pthread_mutex_unlock must be called from the same thread that locked the
mutex.
Swift evolution thread: Discussion: Unavailability from asynchronous contexts
Motivation
The Swift concurrency model allows tasks to suspend and resume on different
threads. While this behaviour allows higher utility of computational resources,
there are some nasty pitfalls that can spring on an unsuspecting programmer. One
such pitfall is the undefined behaviour from unlocking a pthread_mutex_t from
a different thread than the thread that holds the lock. Reading from and writing
to thread-local storage across suspension points can also result in unintended
behavior, since the operation may be resuming on a different thread.
Proposed Solution
We propose introducing a new availability kind to extend the functionality of
@available. The spelling for the new availability kind is noasync and
syntactically appears in the same place as unavailable or deprecated.
The noasync availability kind is applicable to most declarations, but is not
applicable to destructor declarations. Destructors are not explicitly called,
and must be callable from anywhere.
@available(*, noasync)
func doSomethingNefariousWithNoOtherOptions() { }
@available(*, noasync, message: "use our other shnazzy API instead!")
func doSomethingNefariousWithLocks() { }
func asyncFun() async {
// Error: doSomethingNefariousWithNoOtherOptions is unavailable from
// asynchronous contexts
doSomethingNefariousWithNoOtherOptions()
// Error: doSomethingNefariousWithLocks is unavailable from asynchronous
// contexts; use our other shanzzy API instead!
doSomethingNefariousWithLocks()
}
Using an annotated API from an asynchronous context results in an error.
In certain cases, it is possible to use the API safely within an asynchronous
context, but not across suspension points. The attribute is only checked in the
immediate context, so an unavailable function call wrapped in a synchronous
context will not emit the error. This allows functions to wrap the behaviour and
provide safe alternative APIs, like the example below:
func goodAsyncFunc(_ mutex: UnsafeMutablePointer<pthread_mutex_t>, _ op : () -> ()) async {
// not an error, pthread_mutex_lock is wrapped in another function
with_pthread_mutex_lock(mutex, do: op)
}
func with_pthread_mutex_lock<R>(
_ mutex: UnsafeMutablePointer<pthread_mutex_t>,
do op: () throws -> R) rethrows -> R {
switch pthread_mutex_lock(mutex) {
case 0:
defer { pthread_mutex_unlock(mutex) }
return try op()
case EINVAL:
preconditionFailure("Invalid Mutex")
case EDEADLK:
fatalError("Locking would cause a deadlock")
case let value:
fatalError("Unknown pthread_mutex_lock() return value: '\(value)'")
}
}
The above snippet is a safe wrapper for pthread_mutex_lock and
pthread_mutex_unlock, since the lock is not held across suspension points. The
critical section operation must be synchronous for this to hold true though.
The following snippet uses a synchronous closure to call the unavailable
function, circumventing the protection provided by the attribute.
@available(*, noasync)
func pthread_mutex_lock(_ lock: UnsafeMutablePointer<pthread_mutex_t>) {}
func asyncFun(_ mutex : UnsafeMutablePointer<pthread_mutex_t>) async {
// Error! pthread_mutex_lock is unavailable from async contexts
pthread_mutex_lock(mutex)
// Ok! pthread_mutex_lock is not called from an async context
_ = { unavailableFun(mutex) }()
await someAsyncOp()
}
Replacement API
In some cases, it is possible to provide an alternative that is safe. The
with_pthread_mutex_lock is an example of a way to provide a safe way to wrap
locking and unlocking pthread mutexes.
In other cases, it may be safe to use an API from a specific executor. For
example, API that uses thread-local storage isn't safe for consumption by
asynchronous functions in general, but is safe for functions on the MainActor
since it will only use the main thread.
The unavailable API should still be annotated as such, but an alternative
function can be implemented as an extension of an actor.
@available(*, noasync, renamed: "mainactorReadID()", message: "use mainactorReadID instead")
func readIDFromThreadLocal() -> Int { }
@MainActor
func readIDFromMainActor() -> Int { readIDFromThreadLocal() }
func asyncFunc() async {
// Bad, we don't know what thread we're on
let id = readIDFromThreadLocal()
// Good, we know it's coming from the main actor on the main thread.
// Note that we have to jump to the main actor, so there is a suspension.
let id = await readIDFromMainActor()
}
Custom executors are a goal for the language in the future. Restricting an API
to a custom executor is achieved in the same way that restricting the API to a
global actor is done. In many ways, they are the same thing. An actor ensures
that methods and operations on that actor are done on the executor for that
actor. By setting the unownedExecutor of an actor to the desired executor and
creating a wrapper function for the restricted API in that actor, the restricted
API is now only available for use from asynchronous contexts on that actor, and
in extension, that executor.
@available(*, unavailable, renamed: "IOActor.readInt()")
func readIntFromIO() -> String { }
extension IOActor {
// IOActor replacement API goes here
func readInt() -> String { readIntFromIO() }
}
actor MyIOActor : IOActor {
func printInt() {
// Okay! It's synchronous on the IOActor
print(readInt())
}
}
func print(myActor : MyIOActor) async {
// Okay! We only call `readIntFromIO` on the IOActor's executor
print(await myActor.readInt())
}
Additional design details
Verifying that unavailable functions are not used from asynchronous contexts is
done weakly; only unavailable functions called directly from asynchronous
contexts are diagnosed. This avoids the need to recursively typecheck the bodies
of synchronous functions to determine whether they are implicitly available from
asynchronous contexts, or to verify that they are appropriately annotated. While
the typechecker doesn't need to emit diagnostics from synchronous functions, they
cannot be omitted entirely. It is possible to declare asynchronous contexts
inside of synchronous contexts, wherein diagnostics should be emitted.
@available(*, noasync)
func bad2TheBone() {}
func makeABadAsyncClosure() -> () async -> Void {
return { () async -> Void in
bad2TheBone() // Error: Unavailable from asynchronous contexts
}
}
Source Compatibility
Swift 3 and Swift 4 do not have this attribute, so code coming from Swift 3 and
Swift 4 won't be affected.
The attribute will affect any current asynchronous code that currently contains
use of API that are modified with this attribute later. To ease the transition,
we propose that this attribute emits a warning in Swift 5.6, and becomes a full
error in Swift 6. In cases where someone really wants unsafe behavior and enjoys
living on the edge, the diagnostic is easily circumventable by wrapping the API
in a synchronous closure, noted above.
Effect on ABI stability
This feature has no effect on ABI.
Effect on API resilience
The presence of the attribute has no effect on the ABI.
Alternatives Considered
Propagation
The initial discussion focused on how unavailability propagated, including the
following three designs;
- implicitly-inherited unavailability
- explicit unavailability
- thin unavailability
The ultimate decision is to go with the thin checking; both the implicit and
explicit checking have high performance costs and require far more consideration
as they are adding another color to functions.
The attribute is expected to be used for a fairly limited set of specialized
use-cases. The goal is to provide some protection without dramatically impacting
the performance of the compiler.
Implicitly inherited unavailability
Implicitly inheriting unavailability would transitively apply the unavailability
to functions that called an unavailable function. This would have the lowest
developer overhead while ensuring that one could not accidentally use the
unavailable functions indirectly.
@unavailableFromAsync
func blarp1() {}
func blarp2() {
// implicitly makes blarp2 unavailable
blarp1()
}
func asyncFun() async {
// Error: blarp2 is impicitly unavailable from async because of call to blarp1
blarp2()
}
Unfortunately, computing this is very expensive, requiring type-checking the
bodies of every function a given function calls in order to determine if the
declaration is available from an async context. Requiring even partial
type-checking of the function bodies to determine the function declaration is
prohibitively expensive, and is especially detrimental to the performance of
incremental compilation.
We would need an additional attribute to disable the checking for certain
functions that are known to be usable from an async context, even though they
use contain unavailable functions. An example of a safe, but "unavailable"
function is with_pthread_mutex_lock above.
Explicit unavailability
This design behaves much like availability does today. In order to use an
unavailable function, the calling function must be explicitly annotated with the
unavailability attribute or an error is emitted.
Like the implicit unavailability propagation, we still need an additional
attribute to indicate that, while a function may contain unsafe API, it uses
them in a way that is safe for use in asynchronous contexts.
The benefits of this design are that it both ensures that unsafe API are explicitly
handled correctly, avoiding bugs. Additionally, typechecking asynchronous
functions is reasonably performant and does not require recursively
type-checking the bodies of every synchronous function called by the
asynchronous function.
Unfortunately, we would need to walk the bodies of every synchronous function to
ensure that every synchronous function is correctly annotated. This reverses the
benefits of the implicit availability checking, while having a high developer
overhead.
Separate Attribute
We considered using a separate attribute, spelled @unavailableFromAsync, to
annotate the unavailable API. After more consideration, it became apparent that
we would likely need to reimplement much of the functionality of the
@available attribute.
Some thoughts that prompted the move from @unavailableFromAsync to an
availability kind include:
- A given API may have different implementations on different platforms, and
therefore may be implemented in a way that is safe for consumption in
asynchronous contexts in some cases but not others. - An API may be currently implemented in a way that is unsafe for consumption
in asynchronous contexts, but may be safe in the future. - We get
message,renamed, and company, with serializations, for free by
merging this with@available.
Challenges to the merge mostly focus on the difference in the verification model
between this and the other availability modes. The noasync, as discussed
above, is a weaker check and does not require API that is using the unavailable
function to also be annotated. The other availability checks do require that the
availability information be propagated.
Acknowledgments
Thank you Becca for your detailed feedback on the initial discussion.
Thank you Doug for discussions on merging the attribute into @available.
