Making asynchronous operations first-class citizens in Swift

Dear all,

In the past few months I have been using Swift to write Warp (http://warp.one <http://warp.one/&gt;\) a data analysis/ETL app for OS X (±80k LOC). In this app I make heavy use of GCD (dispatch queues) and asynchronous calls. Although Swift has greatly simplified working with asynchronous operations for me compared to Objective C, asynchronous operations are still not really first-class citizens;

The standard library does not really define how errors should be handled in asynchronous scenarios. The throw/catch mechanism obviously doesn't work for asynchronous operations.
While the compiler checks if a function that returns a value returns one in all possible code paths, the compiler currently doesn't check whether an asynchronous function always 'calls back'. Also, while the compiler prevents you from returning twice from a function, it currently does not prevent you from calling a callback more than once (some callbacks actually are intended to be called more than once of course, but in most cases, but many aren't).
Currently you cannot make any assumptions about the queue/thread on which a callback will be called. This is especially problematic when you want to make UI updates from a callback - all you can do is dispatch_async your updates to the main queue.

Re 1, Error handling: in my own code I have defined a type that represents a 'failable' operation (more or less an Optional<T> with either a result object or an error message). I propose the standard library defines such a type before library authors all invent one themselves. My current version is listed below.

/** Fallible<T> represents the outcome of an operation that can either fail (with an error message) or succeed
(returning an instance of T). */
public enum Fallible<T> {
  case Success(T)
  case Failure(String)

  public init<P>(_ other: Fallible<P>) {
    switch other {
    case .Success(let s):
      self = .Success(s as! T)

    case .Failure(let e):
      self = .Failure(e)
    }
  }

  @warn_unused_result(message="Deal with potential failure returned by .use, .require to force success or .maybe to ignore failure.")
  public func use<P>(@noescape block: T -> Fallible<P>) -> Fallible<P> {
    switch self {
    case Success(let box):
      return block(box)

    case Failure(let errString):
      return .Failure(errString)
    }
  }
}

Re 2 (callbacks and returns): the way I currently 'solve' this is to wrap callbacks as follows, so that I get an assertion failure each time a callback is called more than once, so at least I find this out in testing. This doesn't solve the problem of functions not calling back at all:

public func Once<P, R>(block: ((P) -> (R))) -> ((P) -> (R)) {
  var run = false

  #if DEBUG
  return {(p: P) -> (R) in
    assert(!run, "callback called twice!")
    run = true
    return block(p)
  }
  #else
    return block
  #endif
}

it would be great if closures (or callback parameters) could be annotated with an attribute that indicates how many times the callback should be called under normal circumstances, e.g.:

func foo(callback: @once (Result) -> ()) { ... }

In this example, the callback must either be called exactly once in the body of the foo function, or passed exactly once to another function expecting an @once callback, or should be captured in one closure that always calls the callback once. The compiler should check whether all code paths lead to a single callback, and that no code paths lead to multiple callback invocations. The dispatch_sync and dispatch_async functions are examples of asynchronous functions with @once semantics.

Re 3 (the thread on which a callback is invoked): I currently 'solve' this by adding an assert to functions that can only be called on the main thread to ensure they are not called on another thread.

public func AssertMainThread(file: StaticString = __FILE__, line: UInt = __LINE__) {
  assert(NSThread.isMainThread(), "Code at \(file):\(line) must run on main thread!")
}

It might be worthwhile to add an attribute to mark functions as 'must always execute on main thread' (or 'must never execute on main thread' for blocking operations). 'Proving' that such a call actually never happens is more difficult however (but at least the attribute provides proper documentation and we can add assertions that check it automatically in debug mode).

Curious to see what you think. I would be willing to write a more formal/complete proposal for these suggestions, if you think they might be valuable additions to the Swift language and standard library.

Best regards,
Tommy.

Hi, Tommy. Just to make things clear: it's fine for people to discuss this as brainstorming on the list, but addressing concurrency issues is an explicit non-goal for Swift 3 <GitHub - apple/swift-evolution: This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.. We'll probably come back to it in the future, but that's still a ways off.

Best,
Jordan

···

On Dec 9, 2015, at 8:14, Tommy van der Vorst via swift-evolution <swift-evolution@swift.org> wrote:

Dear all,

In the past few months I have been using Swift to write Warp (http://warp.one <http://warp.one/&gt;\) a data analysis/ETL app for OS X (±80k LOC). In this app I make heavy use of GCD (dispatch queues) and asynchronous calls. Although Swift has greatly simplified working with asynchronous operations for me compared to Objective C, asynchronous operations are still not really first-class citizens;

The standard library does not really define how errors should be handled in asynchronous scenarios. The throw/catch mechanism obviously doesn't work for asynchronous operations.
While the compiler checks if a function that returns a value returns one in all possible code paths, the compiler currently doesn't check whether an asynchronous function always 'calls back'. Also, while the compiler prevents you from returning twice from a function, it currently does not prevent you from calling a callback more than once (some callbacks actually are intended to be called more than once of course, but in most cases, but many aren't).
Currently you cannot make any assumptions about the queue/thread on which a callback will be called. This is especially problematic when you want to make UI updates from a callback - all you can do is dispatch_async your updates to the main queue.

Re 1, Error handling: in my own code I have defined a type that represents a 'failable' operation (more or less an Optional<T> with either a result object or an error message). I propose the standard library defines such a type before library authors all invent one themselves. My current version is listed below.

/** Fallible<T> represents the outcome of an operation that can either fail (with an error message) or succeed
(returning an instance of T). */
public enum Fallible<T> {
  case Success(T)
  case Failure(String)

  public init<P>(_ other: Fallible<P>) {
    switch other {
    case .Success(let s):
      self = .Success(s as! T)

    case .Failure(let e):
      self = .Failure(e)
    }
  }

  @warn_unused_result(message="Deal with potential failure returned by .use, .require to force success or .maybe to ignore failure.")
  public func use<P>(@noescape block: T -> Fallible<P>) -> Fallible<P> {
    switch self {
    case Success(let box):
      return block(box)

    case Failure(let errString):
      return .Failure(errString)
    }
  }
}

Re 2 (callbacks and returns): the way I currently 'solve' this is to wrap callbacks as follows, so that I get an assertion failure each time a callback is called more than once, so at least I find this out in testing. This doesn't solve the problem of functions not calling back at all:

public func Once<P, R>(block: ((P) -> (R))) -> ((P) -> (R)) {
  var run = false

  if DEBUG
  return {(p: P) -> (R) in
    assert(!run, "callback called twice!")
    run = true
    return block(p)
  }
  #else
    return block
  #endif
}

it would be great if closures (or callback parameters) could be annotated with an attribute that indicates how many times the callback should be called under normal circumstances, e.g.:

func foo(callback: @once (Result) -> ()) { ... }

In this example, the callback must either be called exactly once in the body of the foo function, or passed exactly once to another function expecting an @once callback, or should be captured in one closure that always calls the callback once. The compiler should check whether all code paths lead to a single callback, and that no code paths lead to multiple callback invocations. The dispatch_sync and dispatch_async functions are examples of asynchronous functions with @once semantics.

Re 3 (the thread on which a callback is invoked): I currently 'solve' this by adding an assert to functions that can only be called on the main thread to ensure they are not called on another thread.

public func AssertMainThread(file: StaticString = __FILE__, line: UInt = __LINE__) {
  assert(NSThread.isMainThread(), "Code at \(file):\(line) must run on main thread!")
}

It might be worthwhile to add an attribute to mark functions as 'must always execute on main thread' (or 'must never execute on main thread' for blocking operations). 'Proving' that such a call actually never happens is more difficult however (but at least the attribute provides proper documentation and we can add assertions that check it automatically in debug mode).

Curious to see what you think. I would be willing to write a more formal/complete proposal for these suggestions, if you think they might be valuable additions to the Swift language and standard library.

Best regards,
Tommy.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

The standard library does not really define how errors should be handled in asynchronous scenarios. The throw/catch mechanism obviously doesn't work for asynchronous operations.
While the compiler checks if a function that returns a value returns one in all possible code paths, the compiler currently doesn't check whether an asynchronous function always 'calls back'. Also, while the compiler prevents you from returning twice from a function, it currently does not prevent you from calling a callback more than once (some callbacks actually are intended to be called more than once of course, but in most cases, but many aren't).

+1. I made a thread earlier about establishing guidelines for asynchronous callbacks, but long term, it would be great to see these promoted to being language constructs. According to the swift-evolution GitHub page, concurrency support is in the works, but is out of scope for Swift 3.0.

Currently you cannot make any assumptions about the queue/thread on which a callback will be called. This is especially problematic when you want to make UI updates from a callback - all you can do is dispatch_async your updates to the main queue.

I have reservations about this, as I usually see most users (when provided with classes that expose their underlying queue properties) over-abusing the main thread. For the specific issue you’re referencing, I’d rather see that addressed in the UI update model of AppKit/UIKit.

Dan