[Concurrency] Fixing race conditions in async/await example

I've read async/await proposal, and I'm thrilled by the possibilities. Here's what I consider the canonical example:
@IBAction func buttonDidClick(sender:AnyObject) {
  beginAsync {
    let image = await processImage()
    imageView.image = image
  }
}
This is exactly the kind of thing I will use async/await for!

But while this example looks extremely elegant, it would suffer from a number of problems in practice:

1. There is no guarantee that you are on the main thread after `await processImage()`
2. There is no way to cancel processing
3. Race Condition: If you click the button a second time before `processImage()` is done, two copies will run simultaneously and you don't know which image will "win".

So I wondered: What would a more thorough example look like in practice? How would I fix all these issues?

After some consideration, I came up with the following minimal example that addresses all these issues:
class ImageProcessingTask {
  var cancelled = false
  func process() async -> Image? { … }
}
var currentTask: ImageProcessingTask?
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  beginAsync {
    guard let image = await task.process() else { return }
    DispatchQueue.main.async {
      guard task.cancelled == false else { return }
      imageView.image = image
    }
  }
}
If my example isn't obvious, I've documented my thinking (and some alternatives) in a gist:
https://gist.github.com/jakob/22c9725caac5125c1273ece93cc2e1e7

Anyway, this more realistic code sample doesn't look nearly as nice any more, and I actually think this could be implemented nicer without async/await:

class ImageProcessingTask {
  var cancelled = false
  func process(completionQueue: DispatchQueue, completionHandler: (Image?)->()) { … }
}
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  task.process(completionQueue: DispatchQueue.main) { (image) in
    guard let image = image else { return }
    guard task.cancelled == false else { return }
    imageView.image = image
  }
}

So I wonder: What's the point of async/await if it doesn't result in nicer code in practice? How can we make async/await more elegant when calling from non-async functions?

Jakob

Maybe this will be handled more gracefully via the actor model.

1. if you're calling from an actor, you'd be called back on its internal queue. If you're calling from the 'main actor' (or just the main thread), you'd be called back on the main queue
2. you would just add a cancel() method to the actor
3. the processImage() method would call suspendAsync() and store the continuation block in an array, once the result is available it would call all the continuation blocks with the result

That would only work if we're able to send messages to the actor while previous messages to processImage() are pending. That's something that's not yet clear to me if actors can work that way.

Thomas

···

On 19 Aug 2017, at 13:56, Jakob Egger via swift-evolution <swift-evolution@swift.org> wrote:

I've read async/await proposal, and I'm thrilled by the possibilities. Here's what I consider the canonical example:
@IBAction func buttonDidClick(sender:AnyObject) {
  beginAsync {
    let image = await processImage()
    imageView.image = image
  }
}
This is exactly the kind of thing I will use async/await for!

But while this example looks extremely elegant, it would suffer from a number of problems in practice:

1. There is no guarantee that you are on the main thread after `await processImage()`
2. There is no way to cancel processing
3. Race Condition: If you click the button a second time before `processImage()` is done, two copies will run simultaneously and you don't know which image will "win".

So I wondered: What would a more thorough example look like in practice? How would I fix all these issues?

After some consideration, I came up with the following minimal example that addresses all these issues:
class ImageProcessingTask {
  var cancelled = false
  func process() async -> Image? { … }
}
var currentTask: ImageProcessingTask?
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  beginAsync {
    guard let image = await task.process() else { return }
    DispatchQueue.main.async {
      guard task.cancelled == false else { return }
      imageView.image = image
    }
  }
}
If my example isn't obvious, I've documented my thinking (and some alternatives) in a gist:
https://gist.github.com/jakob/22c9725caac5125c1273ece93cc2e1e7

Anyway, this more realistic code sample doesn't look nearly as nice any more, and I actually think this could be implemented nicer without async/await:

class ImageProcessingTask {
  var cancelled = false
  func process(completionQueue: DispatchQueue, completionHandler: (Image?)->()) { … }
}
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  task.process(completionQueue: DispatchQueue.main) { (image) in
    guard let image = image else { return }
    guard task.cancelled == false else { return }
    imageView.image = image
  }
}

So I wonder: What's the point of async/await if it doesn't result in nicer code in practice? How can we make async/await more elegant when calling from non-async functions?

Jakob

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Yeah, it's important to understand that coroutines don't directly offer any form of coordination; they only let you thread execution nicely through existing coordination mechanisms. IBActions by themselves don't offer any coordination, so anything more than fire-and-forget is still going to require explicit code. There are some interesting approaches you still might be able to explore to make this kind of thing nicer; for instance, if buttonDidClick didn't directly trigger the task, but instead communicated with a coroutine via synchronous channels in the style of Go, then that coroutine could be responsible for filtering multiple click events, and could also listen for cancellation events. The actor model Chris proposes in his document could conceivably let you wrap up that low-level channel management in a nice OO-looking wrapper.

-Joe

···

On Aug 19, 2017, at 4:56 AM, Jakob Egger via swift-evolution <swift-evolution@swift.org> wrote:

I've read async/await proposal, and I'm thrilled by the possibilities. Here's what I consider the canonical example:
@IBAction func buttonDidClick(sender:AnyObject) {
  beginAsync {
    let image = await processImage()
    imageView.image = image
  }
}
This is exactly the kind of thing I will use async/await for!

But while this example looks extremely elegant, it would suffer from a number of problems in practice:

1. There is no guarantee that you are on the main thread after `await processImage()`
2. There is no way to cancel processing
3. Race Condition: If you click the button a second time before `processImage()` is done, two copies will run simultaneously and you don't know which image will "win".

So I wondered: What would a more thorough example look like in practice? How would I fix all these issues?

After some consideration, I came up with the following minimal example that addresses all these issues:
class ImageProcessingTask {
  var cancelled = false
  func process() async -> Image? { … }
}
var currentTask: ImageProcessingTask?
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  beginAsync {
    guard let image = await task.process() else { return }
    DispatchQueue.main.async {
      guard task.cancelled == false else { return }
      imageView.image = image
    }
  }
}
If my example isn't obvious, I've documented my thinking (and some alternatives) in a gist:
https://gist.github.com/jakob/22c9725caac5125c1273ece93cc2e1e7

Anyway, this more realistic code sample doesn't look nearly as nice any more, and I actually think this could be implemented nicer without async/await:

class ImageProcessingTask {
  var cancelled = false
  func process(completionQueue: DispatchQueue, completionHandler: (Image?)->()) { … }
}
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  task.process(completionQueue: DispatchQueue.main) { (image) in
    guard let image = image else { return }
    guard task.cancelled == false else { return }
    imageView.image = image
  }
}

So I wonder: What's the point of async/await if it doesn't result in nicer code in practice? How can we make async/await more elegant when calling from non-async functions?

I think that the solution you are describing is how RxSwift (ReactiveX)
solves this problem.

I believe Rx, like many other higher level abstractions would benefit from
async, actors behind the scenes, as an implementation detail.

‫בתאריך יום ד׳, 23 באוג׳ 2017 ב-20:41 מאת ‪Joe Groff via swift-evolution‬‏
<‪swift-evolution@swift.org‬‏>:‬

···

On Aug 19, 2017, at 4:56 AM, Jakob Egger via swift-evolution < > swift-evolution@swift.org> wrote:

I've read async/await proposal, and I'm thrilled by the possibilities.
Here's what I consider the canonical example:

@IBAction func buttonDidClick(sender:AnyObject) {
  beginAsync {
    let image = await processImage()
    imageView.image = image
  }
}

This is exactly the kind of thing I will use async/await for!

But while this example looks extremely elegant, it would suffer from a
number of problems in practice:

1. There is no guarantee that you are on the main thread after `await
processImage()`
2. There is no way to cancel processing
3. Race Condition: If you click the button a second time before
`processImage()` is done, two copies will run simultaneously and you don't
know which image will "win".

So I wondered: What would a more thorough example look like in practice?
How would I fix all these issues?

After some consideration, I came up with the following minimal example
that addresses all these issues:

class ImageProcessingTask {
  var cancelled = false
  func process() async -> Image? { … }
}

var currentTask: ImageProcessingTask?
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  beginAsync {
    guard let image = await task.process() else { return }
    DispatchQueue.main.async {
      guard task.cancelled == false else { return }
      imageView.image = image
    }
  }
}

If my example isn't obvious, I've documented my thinking (and some
alternatives) in a gist:
https://gist.github.com/jakob/22c9725caac5125c1273ece93cc2e1e7

Anyway, this more realistic code sample doesn't look nearly as nice any
more, and I actually think this could be implemented nicer without
async/await:

class ImageProcessingTask {
  var cancelled = false
  func process(completionQueue: DispatchQueue, completionHandler:
(Image?)->()) { … }
}
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  task.process(completionQueue: DispatchQueue.main) { (image) in
    guard let image = image else { return }
guard task.cancelled == false else { return }
imageView.image = image
  }
}

So I wonder: What's the point of async/await if it doesn't result in nicer
code in practice? How can we make async/await more elegant when calling
from non-async functions?

Yeah, it's important to understand that coroutines don't directly offer
any form of coordination; they only let you thread execution nicely through
existing coordination mechanisms. IBActions by themselves don't offer any
coordination, so anything more than fire-and-forget is still going to
require explicit code. There are some interesting approaches you still
might be able to explore to make this kind of thing nicer; for instance, if
buttonDidClick didn't directly trigger the task, but instead communicated
with a coroutine via synchronous channels in the style of Go, then that
coroutine could be responsible for filtering multiple click events, and
could also listen for cancellation events. The actor model Chris proposes
in his document could conceivably let you wrap up that low-level channel
management in a nice OO-looking wrapper.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Using a Future library, see below, you can do what you want. In particular
a future that is cancellable is more powerful that the proposed
async/await. Here is an extended example of a typical UI task including
users cancelling tasks that is written using a future library (see below):

    @IBOutlet weak var timeButtonWasPressed: NSTextField!

    static func progress(_ window: NSWindow, _ progress:
NSProgressIndicator) -> Future<String> {
        return AsynchronousFuture(timeout: .seconds(3)) { isCancelled ->
String in // Timeout set to 3 seconds to ensure normal progress closes the
window before timeout does.
            defer { // Make sure the window always closes.
                Thread.executeOnMain {
                    window.close()
                }
            }
            var isFinished = false
            var isClosed = false
            while !(isFinished || isClosed || isCancelled()) {
                Thread.sleep(forTimeInterval: 0.1) // Would do real work
here!
                Thread.executeOnMain {
                    guard window.isVisible else { // Check if user has
closed the window.
                        isClosed = true
                        return
                    }
                    progress.increment(by: 1)
                    if progress.doubleValue >= progress.maxValue { // Check
if work done.
                        isFinished = true
                    }
                }
            }
            if isClosed || isCancelled() { // Cancelled by user closing
window, by call to `cancel`, or by timeout
                throw CancelFuture.cancelled
            }
            return "\(DispatchTime.now().uptimeNanoseconds)"
        }
    }

    var windowOrigin = NSPoint(x: 0, y: NSScreen.main?.visibleFrame.height
?? 0) // First window in top left of screen.

    @IBAction func buttonPushed(_ _: NSButton) {
        let progressFrame = NSRect(x: 10, y: 10, width: 200, height: 100)
// On main thread; pop up a progress window.
        let progress = NSProgressIndicator(frame: progressFrame)
        progress.minValue = 0
        progress.maxValue = 20
        progress.isIndeterminate = false
        let windowFrame = NSRect(x: 0, y: 0, width: progressFrame.width +
20, height: progressFrame.height + 20)
        let window = NSWindow(contentRect: windowFrame, styleMask:
[.titled, .closable], backing: .buffered, defer: false)
        window.contentView?.addSubview(progress)
        windowOrigin = window.cascadeTopLeft(from: windowOrigin) //
Position window.
        window.isReleasedWhenClosed = false // Needed to keep ARC happy!
        window.orderFront(self) // Display the window but don't give it the
focus.
        let _ = AsynchronousFuture { _ -> Void in // Runs on global default
queue.
            let creationTime = ViewController.progress(window,
progress).get ?? "Cancelled" // Progress bar with close to allow user
cancellation and finishes automatically after 2 seconds to allow
cancellation or many button presses to pop up other progress windows.
            Thread.executeOnMain {
                self.timeButtonWasPressed.stringValue = creationTime
            }
        }
    }

The above pops up a window with a progress bar in it every time the user
hits a button (buttonPushed), once the progress bar has completed or is
cancelled the main UI is updated, including noting cancellation by the
user. It can popup multiple windows and cancel them in any order.

This is easier to do with a future library than the proposed async/await
because the library has concepts of: cancellation, timeout, and control
over which queue routines run on.

Future library below:

//
// main.swift
// Future
// Version 0.1
//
// Created by Howard Lovatt on 22/8/17.
// Copyright © 2017 Howard Lovatt.
// This work is licensed under a Creative Commons Attribution 4.0
International License, http://creativecommons.org/licenses/by/4.0/.
//

import Foundation

/// - note:
/// - Written in GCD but execution service would be abstracted for a
'real' version of this proposed `Future`.
/// - It might be necessary to write an atomic class/struct and use it
for _status and isCancelled in CalculatingFuture; see comments after
property declarations.
/// - If _status and isCancelled in CalculatingFuture where atomic then
future would be thread safe.

extension Thread {
    /// Run the given closure on the main thread (thread hops to main) and
*wait* for it to complete before returning its value; useful for updating
and reading UI components.
    /// Checks to see if already executing on the main thread and if so
does not change to main thread before executing closure, since changing to
main when already on main would cause a deadlock.
    /// - note: Not unique to `Future`, hence an extension on `Thread`.
    static func executeOnMain<T>(closure: @escaping () -> T) -> T {
        var result: T?
        if Thread.isMainThread {
            result = closure()
        } else {
            DispatchQueue.main.sync {
                result = closure()
            }
        }
        return result!
    }
}

/// All possible states for a `Future`; a future is in exactly one of these.
enum FutureStatus<T> {
    /// Currently running or waiting to run; has not completed, was not
cancelled, has not timed out, and has not thrown.
    case running

    /// Ran to completion; was not cancelled, did not timeout, and did not
throw, no longer running.
    case completed(result: T)

    /// Was cancelled, timed out, or calculation threw an exception; no
longer running.
    case threw(error: Error)
}

/// An error that signals the future was cancelled.
enum CancelFuture: Error {
    /// Should be thrown by a future's calculation when requested to do so
via its `isCancelled` argument (which arises if the future is cancelled or
if the future times out).
    case cancelled
}

/// Base class for futures; acts like a future that was cancelled, i.e. no
result and threw `CancelFuture.cancelled`.
/// - note:
/// - You would normally program to `Future`, not one of its derived
classes, i.e. arguments, return types, properties, etc. typed as `Future`.
/// - Futures are **not** thread safe; i.e. they cannot be shared between
threads though their results can and they themselves can be inside any
single thread.
/// - This class is useful in its own right; not just a base class, but
as a future that is known to be cancelled.
class Future<T> {
    /// The current state of execution of the future.
    /// - note:
    /// - The status is updated when the future's calculation finishes;
therefore there will be a lag between a cancellation or a timeout and
status reflecting this.
    /// - This status lag is due to the underlying thread system provided
by the operating system that typically does not allow a running thread to
be terminated.
    /// - Because status can lag cancel and timeout; prefer get over
status, for obtaining the result of a future and if detailed reasons for a
failure are not required.
    /// - Status however offers detailed information if a thread
terminates by throwing (including cancellation and time out) and is
therefore very useful for debugging.
    /// - note: In the case of this base class, always cancelled; returns
`.threw(error: CancelFuture.cancelled)`.
    var status: FutureStatus<T> {
        return .threw(error: CancelFuture.cancelled)
    }

    /// Wait until the value of the future is calculated and return it; if
future timed out, if future was cancelled, or if calculation threw, then
return nil.
    /// The intended use of this property is to chain with the nil
coalescing operator, `??`, to provide a default, a retry, or an error
message in the case of failure.
    /// - note:
    /// - Timeout is only checked when `get` is called.
    /// - If a future is cancelled or times out then get will
subsequently return nil; however it might take some time before status
reflects this calculation because status is only updated when the
calculation stops.
    /// - note: In the case of this base class, always return nil.
    var get: T? {
        return nil
    }

    /// Cancel the calculation of the future; if it has not already
completed.
    /// - note:
    /// - Cancellation should cause `CancelFuture.cancelled` to be thrown
and hence the future's status changes to `threw` ('should' because the
calculation can ignore its `isCancelled` argument or throw some other
error).
    /// - `isCancelled` is automatically checked on entry and exit to the
calculation and therefore status will update before and after execution
even if the calculation ignores its argument.
    /// - Cancellation will not be instantaneous and therefore the
future's status will not update immediately; it updates when the
calculation terminates (either by returning a value or via a throw).
    /// - If a future timeouts, it cancels its calculation.
    /// - If the future's calculation respects its `isCancelled` argument
then a timeout will break a deadlock.
    /// - If a future is cancelled by either cancel or a timeout,
subsequent calls to `get` will return nil; even if the calculation is still
running and hence status has not updated.
    /// - note: In the case of this base class, cancel does nothing since
this future is always cancelled.
    func cancel() {}
}

/// A future that calculates its value on the given queue asynchronously
(i.e. its init method returns before the calculation is complete) and has
the given timeout to bound the wait time when `get` is called.
final class AsynchronousFuture<T>: Future<T> {
    private var _status = FutureStatus<T>.running // Really like to mark
this volatile and atomic (it is written in background thread and read in
foreground)!

    override var status: FutureStatus<T> {
        return _status
    }

    private let group = DispatchGroup()

    private let timeoutTime: DispatchTime

    private var isCancelled = false // Really like to mark this volatile
(it is a bool so presumably atomic, but it is set in forground thread and
read in background)!

    /// - note: The default queue is the global queue with default quality
of service.
    /// - note:
    /// Regarding the `timeout` argument:
    /// - Timeout starts from when the future is created, not when `get`
is called.
    /// - The time used for a timeout is processor time; i.e. it excludes
time when the computer is in sleep mode.
    /// - The default timeout is 1 second.
    /// - If the calculation times out then the calculation is cancelled.
    /// - The timeout is only checked when `get` is called; i.e. the
calculation will continue for longer than timeout, potentially
indefinitely, if `get` is not called.
    /// - Also see warning below.
    /// - warning:
    /// Be **very** careful about setting long timeouts; if a deadlock
occurs it is diagnosed/broken by a timeout occurring!
    /// If the calculating method respects its `isCancelled` argument a
timeout will break a deadlock, otherwise it will only detect a deadlock.
    init(queue: DispatchQueue = .global(), timeout: DispatchTimeInterval =
.seconds(1), calculation: @escaping (_ isCancelled: () -> Bool) ->
FutureStatus<T>) {
        self.timeoutTime = DispatchTime.now() + timeout
        super.init() // Have to complete initialization before result can
be calculated.
        queue.async { // Deliberately holds a strong reference to self, so
that a future can be side effecting.
            self.group.enter()
            defer {
                self.group.leave()
            }
            if self.isCancelled { // Future was cancelled before execution
began.
                self._status = .threw(error: CancelFuture.cancelled)
                return
            }
            self._status = calculation {
                self.isCancelled // Pass `isCancelled` to `calculation`
(via a closure so that it isn't copied and therefore reflects its current
value).
            }
            if self.isCancelled { // Future was cancelled during execution.
                self._status = .threw(error: CancelFuture.cancelled)
            }
        }
    }

    /// See above `init` for description.
    /// This `init` accepts a closure that returns a `T`; the above
`init`'s closure returns a `FutureStatus<T>`.
    /// This `init`'s closure is wrapped to return a `FutureStatus<T>` and
this `init` calls the above `init`.
    convenience init(queue: DispatchQueue = .global(), timeout:
DispatchTimeInterval = .seconds(1), calculation: @escaping (_ isCancelled:
() -> Bool) throws -> T) {
        self.init(queue: queue, timeout: timeout) { isCancelled ->
FutureStatus<T> in
            var resultOrError: FutureStatus<T>
            do {
                resultOrError = .completed(result: try
calculation(isCancelled))
            } catch {
                resultOrError = .threw(error: error)
            }
            return resultOrError
        }
    }

    /// See `init` 2 above for description.
    /// This `init` accepts a closure that accepts no arguments, unlike the
closures for the other `init`s that accept `isCancelled`, and returns a
`(T?, Error?)`; the `init`' 2 above's closure returns a `FutureStatus<T>`.
    /// This `init`'s closure is wrapped to return a `FutureStatus<T>` and
this `init` calls the `init` 2 above.
    convenience init(queue: DispatchQueue = .global(), timeout:
DispatchTimeInterval = .seconds(1), calculation: @escaping () -> (T?,
Error?)) {
        self.init(queue: queue, timeout: timeout) { _ -> FutureStatus<T> in
            var resultOrError: FutureStatus<T>
            let (result, error) = calculation()
            if error == nil {
                resultOrError = .completed(result: result!)
            } else {
                resultOrError = .threw(error: error!)
            }
            return resultOrError
        }
    }

    override var get: T? {
        guard !isCancelled else { // Catch waiting for a cancel to actually
happen.
            return nil
        }
        while true { // Loop until not running, so that after a successful
wait the result can be obtained.
            switch _status {
            case .running:
                switch group.wait(timeout: timeoutTime) { // Wait for
calculation completion.
                case .success:
                break // Loop round and test status again to extract result
                case .timedOut:
                    isCancelled = true
                    return nil
                }
            case .completed(let result):
                return result
            case .threw(_):
                return nil
            }
        }
    }

    override func cancel() {
        switch _status {
        case .running:
            isCancelled = true
        case .completed(_):
        return // Cannot cancel a completed future.
        case .threw(_):
            return // Cannot cancel a future that has timed out, been
cancelled, or thrown.
        }
    }
}

/// A future that doesn't need calculating, because the result is already
known.
final class KnownFuture<T>: Future<T> {
    private let result: T

    override var status: FutureStatus<T> {
        return .completed(result: result)
    }

    init(_ result: T) {
        self.result = result
    }

    override var get: T? {
        return result
    }
}

/// A future that doesn't need calculating, because it is known to fail.
final class FailedFuture<T>: Future<T> {
    private let _status: FutureStatus<T>

    override var status: FutureStatus<T> {
        return _status
    }

    init(_ error: Error) {
        _status = .threw(error: error)
    }
}

  -- Howard.

···

On 24 August 2017 at 14:26, Maxim Veksler via swift-evolution < swift-evolution@swift.org> wrote:

I think that the solution you are describing is how RxSwift (ReactiveX)
solves this problem.

I believe Rx, like many other higher level abstractions would benefit from
async, actors behind the scenes, as an implementation detail.

‫בתאריך יום ד׳, 23 באוג׳ 2017 ב-20:41 מאת ‪Joe Groff via swift-evolution‬‏
<‪swift-evolution@swift.org‬‏>:‬

On Aug 19, 2017, at 4:56 AM, Jakob Egger via swift-evolution < >> swift-evolution@swift.org> wrote:

I've read async/await proposal, and I'm thrilled by the possibilities.
Here's what I consider the canonical example:

@IBAction func buttonDidClick(sender:AnyObject) {
  beginAsync {
    let image = await processImage()
    imageView.image = image
  }
}

This is exactly the kind of thing I will use async/await for!

But while this example looks extremely elegant, it would suffer from a
number of problems in practice:

1. There is no guarantee that you are on the main thread after `await
processImage()`
2. There is no way to cancel processing
3. Race Condition: If you click the button a second time before
`processImage()` is done, two copies will run simultaneously and you don't
know which image will "win".

So I wondered: What would a more thorough example look like in practice?
How would I fix all these issues?

After some consideration, I came up with the following minimal example
that addresses all these issues:

class ImageProcessingTask {
  var cancelled = false
  func process() async -> Image? { … }
}

var currentTask: ImageProcessingTask?
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  beginAsync {
    guard let image = await task.process() else { return }
    DispatchQueue.main.async {
      guard task.cancelled == false else { return }
      imageView.image = image
    }
  }
}

If my example isn't obvious, I've documented my thinking (and some
alternatives) in a gist:
https://gist.github.com/jakob/22c9725caac5125c1273ece93cc2e1e7

Anyway, this more realistic code sample doesn't look nearly as nice any
more, and I actually think this could be implemented nicer without
async/await:

class ImageProcessingTask {
  var cancelled = false
  func process(completionQueue: DispatchQueue, completionHandler:
(Image?)->()) { … }
}
@IBAction func buttonDidClick(sender:AnyObject) {
  currentTask?.cancelled = true
  let task = ImageProcessingTask()
  currentTask = task
  task.process(completionQueue: DispatchQueue.main) { (image) in
    guard let image = image else { return }
guard task.cancelled == false else { return }
imageView.image = image
  }
}

So I wonder: What's the point of async/await if it doesn't result in
nicer code in practice? How can we make async/await more elegant when
calling from non-async functions?

Yeah, it's important to understand that coroutines don't directly offer
any form of coordination; they only let you thread execution nicely through
existing coordination mechanisms. IBActions by themselves don't offer any
coordination, so anything more than fire-and-forget is still going to
require explicit code. There are some interesting approaches you still
might be able to explore to make this kind of thing nicer; for instance, if
buttonDidClick didn't directly trigger the task, but instead communicated
with a coroutine via synchronous channels in the style of Go, then that
coroutine could be responsible for filtering multiple click events, and
could also listen for cancellation events. The actor model Chris proposes
in his document could conceivably let you wrap up that low-level channel
management in a nice OO-looking wrapper.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

···

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can
use the underlying threads just as well, i.e. future with async/await is no
better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout,
people should be encouraged to use this; instead because async/await are
language features they will be presumed, incorrectly, to be the best way,
consequently people will get into trouble with deadlocks because they don't
have control.

  3. async/await will require some engineering work and will at best make a
mild syntax improvement and at worst lead to deadlocks, therefore they just
don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

···

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> > wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can
build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

···

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

> My argument goes like this:
>
> 1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.
>
> 2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.
>
> 3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.
>
> Therefore, save some engineering effort and just provide a future library.
>
> To turn the question round another way, in two forms:
>
> 1. What can async/wait do that a future can't?
>
> 2. How will future be improved if async/await is added?
>
>
> -- Howard.
>
> > On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:
> > >
> > > > On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:
> > > >
> > > > In particular a future that is cancellable is more powerful that the proposed async/await.
> > >
> > > It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.
> > >
> > > -Joe
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

···

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Everything that is described in the proposal, like avoiding the pyramid of doom, simplify error handling of asynchronous calls, and so avoid many bug and bad coding practices.

···

Le 26 août 2017 à 07:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> a écrit :

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can’t?

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting
continuation-handlers into either a sync or future, i.e. it is the importer
that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource
and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created
(as soon as the underlying Future is created) and get returns an optional
(also cancel and status would be still be present). Then if you want
control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

···

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < > swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < > swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can
use the underlying threads just as well, i.e. future with async/await is no
better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout,
people should be encouraged to use this; instead because async/await are
language features they will be presumed, incorrectly, to be the best way,
consequently people will get into trouble with deadlocks because they don't
have control.

  3. async/await will require some engineering work and will at best make
a mild syntax improvement and at worst lead to deadlocks, therefore they
just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >> wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can
build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

···

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:
With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Also, as I re-read the proposal, the async/await would be implemented on top of low level co-routines instead of the implementation we’ve seen in other languages as a ‘child’ of Promises/Futures. So Futures or Promises would be a freebee instead of the opposite. Wrapping cancellation would be straightforward and independant of the coroutines, and I tend to think now that this approach is more powerful than going with async/await as sugar on top of Futures.

···

On Aug 27, 2017, 19:59 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

> The running example used in the white paper coded using a Future is:
>
> func processImageData1() -> Future<Image> {
> return AsynchronousFuture { _ -> Image in
> let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> let imageResource = loadWebResource("imagedata.dat")
> let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> }
> }
>
> This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.
>
> This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).
>
> There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.
>
> You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:
>
> func processImageData1() async -> Image {
> let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> let imageResource = loadWebResource("imagedata.dat")
> let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> }
>
> Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:
>
> func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }
>
> > On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:
> > > Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
> > > Futures can’t do that
> > >
> > > On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:
> > > > With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.
> > > >
> > > > Sent from my iPhone
> > > >
> > > > On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:
> > > >
> > > > > My argument goes like this:
> > > > >
> > > > > 1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.
> > > > >
> > > > > 2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.
> > > > >
> > > > > 3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.
> > > > >
> > > > > Therefore, save some engineering effort and just provide a future library.
> > > > >
> > > > > To turn the question round another way, in two forms:
> > > > >
> > > > > 1. What can async/wait do that a future can't?
> > > > >
> > > > > 2. How will future be improved if async/await is added?
> > > > >
> > > > >
> > > > > -- Howard.
> > > > >
> > > > > > On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:
> > > > > > >
> > > > > > > > On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:
> > > > > > > >
> > > > > > > > In particular a future that is cancellable is more powerful that the proposed async/await.
> > > > > > >
> > > > > > > It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.
> > > > > > >
> > > > > > > -Joe
> > > > >
> > > > > _______________________________________________
> > > > > swift-evolution mailing list
> > > > > swift-evolution@swift.org
> > > > > https://lists.swift.org/mailman/listinfo/swift-evolution
> --
> -- Howard.
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

The async/await is very similar to the proposed Future (as I posed earlier)
with regard to completion-handler code, they both re-write the imported
completion-handler function using a closure, the relevant sentence from the
Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel,
in the running example the following lines from the async code are run in
series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide
an async unless you can make a convincing argument that it allows you to
write a better future?

  -- Howard.

···

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it would
require more nesting and more complex error handling. By simplifying the
code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly
why async/await is so valuable. You get simple code while still doing it
correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < > swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting
continuation-handlers into either a sync or future, i.e. it is the importer
that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created
(as soon as the underlying Future is created) and get returns an optional
(also cancel and status would be still be present). Then if you want
control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> > wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can
use the underlying threads just as well, i.e. future with async/await is no
better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout,
people should be encouraged to use this; instead because async/await are
language features they will be presumed, incorrectly, to be the best way,
consequently people will get into trouble with deadlocks because they don't
have control.

  3. async/await will require some engineering work and will at best make
a mild syntax improvement and at worst lead to deadlocks, therefore they
just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>> wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You
can build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

···

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:
With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

What do you think of the idea that several of us have been proposing on another thread? In addition to being able to place ‘await' before an asynchronous function, you could instead write ‘async’ which would simply allow you to defer calling await (presumably until you need to await multiple items at once). You would be required by the compiler to call ‘await’ before being able to access the resulting value.

  let a = async longCalculationA()
  let b = async longCalculationB() //b doesn’t wait for a to complete before starting
  let c = async longCalculationC() //c doesn’t wait for a or b
  let result = await combineCalculations(a: a, b: b, c: c) //waits until a, b, and c are all available

I think this would give us 80% of the value of futures, but with added benefits:

• It allows optimizations which futures do not - One of swift’s core goals is to have the resulting program be as fast/performant as possible. The compiler has a lot of freedom here to optimize as it sees fit, because it doesn’t have to create a programmer inspectable object. The above could be implemented by the compiler behind the scenes using some sort of future, but it could also just combine the continuation blocks in a different way. It can provide different optimizations in different situations.

• It is forward compatible - Even if we just implement this by creating internal futures, because we don’t expose those details, that implementation can be changed at any time without involving evolution.

• It works with any async function - Futures have a different type, and so any function returning a future (especially in a chain of returns) must explicitly return Future<Type>. The proposal above returns the normal type (with a compiler annotation forcing await to be called before it is used). As a result ‘async’ can be used anywhere ‘await’ can be used, even in a chain of ‘async’ returns. The guarantee we are making is that the result will not be used without calling ‘await’.

• It may make actual futures more efficient - None of the points above are arguing against having futures in Swift. I want them in the standard library! With the above as a base, it should be fairly simple to build futures as a framework with all of introspection and cancellation ability that implies. Those futures should also gain some of the optimizations of that base as a result (especially when there are intermediate stages to a long calculation, etc…).

Thanks,
Jon

···

On Aug 27, 2017, at 7:22 PM, Adam Kemp via swift-evolution <swift-evolution@swift.org> wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

···

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

> The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:
>
> > quote_type
> > "Under the hood, the compiler rewrites this code using nested closures ..."
>
> Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:
>
> let dataResource = await loadWebResource("dataprofile.txt")
> let imageResource = await loadWebResource("imagedata.dat")
> The equivalent lines using the proposed Future:
> let dataResource = loadWebResource("dataprofile.txt")
> let imageResource = loadWebResource("imagedata.dat")
> Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.
>
> Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?
>
> -- Howard.
>
> > On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:
> > > This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.
> > >
> > > I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.
> > >
> > > Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).
> > >
> > > That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.
> > >
> > > --
> > > Adam Kemp
> > >
> > > On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:
> > >
> > > > The running example used in the white paper coded using a Future is:
> > > >
> > > > func processImageData1() -> Future<Image> {
> > > > return AsynchronousFuture { _ -> Image in
> > > > let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> > > > let imageResource = loadWebResource("imagedata.dat")
> > > > let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> > > > let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> > > > return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> > > > }
> > > > }
> > > >
> > > > This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.
> > > >
> > > > This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).
> > > >
> > > > There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.
> > > >
> > > > You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:
> > > >
> > > > func processImageData1() async -> Image {
> > > > let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> > > > let imageResource = loadWebResource("imagedata.dat")
> > > > let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> > > > let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> > > > return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> > > > }
> > > >
> > > > Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:
> > > >
> > > > func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }
> > > >
> > > > > On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:
> > > > > > Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
> > > > > > Futures can’t do that
> > > > > >
> > > > > > On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:
> > > > > > > With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.
> > > > > > >
> > > > > > > Sent from my iPhone
> > > > > > >
> > > > > > > On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:
> > > > > > >
> > > > > > > > My argument goes like this:
> > > > > > > >
> > > > > > > > 1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.
> > > > > > > >
> > > > > > > > 2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.
> > > > > > > >
> > > > > > > > 3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.
> > > > > > > >
> > > > > > > > Therefore, save some engineering effort and just provide a future library.
> > > > > > > >
> > > > > > > > To turn the question round another way, in two forms:
> > > > > > > >
> > > > > > > > 1. What can async/wait do that a future can't?
> > > > > > > >
> > > > > > > > 2. How will future be improved if async/await is added?
> > > > > > > >
> > > > > > > >
> > > > > > > > -- Howard.
> > > > > > > >
> > > > > > > > > On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > In particular a future that is cancellable is more powerful that the proposed async/await.
> > > > > > > > > >
> > > > > > > > > > It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.
> > > > > > > > > >
> > > > > > > > > > -Joe
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > swift-evolution mailing list
> > > > > > > > swift-evolution@swift.org
> > > > > > > > https://lists.swift.org/mailman/listinfo/swift-evolution
> > > > --
> > > > -- Howard.
> > > > _______________________________________________
> > > > swift-evolution mailing list
> > > > swift-evolution@swift.org
> > > > https://lists.swift.org/mailman/listinfo/swift-evolution
>

I think the biggest tradeoff is clearer when you look at the examples from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource.get(), imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future. Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(await dataResourceFuture, await imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself can be used with either await or as a future. You get both options with one implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one particular futures implementation. The Task type is commonly used, but async/await does not directly depend on Task. Instead it works with any return type that meets certain requirements (detailed here: https://blogs.msdn.microsoft.com/pfxteam/2011/01/13/await-anything/). Swift could do this using a protocol, which can be retroactively applied using an extension.

Obviously for this to be useful we would need some kind of existing future implementation, but at least we wouldn’t be tied to any particular one. That would mean library maintainers who have already been using their own futures implementations could quickly adopt async/await in their code without having to rewrite their futures library or throw wrappers around every usage of async/await. They could just adopt a protocol (using an extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await use case rather than a generic coroutine implementation (i.e., there would have to be a separate compiler transform for yield return). It’s not clear to me why it should be a goal to have just one generic coroutine feature. The real-world usages of async/await and yield return are different enough that I’m not convinced we could have a single compiler feature that meets the needs of both cleanly.

···

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com> wrote:

Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

One of the biggest incumbents in this space on the server side is Java and
its concurrency is based on futures and works very well (though there are a
lot of libraries built on top of basic futures). Same goes for the Akka
library mentioned in the whitepaper, that is built on top of Scala's
futures. Maybe the bad experiance in C# and JS with futures is more to do
with the implementation of futures in these languages rather than with
futures themselves?

  -- Howard.

···

On 28 August 2017 at 12:35, Florent Vilmart <florent@flovilmart.com> wrote:

Adam, you’re completely right, languages as c# and JS have been through
the path before, (callback, Promises , async/await) I believe Chris’s goal
it to avoid building a promise implementation and go straight to a
coroutines model, which is more deeply integrated with the compiler. I
don’t see a particular trade off, pursuing that route, and the main benefit
is that coroutines can power any asynchronous metaphor (Signals, Streams,
Futures, Promises etc...) which is not true of Futures so i would tend to
think that for the long run, and to maximize usability, async/await/yield
would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or the
other way around). You can have the best of both worlds. We are not losing
anything by having this feature. It would be a huge improvement to have
this as an option.

However, using futures correctly requires more nested closures than you
have shown in your examples to avoid blocking any threads. That's why
you're not seeing the advantage to async/await. You're comparing examples
that have very different behaviors.

That said, I have also expressed my opinion that it is better to build
async/await on top of futures rather than the other way around. I believe
it is more powerful and cleaner to make async/await work with any arbitrary
future type (via a protocol). The alternative (building futures on top of
async/await) requires more code when the two are mixed. I very much prefer
how it's done in C#, where you can freely mix the two models without having
to resort to ad-hoc wrappers, and you can use async/await with any futures
implementation you might already be using.

I really think we should be having more discussion about the tradeoffs
between those two approaches, and I'm concerned that some of the opinions
about how C# does it are not based on a clear and accurate understanding of
how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> > wrote:

The async/await is very similar to the proposed Future (as I posed
earlier) with regard to completion-handler code, they both re-write the
imported completion-handler function using a closure, the relevant sentence
from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel,
in the running example the following lines from the async code are run in
series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide
an async unless you can make a convincing argument that it allows you to
write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it would
require more nesting and more complex error handling. By simplifying the
code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly
why async/await is so valuable. You get simple code while still doing it
correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < >> swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ??
Resource(path: "Default data resource or prompt user"), imageResource.get
?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or
prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by
converting continuation-handlers into either a sync or future, i.e. it is
the importer that eliminates the nesting by translating the code
automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is
created (as soon as the underlying Future is created) and get returns an
optional (also cancel and status would be still be present). Then if you
want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> >> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >>> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can
use the underlying threads just as well, i.e. future with async/await is no
better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout,
people should be encouraged to use this; instead because async/await are
language features they will be presumed, incorrectly, to be the best way,
consequently people will get into trouble with deadlocks because they don't
have control.

  3. async/await will require some engineering work and will at best
make a mild syntax improvement and at worst lead to deadlocks, therefore
they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future
library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>>> wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You
can build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Yes I do think the async keyword to start execution without waiting is an
improvement, but would still like to have cancel, thread control, timeout,
and improved debugging.

  -- Howard.

···

On 28 August 2017 at 13:21, Jonathan Hull <jhull@gbis.com> wrote:

What do you think of the idea that several of us have been proposing on
another thread? In addition to being able to place ‘await' before an
asynchronous function, you could instead write ‘async’ which would simply
allow you to defer calling await (presumably until you need to await
multiple items at once). You would be required by the compiler to call
‘await’ before being able to access the resulting value.

let a = async longCalculationA()
let b = async longCalculationB() //b doesn’t wait for a to complete before
starting
let c = async longCalculationC() //c doesn’t wait for a or b
let result = await combineCalculations(a: a, b: b, c: c) //waits until a,
b, and c are all available

I think this would give us 80% of the value of futures, but with added
benefits:

• *It* *allows* *optimizations which futures do not* - One of swift’s
core goals is to have the resulting program be as fast/performant as
possible. The compiler has a lot of freedom here to optimize as it sees
fit, because it doesn’t have to create a programmer inspectable object.
The above could be implemented by the compiler behind the scenes using some
sort of future, but it could also just combine the continuation blocks in a
different way. It can provide different optimizations in different
situations.

• *It is forward compatible* - Even if we just implement this by creating
internal futures, because we don’t expose those details, that
implementation can be changed at any time without involving evolution.

• *It works with any async function* - Futures have a different type, and
so any function returning a future (especially in a chain of returns) must
explicitly return Future<Type>. The proposal above returns the normal type
(with a compiler annotation forcing await to be called before it is used).
As a result ‘async’ can be used anywhere ‘await’ can be used, even in a
chain of ‘async’ returns. The guarantee we are making is that the result
will not be used without calling ‘await’.

• *It may make actual futures more efficient* - None of the points above
are arguing against having futures in Swift. I want them in the standard
library! With the above as a base, it should be fairly simple to build
futures as a framework with all of introspection and cancellation ability
that implies. Those futures should also gain some of the optimizations of
that base as a result (especially when there are intermediate stages to a
long calculation, etc…).

Thanks,
Jon

On Aug 27, 2017, at 7:22 PM, Adam Kemp via swift-evolution < > swift-evolution@swift.org> wrote:

As has been explained, futures can be built on top of async/await (or the
other way around). You can have the best of both worlds. We are not losing
anything by having this feature. It would be a huge improvement to have
this as an option.

However, using futures correctly requires more nested closures than you
have shown in your examples to avoid blocking any threads. That's why
you're not seeing the advantage to async/await. You're comparing examples
that have very different behaviors.

That said, I have also expressed my opinion that it is better to build
async/await on top of futures rather than the other way around. I believe
it is more powerful and cleaner to make async/await work with any arbitrary
future type (via a protocol). The alternative (building futures on top of
async/await) requires more code when the two are mixed. I very much prefer
how it's done in C#, where you can freely mix the two models without having
to resort to ad-hoc wrappers, and you can use async/await with any futures
implementation you might already be using.

I really think we should be having more discussion about the tradeoffs
between those two approaches, and I'm concerned that some of the opinions
about how C# does it are not based on a clear and accurate understanding of
how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> > wrote:

The async/await is very similar to the proposed Future (as I posed
earlier) with regard to completion-handler code, they both re-write the
imported completion-handler function using a closure, the relevant sentence
from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel,
in the running example the following lines from the async code are run in
series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide
an async unless you can make a convincing argument that it allows you to
write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it would
require more nesting and more complex error handling. By simplifying the
code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly
why async/await is so valuable. You get simple code while still doing it
correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < >> swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ??
Resource(path: "Default data resource or prompt user"), imageResource.get
?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or
prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by
converting continuation-handlers into either a sync or future, i.e. it is
the importer that eliminates the nesting by translating the code
automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is
created (as soon as the underlying Future is created) and get returns an
optional (also cancel and status would be still be present). Then if you
want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> >> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >>> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can
use the underlying threads just as well, i.e. future with async/await is no
better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout,
people should be encouraged to use this; instead because async/await are
language features they will be presumed, incorrectly, to be the best way,
consequently people will get into trouble with deadlocks because they don't
have control.

  3. async/await will require some engineering work and will at best
make a mild syntax improvement and at worst lead to deadlocks, therefore
they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future
library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>>> wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You
can build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Terms of Service

Privacy Policy

Cookie Policy