[Concurrency] Fixing race conditions in async/await example

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

On real systems going wide concurrently (and I will not call this parallelism on purpose, parallelism is about doing the same compute on different CPUs) is not often a win provided the thing you're doing is complex enough.
The reason for it is that most of these subsystems, and loadWebResource is a perfect example of this, use constrained resources that use locks and the like.

In practice, you would really want all resources that you load from the internet to be handled from the same execution context (whether it's a dispatch queue, a thread or a CFRunloop is absolutely not relevant), because it's very likely that you'll end up contending concurrent executions of these loads through locks and the various shared resources in use (like your NIC and the networking stack).

We've learned over the last 10 years of working on a massively concurrent system because of libdispatch making it too easy (some will say thanks to, but these aren't the one who need to cope with that it does to the OS :P), is that going wide by default is a design mistake and is very difficult to optimize when it goes bad.

The right design is to have silos of execution contexts that are very few, as few as you can, and parallelize (and here I mean parallelize) the operations that benefit from using several compute units when you have proved that you do scale linearly (and scaling linearly is VERY hard, even for low values of NCPU as soon as what you do is complex enough)

To me (but I'm catching up with the thread and maybe there have been refinements in that regard already), the loseness of the "execution context" that will run your completion handler here is a liability for the implementation of the runtime. Go for example is not doing that well, for the longest time the environment they recommended had a very small amount of physical threads because of scalability concerns around their goroutines scheduler. I'm mentioning go as an example of a language which is implementing CSP as a core concept, and has had (I'm not quite sure where they stand these days) quite a lot of trouble scaling the implementation.

IMSHO dispatch_get_global_queue() is in practice on of the worst thing that the dispatch API provides, because despite all the best efforts of the runtime, there aren't enough information at runtime about your operations/actors/... to understand what your intent is and optimize for it. Which means that the language should make sure that (1) the anti-pattern is not the default behavior and (2) the interface provides a way to give and propagate the hints (dependency relationships, ordering, execution context, priorities, ...) the runtime will potentially need upfront.

···

On Aug 27, 2017, at 6:02 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I can see where you're going with that.
For me, the big unknown (and unspecified so far) is

func process() async -> Image { ... }

let result = process() // what is result? CoroutineType<Image> ?

or am I missing something here?

···

On 28 août 2017 16:07 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

I think the biggest tradeoff is clearer when you look at the examples from the proposal where futures are built on top of async/await:

> func processImageData1a() async -> Image {
> let dataResource = Future { await loadWebResource("dataprofile.txt") }
> let imageResource = Future { await loadWebResource("imagedata.dat") }
>
> // ... other stuff can go here to cover load latency...
>
> let imageTmp = await decodeImage(dataResource.get(), imageResource.get())
> let imageResult = await dewarpAndCleanupImage(imageTmp)
> return imageResult
> }

With this approach you have to wrap each call site to create a future. Compare to this:

> func processImageData1a() -> Future<Image> {
> let dataResourceFuture = loadWebResource("dataprofile.txt”);
> let imageResourceFuture = loadWebResource("imagedata.dat”);
>
> // ... other stuff can go here to cover load latency...
>
> let imageTmp = await decodeImage(await dataResourceFuture, await imageResourceFuture)
> let imageResult = await dewarpAndCleanupImage(imageTmp)
> return imageResult
> }

Here, not only are the explicit wrappers gone, but this function itself can be used with either await or as a future. You get both options with one implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one particular futures implementation. The Task type is commonly used, but async/await does not directly depend on Task. Instead it works with any return type that meets certain requirements (detailed here: await anything; - .NET Parallel Programming). Swift could do this using a protocol, which can be retroactively applied using an extension.

Obviously for this to be useful we would need some kind of existing future implementation, but at least we wouldn’t be tied to any particular one. That would mean library maintainers who have already been using their own futures implementations could quickly adopt async/await in their code without having to rewrite their futures library or throw wrappers around every usage of async/await. They could just adopt a protocol (using an extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await use case rather than a generic coroutine implementation (i.e., there would have to be a separate compiler transform for yield return). It’s not clear to me why it should be a goal to have just one generic coroutine feature. The real-world usages of async/await and yield return are different enough that I’m not convinced we could have a single compiler feature that meets the needs of both cleanly.

> On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com> wrote:
>
> Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.
>
> On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:
> > As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.
> >
> > However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.
> >
> > That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.
> >
> > I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.
> >
> > --
> > Adam Kemp
> >
> > On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> wrote:
> >
> > > The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:
> > >
> > > > "Under the hood, the compiler rewrites this code using nested closures ..."
> > >
> > > Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:
> > >
> > > let dataResource = await loadWebResource("dataprofile.txt")
> > > let imageResource = await loadWebResource("imagedata.dat")
> > > The equivalent lines using the proposed Future:
> > > let dataResource = loadWebResource("dataprofile.txt")
> > > let imageResource = loadWebResource("imagedata.dat")
> > > Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.
> > >
> > > Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?
> > >
> > > -- Howard.
> > >
> > > > On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:
> > > > > This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.
> > > > >
> > > > > I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.
> > > > >
> > > > > Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).
> > > > >
> > > > > That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.
> > > > >
> > > > > --
> > > > > Adam Kemp
> > > > >
> > > > > On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:
> > > > >
> > > > > > The running example used in the white paper coded using a Future is:
> > > > > >
> > > > > > func processImageData1() -> Future<Image> {
> > > > > > return AsynchronousFuture { _ -> Image in
> > > > > > let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> > > > > > let imageResource = loadWebResource("imagedata.dat")
> > > > > > let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> > > > > > let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> > > > > > return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.
> > > > > >
> > > > > > This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).
> > > > > >
> > > > > > There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.
> > > > > >
> > > > > > You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:
> > > > > >
> > > > > > func processImageData1() async -> Image {
> > > > > > let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> > > > > > let imageResource = loadWebResource("imagedata.dat")
> > > > > > let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> > > > > > let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> > > > > > return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> > > > > > }
> > > > > >
> > > > > > Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:
> > > > > >
> > > > > > func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }
> > > > > >
> > > > > > > On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:
> > > > > > > > Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
> > > > > > > > Futures can’t do that
> > > > > > > >
> > > > > > > > On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:
> > > > > > > > > With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.
> > > > > > > > >
> > > > > > > > > Sent from my iPhone
> > > > > > > > >
> > > > > > > > > On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:
> > > > > > > > >
> > > > > > > > > > My argument goes like this:
> > > > > > > > > >
> > > > > > > > > > 1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.
> > > > > > > > > >
> > > > > > > > > > 2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.
> > > > > > > > > >
> > > > > > > > > > 3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.
> > > > > > > > > >
> > > > > > > > > > Therefore, save some engineering effort and just provide a future library.
> > > > > > > > > >
> > > > > > > > > > To turn the question round another way, in two forms:
> > > > > > > > > >
> > > > > > > > > > 1. What can async/wait do that a future can't?
> > > > > > > > > >
> > > > > > > > > > 2. How will future be improved if async/await is added?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > -- Howard.
> > > > > > > > > >
> > > > > > > > > > > On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > In particular a future that is cancellable is more powerful that the proposed async/await.
> > > > > > > > > > > >
> > > > > > > > > > > > It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.
> > > > > > > > > > > >
> > > > > > > > > > > > -Joe
> > > > > > > > > >
> > > > > > > > > > _______________________________________________
> > > > > > > > > > swift-evolution mailing list
> > > > > > > > > > swift-evolution@swift.org
> > > > > > > > > > https://lists.swift.org/mailman/listinfo/swift-evolution
> > > > > > --
> > > > > > -- Howard.
> > > > > > _______________________________________________
> > > > > > swift-evolution mailing list
> > > > > > swift-evolution@swift.org
> > > > > > https://lists.swift.org/mailman/listinfo/swift-evolution
> > >

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No future
type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource, imageResource) //
Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle with
one more type.

···

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution < swift-evolution@swift.org> escreveu:

I think the biggest tradeoff is clearer when you look at the examples from
the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource.get(),
imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future.
Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(await dataResourceFuture, await
imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself
can be used with either await or as a future. You get both options with one
implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one
particular futures implementation. The Task type is commonly used, but
async/await does not directly depend on Task. Instead it works with any
return type that meets certain requirements (detailed here:
await anything; - .NET Parallel Programming).
Swift could do this using a protocol, which can be retroactively applied
using an extension.

Obviously for this to be useful we would need some kind of existing future
implementation, but at least we wouldn’t be tied to any particular one.
That would mean library maintainers who have already been using their own
futures implementations could quickly adopt async/await in their code
without having to rewrite their futures library or throw wrappers around
every usage of async/await. They could just adopt a protocol (using an
extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await use
case rather than a generic coroutine implementation (i.e., there would have
to be a separate compiler transform for yield return). It’s not clear to me
why it should be a goal to have just one generic coroutine feature. The
real-world usages of async/await and yield return are different enough that
I’m not convinced we could have a single compiler feature that meets the
needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com> > wrote:

Adam, you’re completely right, languages as c# and JS have been through
the path before, (callback, Promises , async/await) I believe Chris’s goal
it to avoid building a promise implementation and go straight to a
coroutines model, which is more deeply integrated with the compiler. I
don’t see a particular trade off, pursuing that route, and the main benefit
is that coroutines can power any asynchronous metaphor (Signals, Streams,
Futures, Promises etc...) which is not true of Futures so i would tend to
think that for the long run, and to maximize usability, async/await/yield
would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or the
other way around). You can have the best of both worlds. We are not losing
anything by having this feature. It would be a huge improvement to have
this as an option.

However, using futures correctly requires more nested closures than you
have shown in your examples to avoid blocking any threads. That's why
you're not seeing the advantage to async/await. You're comparing examples
that have very different behaviors.

That said, I have also expressed my opinion that it is better to build
async/await on top of futures rather than the other way around. I believe
it is more powerful and cleaner to make async/await work with any arbitrary
future type (via a protocol). The alternative (building futures on top of
async/await) requires more code when the two are mixed. I very much prefer
how it's done in C#, where you can freely mix the two models without having
to resort to ad-hoc wrappers, and you can use async/await with any futures
implementation you might already be using.

I really think we should be having more discussion about the tradeoffs
between those two approaches, and I'm concerned that some of the opinions
about how C# does it are not based on a clear and accurate understanding of
how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> > wrote:

The async/await is very similar to the proposed Future (as I posed
earlier) with regard to completion-handler code, they both re-write the
imported completion-handler function using a closure, the relevant sentence
from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel,
in the running example the following lines from the async code are run in
series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide
an async unless you can make a convincing argument that it allows you to
write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it would
require more nesting and more complex error handling. By simplifying the
code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly
why async/await is so valuable. You get simple code while still doing it
correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < >> swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ??
Resource(path: "Default data resource or prompt user"), imageResource.get
?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or
prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by
converting continuation-handlers into either a sync or future, i.e. it is
the importer that eliminates the nesting by translating the code
automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is
created (as soon as the underlying Future is created) and get returns an
optional (also cancel and status would be still be present). Then if you
want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> >> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >>> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can
use the underlying threads just as well, i.e. future with async/await is no
better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout,
people should be encouraged to use this; instead because async/await are
language features they will be presumed, incorrectly, to be the best way,
consequently people will get into trouble with deadlocks because they don't
have control.

  3. async/await will require some engineering work and will at best
make a mild syntax improvement and at worst lead to deadlocks, therefore
they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future
library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>>> wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You
can build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

One of the biggest incumbents in this space on the server side is Java and its concurrency is based on futures and works very well (though there are a lot of libraries built on top of basic futures).

Most server side libraries don’t use Java Future as they force blocking at some point to get the future result. They instead have there own implementation that provide async completion handler (ListenableFuture, …), which result in the pattern we are trying to avoid with coroutine and async/await. This is not a very good example.

···

Le 28 août 2017 à 06:14, Howard Lovatt via swift-evolution <swift-evolution@swift.org> a écrit :

On 28 August 2017 at 12:35, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>>, wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Thank you Pierre for writing this. I wanted to say this for a while, but I am too busy these days to do it myself.

Guys, please don’t miss what he has to say:

···

On Aug 31, 2017, at 5:49 PM, Pierre Habouzit via swift-evolution <swift-evolution@swift.org> wrote:

On real systems going wide concurrently (and I will not call this parallelism on purpose, parallelism is about doing the same compute on different CPUs) is not often a win provided the thing you're doing is complex enough.
The reason for it is that most of these subsystems, and loadWebResource is a perfect example of this, use constrained resources that use locks and the like.

In practice, you would really want all resources that you load from the internet to be handled from the same execution context (whether it's a dispatch queue, a thread or a CFRunloop is absolutely not relevant), because it's very likely that you'll end up contending concurrent executions of these loads through locks and the various shared resources in use (like your NIC and the networking stack).

We've learned over the last 10 years of working on a massively concurrent system because of libdispatch making it too easy (some will say thanks to, but these aren't the one who need to cope with that it does to the OS :P), is that going wide by default is a design mistake and is very difficult to optimize when it goes bad.

The right design is to have silos of execution contexts that are very few, as few as you can, and parallelize (and here I mean parallelize) the operations that benefit from using several compute units when you have proved that you do scale linearly (and scaling linearly is VERY hard, even for low values of NCPU as soon as what you do is complex enough)

To me (but I'm catching up with the thread and maybe there have been refinements in that regard already), the loseness of the "execution context" that will run your completion handler here is a liability for the implementation of the runtime. Go for example is not doing that well, for the longest time the environment they recommended had a very small amount of physical threads because of scalability concerns around their goroutines scheduler. I'm mentioning go as an example of a language which is implementing CSP as a core concept, and has had (I'm not quite sure where they stand these days) quite a lot of trouble scaling the implementation.

IMSHO dispatch_get_global_queue() is in practice on of the worst thing that the dispatch API provides, because despite all the best efforts of the runtime, there aren't enough information at runtime about your operations/actors/... to understand what your intent is and optimize for it. Which means that the language should make sure that (1) the anti-pattern is not the default behavior and (2) the interface provides a way to give and propagate the hints (dependency relationships, ordering, execution context, priorities, ...) the runtime will potentially need upfront.

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

In practice, you would really want all resources that you load from the internet to be handled from the same execution context (whether it's a dispatch queue, a thread or a CFRunloop is absolutely not relevant), because it's very likely that you'll end up contending concurrent executions of these loads through locks and the various shared resources in use (like your NIC and the networking stack).

I completely agree with this point, and this is one of the nice things about the model as described: It allows the programmer to describe concurrent abstractions that make sense for their app, WITHOUT being bound to how the async operations themselves are implemented. I don’t expect the whole world to be reimplemented, but if it was, I’d expect that there would end up being one “network card actor” for each physical nic that is present. No matter how many concurrent requests are started by clients, they’d be serially enqueued on the actors queue, and the requests would presumably be serviced in parallel using efficient shared mutable state WITHIN the network card actor.

On real systems going wide concurrently is not often a win provided the thing you're doing is complex enough.
The reason for it is that most of these subsystems, and loadWebResource is a perfect example of this, use constrained resources that use locks and the like.

I agree with you if you’re talking about a single machine, but if you’re talking about distributing across a cluster of a thousand machines, then I disagree.

-Chris

···

On Aug 31, 2017, at 5:48 PM, Pierre Habouzit via swift-evolution <swift-evolution@swift.org> wrote:

By the rules of the proposal I think this is a compiler error.

In C# the async keyword does not affect the caller at all. The above example would be like this (using C# syntax):

Task<Image> Process();

Task<Image> result1 = Process();
Image result2 = await Process();

Some people dislike this behavior. If you forget the await keyword you get the future type instead of the actual result. That’s a little bit confusing.

However, you would only be confused until you try to compile your code and find that the type isn’t what you think it is. I can’t think of a scenario in which you would actually be able to compile the code and get the wrong result and have trouble understanding why. You would just get an error from the compiler when trying to access properties that don’t exist (or, more likely, notice that code completion is not giving you what you expect).

Personally I think this is a feature, not a bug. This is what allows you to easily mix async/await with futures.

···

On Aug 28, 2017, at 1:14 PM, Florent Vilmart <florent@flovilmart.com> wrote:

I can see where you're going with that.
For me, the big unknown (and unspecified so far) is

func process() async -> Image { ... }

let result = process() // what is result? CoroutineType<Image> ?

or am I missing something here?

How would these anonymous types get composed? If I wanted to implement a function that takes a collection of futures and wait on it, how would I do that? That is, how would I implement the equivalent of C#’s Task.WhenAll and Task.WhenAny methods?

More generally, how do you pass one of these typeless futures to some other function so that we can do the waiting there?

···

On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf@gmail.com> wrote:

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No future type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource, imageResource) // Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle with one more type.

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> escreveu:
I think the biggest tradeoff is clearer when you look at the examples from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource.get(), imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future. Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(await dataResourceFuture, await imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself can be used with either await or as a future. You get both options with one implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one particular futures implementation. The Task type is commonly used, but async/await does not directly depend on Task. Instead it works with any return type that meets certain requirements (detailed here: await anything; - .NET Parallel Programming). Swift could do this using a protocol, which can be retroactively applied using an extension.

Obviously for this to be useful we would need some kind of existing future implementation, but at least we wouldn’t be tied to any particular one. That would mean library maintainers who have already been using their own futures implementations could quickly adopt async/await in their code without having to rewrite their futures library or throw wrappers around every usage of async/await. They could just adopt a protocol (using an extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await use case rather than a generic coroutine implementation (i.e., there would have to be a separate compiler transform for yield return). It’s not clear to me why it should be a goal to have just one generic coroutine feature. The real-world usages of async/await and yield return are different enough that I’m not convinced we could have a single compiler feature that meets the needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:

Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>>, wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

We don't need to this now!

Again: (Using proposal words)

"It is important to understand that this is proposing compiler support that
is completely concurrency runtime-agnostic. This proposal does not include
a new runtime model (like "actors") - it works just as well with GCD as
with pthreads or another API. Furthermore, unlike designs in other
languages, it is independent of specific coordination mechanisms, such as
futures or channels, allowing these to be built as library feature"

and

"This proposal does not formally propose a Future type, or any other
coordination abstractions. There are many rational designs for futures, and
a lot of experience working with them. On the other hand, there are also
completely different coordination primitives that can be used with this
coroutine design, and incorporating them into this proposal only makes it
larger."

and

We focus on task-based concurrency abstractions commonly encountered in
client and server applications, particularly those that are highly event
driven (e.g. responding to UI events or requests from clients). This does
not attempt to be a comprehensive survey of all possible options, nor does
it attempt to solve all possible problems in the space of concurrency.
Instead, it outlines a single coherent design thread that can be built over
the span of years to incrementally drive Swift to further greatness.

and

This proposal has been kept intentionally minimal, but there are many
possible ways to expand this in the future.

....

The point is: No Future type is indeed proposed yet!

The proposal try to include de "minimum" required to implement a basic
async/await to solve the problem created by the GCD! (Pyramid of doom)

The question is: How do you do the same using dispatch_async ?
dispatch_async also does not return nothing to do what you are intentend do
do!

Algo, by Swift 5 manifesto, there's no compromise to make a "complete"
concurrency model by this time!

My intention is only make parity to dispatch_async, but also make the
ground free to make more complex implementation like Futures in another
round on top of this one.

This 'async T' can be a real type in the future? Maybe will... But doesn't
matter now! Now we only need to is some kind of type which need to be
unwrapped using await before use. Maybe this intermediary/virtual type can
be a real thing and gain some abilities at some point! Maybe a full Future
type, why not?

···

Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp@apple.com> escreveu:

How would these anonymous types get composed? If I wanted to implement a
function that takes a collection of futures and wait on it, how would I do
that? That is, how would I implement the equivalent of C#’s Task.WhenAll
and Task.WhenAny methods?

More generally, how do you pass one of these typeless futures to some
other function so that we can do the waiting there?

On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf@gmail.com> wrote:

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No
future type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource, imageResource) //
Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle with
one more type.

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution < > swift-evolution@swift.org> escreveu:

I think the biggest tradeoff is clearer when you look at the examples
from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource.get(),
imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future.
Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(await dataResourceFuture, await
imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself
can be used with either await or as a future. You get both options with one
implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one
particular futures implementation. The Task type is commonly used, but
async/await does not directly depend on Task. Instead it works with any
return type that meets certain requirements (detailed here:
await anything; - .NET Parallel Programming).
Swift could do this using a protocol, which can be retroactively applied
using an extension.

Obviously for this to be useful we would need some kind of existing
future implementation, but at least we wouldn’t be tied to any particular
one. That would mean library maintainers who have already been using their
own futures implementations could quickly adopt async/await in their code
without having to rewrite their futures library or throw wrappers around
every usage of async/await. They could just adopt a protocol (using an
extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await
use case rather than a generic coroutine implementation (i.e., there would
have to be a separate compiler transform for yield return). It’s not clear
to me why it should be a goal to have just one generic coroutine feature.
The real-world usages of async/await and yield return are different enough
that I’m not convinced we could have a single compiler feature that meets
the needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com> >> wrote:

Adam, you’re completely right, languages as c# and JS have been through
the path before, (callback, Promises , async/await) I believe Chris’s goal
it to avoid building a promise implementation and go straight to a
coroutines model, which is more deeply integrated with the compiler. I
don’t see a particular trade off, pursuing that route, and the main benefit
is that coroutines can power any asynchronous metaphor (Signals, Streams,
Futures, Promises etc...) which is not true of Futures so i would tend to
think that for the long run, and to maximize usability, async/await/yield
would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or the
other way around). You can have the best of both worlds. We are not losing
anything by having this feature. It would be a huge improvement to have
this as an option.

However, using futures correctly requires more nested closures than you
have shown in your examples to avoid blocking any threads. That's why
you're not seeing the advantage to async/await. You're comparing examples
that have very different behaviors.

That said, I have also expressed my opinion that it is better to build
async/await on top of futures rather than the other way around. I believe
it is more powerful and cleaner to make async/await work with any arbitrary
future type (via a protocol). The alternative (building futures on top of
async/await) requires more code when the two are mixed. I very much prefer
how it's done in C#, where you can freely mix the two models without having
to resort to ad-hoc wrappers, and you can use async/await with any futures
implementation you might already be using.

I really think we should be having more discussion about the tradeoffs
between those two approaches, and I'm concerned that some of the opinions
about how C# does it are not based on a clear and accurate understanding of
how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> >> wrote:

The async/await is very similar to the proposed Future (as I posed
earlier) with regard to completion-handler code, they both re-write the
imported completion-handler function using a closure, the relevant sentence
from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures
..."

Unlike the proposed future code the async code is not naturally parallel,
in the running example the following lines from the async code are run in
series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why
provide an async unless you can make a convincing argument that it allows
you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it
would require more nesting and more complex error handling. By simplifying
the code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is
exactly why async/await is so valuable. You get simple code while still
doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ??
Resource(path: "Default data resource or prompt user"), imageResource.get
?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or
prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by
converting continuation-handlers into either a sync or future, i.e. it is
the importer that eliminates the nesting by translating the code
automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is
created (as soon as the underlying Future is created) and get returns an
optional (also cancel and status would be still be present). Then if you
want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> >>> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >>>> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >>>> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you
can use the underlying threads just as well, i.e. future with async/await
is no better than future without.

  2. Since future is more powerful, thread control, cancel, and
timeout, people should be encouraged to use this; instead because
async/await are language features they will be presumed, incorrectly, to be
the best way, consequently people will get into trouble with deadlocks
because they don't have control.

  3. async/await will require some engineering work and will at best
make a mild syntax improvement and at worst lead to deadlocks, therefore
they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future
library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>>>> wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You
can build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

If you are asking about how ‘async’ (which defers ‘await’) composes, it actually composes completely naturally… and I believe that should be provable.

Any result from using ‘async’ has to have ‘await’ called on it before it can be used. The key question is: what should we do if it is returned from an async function without having ‘await’ called on it? Well, the function itself had to be called with either ‘await’ or ‘async’. If it was called with await, then we are essentially calling await on the return value at that point. If it was called with ‘async’, then the result will be marked with the same compiler flag which forces ‘await’ to be called on it before it can be used. We see that it’s state has been preserved through the return. As a result we can compose things completely naturally (without any additional rules/requirements beyond what we would have to do anyway).

Thanks,
Jon

···

On Aug 28, 2017, at 1:33 PM, Adam Kemp via swift-evolution <swift-evolution@swift.org> wrote:

How would these anonymous types get composed? If I wanted to implement a function that takes a collection of futures and wait on it, how would I do that? That is, how would I implement the equivalent of C#’s Task.WhenAll and Task.WhenAny methods?

More generally, how do you pass one of these typeless futures to some other function so that we can do the waiting there?

On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf@gmail.com <mailto:wallacyf@gmail.com>> wrote:

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No future type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource, imageResource) // Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle with one more type.

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> escreveu:
I think the biggest tradeoff is clearer when you look at the examples from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource.get(), imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future. Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(await dataResourceFuture, await imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself can be used with either await or as a future. You get both options with one implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one particular futures implementation. The Task type is commonly used, but async/await does not directly depend on Task. Instead it works with any return type that meets certain requirements (detailed here: await anything; - .NET Parallel Programming). Swift could do this using a protocol, which can be retroactively applied using an extension.

Obviously for this to be useful we would need some kind of existing future implementation, but at least we wouldn’t be tied to any particular one. That would mean library maintainers who have already been using their own futures implementations could quickly adopt async/await in their code without having to rewrite their futures library or throw wrappers around every usage of async/await. They could just adopt a protocol (using an extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await use case rather than a generic coroutine implementation (i.e., there would have to be a separate compiler transform for yield return). It’s not clear to me why it should be a goal to have just one generic coroutine feature. The real-world usages of async/await and yield return are different enough that I’m not convinced we could have a single compiler feature that meets the needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:

Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>>, wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

One of the biggest incumbents in this space on the server side is Java and its concurrency is based on futures and works very well (though there are a lot of libraries built on top of basic futures).

Most server side libraries don’t use Java Future as they force blocking at some point to get the future result. They instead have there own implementation that provide async completion handler (ListenableFuture, …), which result in the pattern we are trying to avoid with coroutine and async/await. This is not a very good example.

JavaScript Promise + async/await and C# Task + async/await are good example though.

···

On 28 Aug 2017, at 23:14, Jean-Daniel via swift-evolution <swift-evolution@swift.org> wrote:

Le 28 août 2017 à 06:14, Howard Lovatt via swift-evolution <swift-evolution@swift.org> a écrit :

On 28 August 2017 at 12:35, Florent Vilmart <florent@flovilmart.com> wrote:
Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:
As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:
With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I don't really get your point ListenableFuture is a Future, so anything
using ListenableFuture is using Future. As I said in the original message
"... there are a lot of libraries built on top of basic futures ...".

I am pointing out that people actually use Future in Java as the building
block, e.g. ListenableFuture. Therefore Swift would benefit from something
comparable and if async/await doesn't lead to a better future than you can
code using GCD then there is no point.

  -- Howard.

···

On 29 August 2017 at 07:14, Jean-Daniel <mailing@xenonium.com> wrote:

Le 28 août 2017 à 06:14, Howard Lovatt via swift-evolution < > swift-evolution@swift.org> a écrit :

One of the biggest incumbents in this space on the server side is Java and
its concurrency is based on futures and works very well (though there are a
lot of libraries built on top of basic futures).

Most server side libraries don’t use Java Future as they force blocking at
some point to get the future result. They instead have there own
implementation that provide async completion handler (ListenableFuture, …),
which result in the pattern we are trying to avoid with coroutine and
async/await. This is not a very good example.

On 28 August 2017 at 12:35, Florent Vilmart <florent@flovilmart.com> > wrote:

Adam, you’re completely right, languages as c# and JS have been through
the path before, (callback, Promises , async/await) I believe Chris’s goal
it to avoid building a promise implementation and go straight to a
coroutines model, which is more deeply integrated with the compiler. I
don’t see a particular trade off, pursuing that route, and the main benefit
is that coroutines can power any asynchronous metaphor (Signals, Streams,
Futures, Promises etc...) which is not true of Futures so i would tend to
think that for the long run, and to maximize usability, async/await/yield
would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or the
other way around). You can have the best of both worlds. We are not losing
anything by having this feature. It would be a huge improvement to have
this as an option.

However, using futures correctly requires more nested closures than you
have shown in your examples to avoid blocking any threads. That's why
you're not seeing the advantage to async/await. You're comparing examples
that have very different behaviors.

That said, I have also expressed my opinion that it is better to build
async/await on top of futures rather than the other way around. I believe
it is more powerful and cleaner to make async/await work with any arbitrary
future type (via a protocol). The alternative (building futures on top of
async/await) requires more code when the two are mixed. I very much prefer
how it's done in C#, where you can freely mix the two models without having
to resort to ad-hoc wrappers, and you can use async/await with any futures
implementation you might already be using.

I really think we should be having more discussion about the tradeoffs
between those two approaches, and I'm concerned that some of the opinions
about how C# does it are not based on a clear and accurate understanding of
how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> >> wrote:

The async/await is very similar to the proposed Future (as I posed
earlier) with regard to completion-handler code, they both re-write the
imported completion-handler function using a closure, the relevant sentence
from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures
..."

Unlike the proposed future code the async code is not naturally parallel,
in the running example the following lines from the async code are run in
series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why
provide an async unless you can make a convincing argument that it allows
you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it
would require more nesting and more complex error handling. By simplifying
the code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is
exactly why async/await is so valuable. You get simple code while still
doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < >>> swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ??
Resource(path: "Default data resource or prompt user"), imageResource.get
?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or
prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by
converting continuation-handlers into either a sync or future, i.e. it is
the importer that eliminates the nesting by translating the code
automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is
created (as soon as the underlying Future is created) and get returns an
optional (also cancel and status would be still be present). Then if you
want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> >>> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >>>> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >>>> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you
can use the underlying threads just as well, i.e. future with async/await
is no better than future without.

  2. Since future is more powerful, thread control, cancel, and
timeout, people should be encouraged to use this; instead because
async/await are language features they will be presumed, incorrectly, to be
the best way, consequently people will get into trouble with deadlocks
because they don't have control.

  3. async/await will require some engineering work and will at best
make a mild syntax improvement and at worst lead to deadlocks, therefore
they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future
library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>>>> wrote:

In particular a future that is cancellable is more powerful that the
proposed async/await.

It's not more powerful; the features are to some degree disjoint. You
can build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I know what the proposal said. I’m making a case that there is value in doing it differently.

The composability of futures is valuable. Mixing and matching async/await with futures is also valuable. The queue-returning behavior that you can get from futures is also valuable, and building async/await on top of futures means async/await can get that for free.

Maybe you don’t value those things, which is fine. But I do, and maybe other people do too. That’s why we’re having a discussion about it.

It can also be valuable having a minimal implementation, but we have to acknowledge that it comes with a downside as well. The problem with doing a minimal implementation is that you can be stuck with the consequences for a long time. I want to make sure that we’re not stuck with the consequences of a minimal implementation that doesn’t adequately address the problems that async/await should be addressing. I’d hate for Swift to get an async/await that is so weak that it has to be augmented by tedious boilerplate code before it’s useful.

···

On Aug 28, 2017, at 1:54 PM, Wallacy <wallacyf@gmail.com> wrote:

We don't need to this now!

Again: (Using proposal words)

"It is important to understand that this is proposing compiler support that is completely concurrency runtime-agnostic. This proposal does not include a new runtime model (like "actors") - it works just as well with GCD as with pthreads or another API. Furthermore, unlike designs in other languages, it is independent of specific coordination mechanisms, such as futures or channels, allowing these to be built as library feature"

and

"This proposal does not formally propose a Future type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and incorporating them into this proposal only makes it larger."

and

We focus on task-based concurrency abstractions commonly encountered in client and server applications, particularly those that are highly event driven (e.g. responding to UI events or requests from clients). This does not attempt to be a comprehensive survey of all possible options, nor does it attempt to solve all possible problems in the space of concurrency. Instead, it outlines a single coherent design thread that can be built over the span of years to incrementally drive Swift to further greatness.

and

This proposal has been kept intentionally minimal, but there are many possible ways to expand this in the future.

....

The point is: No Future type is indeed proposed yet!

The proposal try to include de "minimum" required to implement a basic async/await to solve the problem created by the GCD! (Pyramid of doom)

The question is: How do you do the same using dispatch_async ? dispatch_async also does not return nothing to do what you are intentend do do!

Algo, by Swift 5 manifesto, there's no compromise to make a "complete" concurrency model by this time!

My intention is only make parity to dispatch_async, but also make the ground free to make more complex implementation like Futures in another round on top of this one.

This 'async T' can be a real type in the future? Maybe will... But doesn't matter now! Now we only need to is some kind of type which need to be unwrapped using await before use. Maybe this intermediary/virtual type can be a real thing and gain some abilities at some point! Maybe a full Future type, why not?

Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> escreveu:
How would these anonymous types get composed? If I wanted to implement a function that takes a collection of futures and wait on it, how would I do that? That is, how would I implement the equivalent of C#’s Task.WhenAll and Task.WhenAny methods?

More generally, how do you pass one of these typeless futures to some other function so that we can do the waiting there?

On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf@gmail.com <mailto:wallacyf@gmail.com>> wrote:

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No future type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource, imageResource) // Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle with one more type.

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> escreveu:
I think the biggest tradeoff is clearer when you look at the examples from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource.get(), imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future. Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(await dataResourceFuture, await imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself can be used with either await or as a future. You get both options with one implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one particular futures implementation. The Task type is commonly used, but async/await does not directly depend on Task. Instead it works with any return type that meets certain requirements (detailed here: await anything; - .NET Parallel Programming). Swift could do this using a protocol, which can be retroactively applied using an extension.

Obviously for this to be useful we would need some kind of existing future implementation, but at least we wouldn’t be tied to any particular one. That would mean library maintainers who have already been using their own futures implementations could quickly adopt async/await in their code without having to rewrite their futures library or throw wrappers around every usage of async/await. They could just adopt a protocol (using an extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await use case rather than a generic coroutine implementation (i.e., there would have to be a separate compiler transform for yield return). It’s not clear to me why it should be a goal to have just one generic coroutine feature. The real-world usages of async/await and yield return are different enough that I’m not convinced we could have a single compiler feature that meets the needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:

Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>>, wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

It actually removes the need of implementing a future library, with proper extension of collection types, we’ll be able to implement collect, zip, race, map, flatMap etc... most probably part of the Stdlib

···

On Aug 28, 2017, 21:25 -0400, Howard Lovatt via swift-evolution <swift-evolution@swift.org>, wrote:

I don't really get your point ListenableFuture is a Future, so anything using ListenableFuture is using Future. As I said in the original message "... there are a lot of libraries built on top of basic futures ...".

I am pointing out that people actually use Future in Java as the building block, e.g. ListenableFuture. Therefore Swift would benefit from something comparable and if async/await doesn't lead to a better future than you can code using GCD then there is no point.

-- Howard.

> On 29 August 2017 at 07:14, Jean-Daniel <mailing@xenonium.com> wrote:
> >
> > > Le 28 août 2017 à 06:14, Howard Lovatt via swift-evolution <swift-evolution@swift.org> a écrit :
> > >
> > > One of the biggest incumbents in this space on the server side is Java and its concurrency is based on futures and works very well (though there are a lot of libraries built on top of basic futures).
> >
> > Most server side libraries don’t use Java Future as they force blocking at some point to get the future result. They instead have there own implementation that provide async completion handler (ListenableFuture, …), which result in the pattern we are trying to avoid with coroutine and async/await. This is not a very good example.
> >
> > > > On 28 August 2017 at 12:35, Florent Vilmart <florent@flovilmart.com> wrote:
> > > > > Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.
> > > > >
> > > > > On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:
> > > > > > As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.
> > > > > >
> > > > > > However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.
> > > > > >
> > > > > > That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.
> > > > > >
> > > > > > I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.
> > > > > >
> > > > > > --
> > > > > > Adam Kemp
> > > > > >
> > > > > > On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> wrote:
> > > > > >
> > > > > > > The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:
> > > > > > >
> > > > > > > > quote_type
> > > > > > > > "Under the hood, the compiler rewrites this code using nested closures ..."
> > > > > > >
> > > > > > > Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:
> > > > > > >
> > > > > > > let dataResource = await loadWebResource("dataprofile.txt")
> > > > > > > let imageResource = await loadWebResource("imagedata.dat")
> > > > > > > The equivalent lines using the proposed Future:
> > > > > > > let dataResource = loadWebResource("dataprofile.txt")
> > > > > > > let imageResource = loadWebResource("imagedata.dat")
> > > > > > > Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.
> > > > > > >
> > > > > > > Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?
> > > > > > >
> > > > > > > -- Howard.
> > > > > > >
> > > > > > > > On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:
> > > > > > > > > This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.
> > > > > > > > >
> > > > > > > > > I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.
> > > > > > > > >
> > > > > > > > > Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).
> > > > > > > > >
> > > > > > > > > That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Adam Kemp
> > > > > > > > >
> > > > > > > > > On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:
> > > > > > > > >
> > > > > > > > > > The running example used in the white paper coded using a Future is:
> > > > > > > > > >
> > > > > > > > > > func processImageData1() -> Future<Image> {
> > > > > > > > > > return AsynchronousFuture { _ -> Image in
> > > > > > > > > > let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> > > > > > > > > > let imageResource = loadWebResource("imagedata.dat")
> > > > > > > > > > let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> > > > > > > > > > let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> > > > > > > > > > return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> > > > > > > > > > }
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.
> > > > > > > > > >
> > > > > > > > > > This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).
> > > > > > > > > >
> > > > > > > > > > There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.
> > > > > > > > > >
> > > > > > > > > > You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:
> > > > > > > > > >
> > > > > > > > > > func processImageData1() async -> Image {
> > > > > > > > > > let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
> > > > > > > > > > let imageResource = loadWebResource("imagedata.dat")
> > > > > > > > > > let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
> > > > > > > > > > let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
> > > > > > > > > > return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:
> > > > > > > > > >
> > > > > > > > > > func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }
> > > > > > > > > >
> > > > > > > > > > > On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com> wrote:
> > > > > > > > > > > > Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
> > > > > > > > > > > > Futures can’t do that
> > > > > > > > > > > >
> > > > > > > > > > > > On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org>, wrote:
> > > > > > > > > > > > > With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Sent from my iPhone
> > > > > > > > > > > > >
> > > > > > > > > > > > > On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > My argument goes like this:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Therefore, save some engineering effort and just provide a future library.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > To turn the question round another way, in two forms:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 1. What can async/wait do that a future can't?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2. How will future be improved if async/await is added?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -- Howard.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > In particular a future that is cancellable is more powerful that the proposed async/await.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > -Joe
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > > swift-evolution mailing list
> > > > > > > > > > > > > > swift-evolution@swift.org
> > > > > > > > > > > > > > https://lists.swift.org/mailman/listinfo/swift-evolution
> > > > > > > > > > --
> > > > > > > > > > -- Howard.
> > > > > > > > > > _______________________________________________
> > > > > > > > > > swift-evolution mailing list
> > > > > > > > > > swift-evolution@swift.org
> > > > > > > > > > https://lists.swift.org/mailman/listinfo/swift-evolution
> > > > > > >
> > >
> > > _______________________________________________
> > > swift-evolution mailing list
> > > swift-evolution@swift.org
> > > https://lists.swift.org/mailman/listinfo/swift-evolution
> >

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I know what the proposal said. I’m making a case that there is value in
doing it differently.

The composability of futures is valuable. Mixing and matching async/await
with futures is also valuable. The queue-returning behavior that you can
get from futures is also valuable, and building async/await on top of
futures means async/await can get that for free.

Why couldn't you mix and match async/await and futures and get the
queue-return behavior of futures if futures are built on top of async/await
instead off the other way around?

Maybe you don’t value those things, which is fine. But I do, and maybe

···

On Mon, Aug 28, 2017 at 16:10 Adam Kemp via swift-evolution < swift-evolution@swift.org> wrote:

other people do too. That’s why we’re having a discussion about it.

It can also be valuable having a minimal implementation, but we have to
acknowledge that it comes with a downside as well. The problem with doing a
minimal implementation is that you can be stuck with the consequences for a
long time. I want to make sure that we’re not stuck with the consequences
of a minimal implementation that doesn’t adequately address the problems
that async/await should be addressing. I’d hate for Swift to get an
async/await that is so weak that it has to be augmented by tedious
boilerplate code before it’s useful.

On Aug 28, 2017, at 1:54 PM, Wallacy <wallacyf@gmail.com> wrote:

We don't need to this now!

Again: (Using proposal words)

"It is important to understand that this is proposing compiler support
that is completely concurrency runtime-agnostic. This proposal does not
include a new runtime model (like "actors") - it works just as well with
GCD as with pthreads or another API. Furthermore, unlike designs in other
languages, it is independent of specific coordination mechanisms, such as
futures or channels, allowing these to be built as library feature"

and

"This proposal does not formally propose a Future type, or any other
coordination abstractions. There are many rational designs for futures, and
a lot of experience working with them. On the other hand, there are also
completely different coordination primitives that can be used with this
coroutine design, and incorporating them into this proposal only makes it
larger."

and

We focus on task-based concurrency abstractions commonly encountered in
client and server applications, particularly those that are highly event
driven (e.g. responding to UI events or requests from clients). This does
not attempt to be a comprehensive survey of all possible options, nor does
it attempt to solve all possible problems in the space of concurrency.
Instead, it outlines a single coherent design thread that can be built over
the span of years to incrementally drive Swift to further greatness.

and

This proposal has been kept intentionally minimal, but there are many
possible ways to expand this in the future.

....

The point is: No Future type is indeed proposed yet!

The proposal try to include de "minimum" required to implement a basic
async/await to solve the problem created by the GCD! (Pyramid of doom)

The question is: How do you do the same using dispatch_async ?
dispatch_async also does not return nothing to do what you are intentend do
do!

Algo, by Swift 5 manifesto, there's no compromise to make a "complete"
concurrency model by this time!

My intention is only make parity to dispatch_async, but also make the
ground free to make more complex implementation like Futures in another
round on top of this one.

This 'async T' can be a real type in the future? Maybe will... But doesn't
matter now! Now we only need to is some kind of type which need to be
unwrapped using await before use. Maybe this intermediary/virtual type can
be a real thing and gain some abilities at some point! Maybe a full Future
type, why not?

Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp@apple.com> > escreveu:

How would these anonymous types get composed? If I wanted to implement a
function that takes a collection of futures and wait on it, how would I do
that? That is, how would I implement the equivalent of C#’s Task.WhenAll
and Task.WhenAny methods?

More generally, how do you pass one of these typeless futures to some
other function so that we can do the waiting there?

On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf@gmail.com> wrote:

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No
future type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource, imageResource) //
Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle
with one more type.

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution < >> swift-evolution@swift.org> escreveu:

I think the biggest tradeoff is clearer when you look at the examples
from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource.get(),
imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future.
Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(await dataResourceFuture, await
imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself
can be used with either await or as a future. You get both options with one
implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one
particular futures implementation. The Task type is commonly used, but
async/await does not directly depend on Task. Instead it works with any
return type that meets certain requirements (detailed here:
await anything; - .NET Parallel Programming).
Swift could do this using a protocol, which can be retroactively applied
using an extension.

Obviously for this to be useful we would need some kind of existing
future implementation, but at least we wouldn’t be tied to any particular
one. That would mean library maintainers who have already been using their
own futures implementations could quickly adopt async/await in their code
without having to rewrite their futures library or throw wrappers around
every usage of async/await. They could just adopt a protocol (using an
extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await
use case rather than a generic coroutine implementation (i.e., there would
have to be a separate compiler transform for yield return). It’s not clear
to me why it should be a goal to have just one generic coroutine feature.
The real-world usages of async/await and yield return are different enough
that I’m not convinced we could have a single compiler feature that meets
the needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com> >>> wrote:

Adam, you’re completely right, languages as c# and JS have been through
the path before, (callback, Promises , async/await) I believe Chris’s goal
it to avoid building a promise implementation and go straight to a
coroutines model, which is more deeply integrated with the compiler. I
don’t see a particular trade off, pursuing that route, and the main benefit
is that coroutines can power any asynchronous metaphor (Signals, Streams,
Futures, Promises etc...) which is not true of Futures so i would tend to
think that for the long run, and to maximize usability, async/await/yield
would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or
the other way around). You can have the best of both worlds. We are not
losing anything by having this feature. It would be a huge improvement to
have this as an option.

However, using futures correctly requires more nested closures than you
have shown in your examples to avoid blocking any threads. That's why
you're not seeing the advantage to async/await. You're comparing examples
that have very different behaviors.

That said, I have also expressed my opinion that it is better to build
async/await on top of futures rather than the other way around. I believe
it is more powerful and cleaner to make async/await work with any arbitrary
future type (via a protocol). The alternative (building futures on top of
async/await) requires more code when the two are mixed. I very much prefer
how it's done in C#, where you can freely mix the two models without having
to resort to ad-hoc wrappers, and you can use async/await with any futures
implementation you might already be using.

I really think we should be having more discussion about the tradeoffs
between those two approaches, and I'm concerned that some of the opinions
about how C# does it are not based on a clear and accurate understanding of
how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> >>> wrote:

The async/await is very similar to the proposed Future (as I posed
earlier) with regard to completion-handler code, they both re-write the
imported completion-handler function using a closure, the relevant sentence
from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures
..."

Unlike the proposed future code the async code is not naturally
parallel, in the running example the following lines from the async code
are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why
provide an async unless you can make a convincing argument that it allows
you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it
would require more nesting and more complex error handling. By simplifying
the code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is
exactly why async/await is so valuable. You get simple code while still
doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < >>>> swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ??
Resource(path: "Default data resource or prompt user"), imageResource.get
?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or
prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by
converting continuation-handlers into either a sync or future, i.e. it is
the importer that eliminates the nesting by translating the code
automatically.

This example using Future also demonstrates three advantages of Future:
they are naturally parallel (dataResource and imageResource lines run in
parallel), they timeout automatically (get returns nil if the Future has
taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is
created (as soon as the underlying Future is created) and get returns an
optional (also cancel and status would be still be present). Then if you
want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart < >>>> florent@flovilmart.com> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >>>>> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >>>>> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you
can use the underlying threads just as well, i.e. future with async/await
is no better than future without.

  2. Since future is more powerful, thread control, cancel, and
timeout, people should be encouraged to use this; instead because
async/await are language features they will be presumed, incorrectly, to be
the best way, consequently people will get into trouble with deadlocks
because they don't have control.

  3. async/await will require some engineering work and will at best
make a mild syntax improvement and at worst lead to deadlocks, therefore
they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future
library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>>>>> wrote:

In particular a future that is cancellable is more powerful that
the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You
can build a Future abstraction and then use async/await to sugar code that
threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Sorry, I was away for a while so I couldn’t respond to this right away.

···

On Aug 29, 2017, at 12:29 AM, Jonathan Hull <jhull@gbis.com> wrote:

If you are asking about how ‘async’ (which defers ‘await’) composes, it actually composes completely naturally… and I believe that should be provable.

Can you show me specifically how that would work, then? For instance, let’s say that we want to start N (unknown, i.e., not constant) async requests and then, only once all of them have begun, wait for all them to complete. What would that code look like?

Extending that example, let’s say we wanted to use one function to start the N requests and then use a different function to wait on them. What would be the signatures of these functions?

I know what the proposal said. I’m making a case that there is value in doing it differently.

The composability of futures is valuable. Mixing and matching async/await with futures is also valuable. The queue-returning behavior that you can get from futures is also valuable, and building async/await on top of futures means async/await can get that for free.

Why couldn't you mix and match async/await and futures and get the queue-return behavior of futures if futures are built on top of async/await instead off the other way around?

We could, but the syntax is much worse. Contrast:

async/await built on top of Futures

let image = preprocessImage(downloadImage())
let text = translate(downloadText())
await render(image: image, text: text)

Futures built on top of async/await

let image = Future(downloadImage).then({ preprocessImage($0) })
let text = Future(downloadText).then({ translate($0) })
await render(image: image.get(), text: text.get())

···

On 29 Aug 2017, at 02:22, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:
On Mon, Aug 28, 2017 at 16:10 Adam Kemp via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Maybe you don’t value those things, which is fine. But I do, and maybe other people do too. That’s why we’re having a discussion about it.

It can also be valuable having a minimal implementation, but we have to acknowledge that it comes with a downside as well. The problem with doing a minimal implementation is that you can be stuck with the consequences for a long time. I want to make sure that we’re not stuck with the consequences of a minimal implementation that doesn’t adequately address the problems that async/await should be addressing. I’d hate for Swift to get an async/await that is so weak that it has to be augmented by tedious boilerplate code before it’s useful.

On Aug 28, 2017, at 1:54 PM, Wallacy <wallacyf@gmail.com <mailto:wallacyf@gmail.com>> wrote:

We don't need to this now!

Again: (Using proposal words)

"It is important to understand that this is proposing compiler support that is completely concurrency runtime-agnostic. This proposal does not include a new runtime model (like "actors") - it works just as well with GCD as with pthreads or another API. Furthermore, unlike designs in other languages, it is independent of specific coordination mechanisms, such as futures or channels, allowing these to be built as library feature"

and

"This proposal does not formally propose a Future type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and incorporating them into this proposal only makes it larger."

and

We focus on task-based concurrency abstractions commonly encountered in client and server applications, particularly those that are highly event driven (e.g. responding to UI events or requests from clients). This does not attempt to be a comprehensive survey of all possible options, nor does it attempt to solve all possible problems in the space of concurrency. Instead, it outlines a single coherent design thread that can be built over the span of years to incrementally drive Swift to further greatness.

and

This proposal has been kept intentionally minimal, but there are many possible ways to expand this in the future.

....

The point is: No Future type is indeed proposed yet!

The proposal try to include de "minimum" required to implement a basic async/await to solve the problem created by the GCD! (Pyramid of doom)

The question is: How do you do the same using dispatch_async ? dispatch_async also does not return nothing to do what you are intentend do do!

Algo, by Swift 5 manifesto, there's no compromise to make a "complete" concurrency model by this time!

My intention is only make parity to dispatch_async, but also make the ground free to make more complex implementation like Futures in another round on top of this one.

This 'async T' can be a real type in the future? Maybe will... But doesn't matter now! Now we only need to is some kind of type which need to be unwrapped using await before use. Maybe this intermediary/virtual type can be a real thing and gain some abilities at some point! Maybe a full Future type, why not?

Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> escreveu:
How would these anonymous types get composed? If I wanted to implement a function that takes a collection of futures and wait on it, how would I do that? That is, how would I implement the equivalent of C#’s Task.WhenAll and Task.WhenAny methods?

More generally, how do you pass one of these typeless futures to some other function so that we can do the waiting there?

On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf@gmail.com <mailto:wallacyf@gmail.com>> wrote:

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No future type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource, imageResource) // Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle with one more type.

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> escreveu:
I think the biggest tradeoff is clearer when you look at the examples from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt") }
  let imageResource = Future { await loadWebResource("imagedata.dat") }
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(dataResource.get(), imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future. Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);
  
  // ... other stuff can go here to cover load latency...
  
  let imageTmp = await decodeImage(await dataResourceFuture, await imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself can be used with either await or as a future. You get both options with one implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one particular futures implementation. The Task type is commonly used, but async/await does not directly depend on Task. Instead it works with any return type that meets certain requirements (detailed here: await anything; - .NET Parallel Programming). Swift could do this using a protocol, which can be retroactively applied using an extension.

Obviously for this to be useful we would need some kind of existing future implementation, but at least we wouldn’t be tied to any particular one. That would mean library maintainers who have already been using their own futures implementations could quickly adopt async/await in their code without having to rewrite their futures library or throw wrappers around every usage of async/await. They could just adopt a protocol (using an extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await use case rather than a generic coroutine implementation (i.e., there would have to be a separate compiler transform for yield return). It’s not clear to me why it should be a goal to have just one generic coroutine feature. The real-world usages of async/await and yield return are different enough that I’m not convinced we could have a single compiler feature that meets the needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:

Adam, you’re completely right, languages as c# and JS have been through the path before, (callback, Promises , async/await) I believe Chris’s goal it to avoid building a promise implementation and go straight to a coroutines model, which is more deeply integrated with the compiler. I don’t see a particular trade off, pursuing that route, and the main benefit is that coroutines can power any asynchronous metaphor (Signals, Streams, Futures, Promises etc...) which is not true of Futures so i would tend to think that for the long run, and to maximize usability, async/await/yield would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>>, wrote:

As has been explained, futures can be built on top of async/await (or the other way around). You can have the best of both worlds. We are not losing anything by having this feature. It would be a huge improvement to have this as an option.

However, using futures correctly requires more nested closures than you have shown in your examples to avoid blocking any threads. That's why you're not seeing the advantage to async/await. You're comparing examples that have very different behaviors.

That said, I have also expressed my opinion that it is better to build async/await on top of futures rather than the other way around. I believe it is more powerful and cleaner to make async/await work with any arbitrary future type (via a protocol). The alternative (building futures on top of async/await) requires more code when the two are mixed. I very much prefer how it's done in C#, where you can freely mix the two models without having to resort to ad-hoc wrappers, and you can use async/await with any futures implementation you might already be using.

I really think we should be having more discussion about the tradeoffs between those two approaches, and I'm concerned that some of the opinions about how C# does it are not based on a clear and accurate understanding of how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

The async/await is very similar to the proposed Future (as I posed earlier) with regard to completion-handler code, they both re-write the imported completion-handler function using a closure, the relevant sentence from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures ..."

Unlike the proposed future code the async code is not naturally parallel, in the running example the following lines from the async code are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")
The equivalent lines using the proposed Future:
  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")
Run in parallel and therefore are potentially faster assuming that resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why provide an async unless you can make a convincing argument that it allows you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com <mailto:adam.kemp@apple.com>> wrote:
This example still has nested closures (to create a Future), and still relies on a synchronous get method that will block a thread. Async/await does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good example of using futures. If you’re using a synchronous get method then you’re not using futures properly. They’re supposed to make it easy to avoid writing blocking code. This example just does the blocking call on some other thread.

Doing it properly would show the benefits of async/await because it would require more nesting and more complex error handling. By simplifying the code you’ve made a comparison between proper asynchronous code (with async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is exactly why async/await is so valuable. You get simple code while still doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by converting continuation-handlers into either a sync or future, i.e. it is the importer that eliminates the nesting by translating the code automatically.

This example using Future also demonstrates three advantages of Future: they are naturally parallel (dataResource and imageResource lines run in parallel), they timeout automatically (get returns nil if the Future has taken too long), and if there is a failure (for any reason including timeout) it provides a method of either detecting the failure or providing a default (get returns nil on failure).

There are a three of other advantages a Future has that this example doesn’t show: control over which thread the Future runs on, Futures can be cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") // dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path: "Default data resource or prompt user"), imageResource.get ?? Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is created (as soon as the underlying Future is created) and get returns an optional (also cancel and status would be still be present). Then if you want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout: .seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <florent@flovilmart.com <mailto:florent@flovilmart.com>> wrote:
Howard, with async / await, the code is flat and you don’t have to unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

With both he now built in promises in Node8 as well as libraries like Bluebird there was ample time to evaluate them and convert/auto convert at times libraries that loved callback pyramids of doom when the flow grows complex into promise based chains. Converting to Promises seems magical for the simple case, but can quickly descend in hard to follow flows and hard to debug errors when you move to non trivial multi path scenarios. JS is now solving it with their implementation of async/await, but the point is that without the full picture any single solution would break horribly in real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you can use the underlying threads just as well, i.e. future with async/await is no better than future without.

  2. Since future is more powerful, thread control, cancel, and timeout, people should be encouraged to use this; instead because async/await are language features they will be presumed, incorrectly, to be the best way, consequently people will get into trouble with deadlocks because they don't have control.

  3. async/await will require some engineering work and will at best make a mild syntax improvement and at worst lead to deadlocks, therefore they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com <mailto:howard.lovatt@gmail.com>> wrote:

In particular a future that is cancellable is more powerful that the proposed async/await.

It's not more powerful; the features are to some degree disjoint. You can build a Future abstraction and then use async/await to sugar code that threads computation through futures. Getting back to Jakob's example, someone (maybe the Clang importer, maybe Apple's framework developers in an overlay) will still need to build infrastructure on top of IBActions and other currently ad-hoc signalling mechanisms to integrate them into a more expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

--
-- Howard.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

My hope would be for something along these lines:

func createDownloadTasks(for urls: [URL]) -> [async Data] {
    return urls.map { url in async downloadResource(url) }
}
func await(all tasks: [async Data]) async -> [Data] {
    return tasks.map { task in await task }
}

-Thorsten

···

Am 11.09.2017 um 22:49 schrieb Adam Kemp via swift-evolution <swift-evolution@swift.org>:

Sorry, I was away for a while so I couldn’t respond to this right away.

On Aug 29, 2017, at 12:29 AM, Jonathan Hull <jhull@gbis.com> wrote:

If you are asking about how ‘async’ (which defers ‘await’) composes, it actually composes completely naturally… and I believe that should be provable.

Can you show me specifically how that would work, then? For instance, let’s say that we want to start N (unknown, i.e., not constant) async requests and then, only once all of them have begun, wait for all them to complete. What would that code look like?

Extending that example, let’s say we wanted to use one function to start the N requests and then use a different function to wait on them. What would be the signatures of these functions?

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

@David,

Using the `Future` library based on GCD that I have previously posted your
example would be:

let image = preprocessImage(downloadImage()) // These first two lines
run in parallellet text = translate(downloadText())render(image:
image.get ?? defaultImage, text: text.get ?? defaultText)

The main difference, and I would argue an improvement, is that the `Future`
version handles errors.

So what advantage does async/await have over a `Future` library we can
write today?

  -- Howard.

···

On 29 August 2017 at 15:28, David Hart via swift-evolution < swift-evolution@swift.org> wrote:

On 29 Aug 2017, at 02:22, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

On Mon, Aug 28, 2017 at 16:10 Adam Kemp via swift-evolution < > swift-evolution@swift.org> wrote:

I know what the proposal said. I’m making a case that there is value in
doing it differently.

The composability of futures is valuable. Mixing and matching async/await
with futures is also valuable. The queue-returning behavior that you can
get from futures is also valuable, and building async/await on top of
futures means async/await can get that for free.

Why couldn't you mix and match async/await and futures and get the
queue-return behavior of futures if futures are built on top of async/await
instead off the other way around?

We could, but the syntax is much worse. Contrast:

*async/await built on top of Futures*

let image = preprocessImage(downloadImage())let text = translate(downloadText())
await render(image: image, text: text)

*Futures built on top of async/await*

let image = Future(downloadImage).then({ preprocessImage($0) })let text = Future(downloadText).then({ translate($0) })
await render(image: image.get(), text: text.get())

Maybe you don’t value those things, which is fine. But I do, and maybe

other people do too. That’s why we’re having a discussion about it.

It can also be valuable having a minimal implementation, but we have to
acknowledge that it comes with a downside as well. The problem with doing a
minimal implementation is that you can be stuck with the consequences for a
long time. I want to make sure that we’re not stuck with the consequences
of a minimal implementation that doesn’t adequately address the problems
that async/await should be addressing. I’d hate for Swift to get an
async/await that is so weak that it has to be augmented by tedious
boilerplate code before it’s useful.

On Aug 28, 2017, at 1:54 PM, Wallacy <wallacyf@gmail.com> wrote:

We don't need to this now!

Again: (Using proposal words)

"It is important to understand that this is proposing compiler support
that is completely concurrency runtime-agnostic. This proposal does not
include a new runtime model (like "actors") - it works just as well with
GCD as with pthreads or another API. Furthermore, unlike designs in other
languages, it is independent of specific coordination mechanisms, such as
futures or channels, allowing these to be built as library feature"

and

"This proposal does not formally propose a Future type, or any other
coordination abstractions. There are many rational designs for futures, and
a lot of experience working with them. On the other hand, there are also
completely different coordination primitives that can be used with this
coroutine design, and incorporating them into this proposal only makes it
larger."

and

We focus on task-based concurrency abstractions commonly encountered in
client and server applications, particularly those that are highly event
driven (e.g. responding to UI events or requests from clients). This does
not attempt to be a comprehensive survey of all possible options, nor does
it attempt to solve all possible problems in the space of concurrency.
Instead, it outlines a single coherent design thread that can be built over
the span of years to incrementally drive Swift to further greatness.

and

This proposal has been kept intentionally minimal, but there are many
possible ways to expand this in the future.

....

The point is: No Future type is indeed proposed yet!

The proposal try to include de "minimum" required to implement a basic
async/await to solve the problem created by the GCD! (Pyramid of doom)

The question is: How do you do the same using dispatch_async ?
dispatch_async also does not return nothing to do what you are intentend do
do!

Algo, by Swift 5 manifesto, there's no compromise to make a "complete"
concurrency model by this time!

My intention is only make parity to dispatch_async, but also make the
ground free to make more complex implementation like Futures in another
round on top of this one.

This 'async T' can be a real type in the future? Maybe will... But
doesn't matter now! Now we only need to is some kind of type which need to
be unwrapped using await before use. Maybe this intermediary/virtual type
can be a real thing and gain some abilities at some point! Maybe a full
Future type, why not?

Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp@apple.com> >> escreveu:

How would these anonymous types get composed? If I wanted to implement a
function that takes a collection of futures and wait on it, how would I do
that? That is, how would I implement the equivalent of C#’s Task.WhenAll
and Task.WhenAny methods?

More generally, how do you pass one of these typeless futures to some
other function so that we can do the waiting there?

On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf@gmail.com> wrote:

And that's why I (and others) are suggesting:

func processImageData1a() async -> Image {
  let dataResource = async loadWebResource("dataprofile.txt") // No
future type here... Just another way to call dispatch_async under the hood.
  let imageResource = async loadWebResource("imagedata.dat")

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource, imageResource) //
Compiles force await call here...
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

And now we gain all advantages of async/await again without to handle
with one more type.

Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution < >>> swift-evolution@swift.org> escreveu:

I think the biggest tradeoff is clearer when you look at the examples
from the proposal where futures are built on top of async/await:

func processImageData1a() async -> Image {
  let dataResource = Future { await loadWebResource("dataprofile.txt")
}
  let imageResource = Future { await loadWebResource("imagedata.dat") }

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(dataResource.get(),
imageResource.get())
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

With this approach you have to wrap each call site to create a future.
Compare to this:

func processImageData1a() -> Future<Image> {
  let dataResourceFuture = loadWebResource("dataprofile.txt”);
  let imageResourceFuture = loadWebResource("imagedata.dat”);

  // ... other stuff can go here to cover load latency...

  let imageTmp = await decodeImage(await dataResourceFuture, await
imageResourceFuture)
  let imageResult = await dewarpAndCleanupImage(imageTmp)
  return imageResult
}

Here, not only are the explicit wrappers gone, but this function itself
can be used with either await or as a future. You get both options with one
implementation.

As I’ve mentioned before, C#’s implementation is not tied to any one
particular futures implementation. The Task type is commonly used, but
async/await does not directly depend on Task. Instead it works with any
return type that meets certain requirements (detailed here:
await anything; - .NET Parallel Programming).
Swift could do this using a protocol, which can be retroactively applied
using an extension.

Obviously for this to be useful we would need some kind of existing
future implementation, but at least we wouldn’t be tied to any particular
one. That would mean library maintainers who have already been using their
own futures implementations could quickly adopt async/await in their code
without having to rewrite their futures library or throw wrappers around
every usage of async/await. They could just adopt a protocol (using an
extension, even) and get async/await support for free.

The downside is that this feature would be specific to the async/await
use case rather than a generic coroutine implementation (i.e., there would
have to be a separate compiler transform for yield return). It’s not clear
to me why it should be a goal to have just one generic coroutine feature.
The real-world usages of async/await and yield return are different enough
that I’m not convinced we could have a single compiler feature that meets
the needs of both cleanly.

On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent@flovilmart.com> >>>> wrote:

Adam, you’re completely right, languages as c# and JS have been through
the path before, (callback, Promises , async/await) I believe Chris’s goal
it to avoid building a promise implementation and go straight to a
coroutines model, which is more deeply integrated with the compiler. I
don’t see a particular trade off, pursuing that route, and the main benefit
is that coroutines can power any asynchronous metaphor (Signals, Streams,
Futures, Promises etc...) which is not true of Futures so i would tend to
think that for the long run, and to maximize usability, async/await/yield
would probably be the way to go.

On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp@apple.com>, wrote:

As has been explained, futures can be built on top of async/await (or
the other way around). You can have the best of both worlds. We are not
losing anything by having this feature. It would be a huge improvement to
have this as an option.

However, using futures correctly requires more nested closures than you
have shown in your examples to avoid blocking any threads. That's why
you're not seeing the advantage to async/await. You're comparing examples
that have very different behaviors.

That said, I have also expressed my opinion that it is better to build
async/await on top of futures rather than the other way around. I believe
it is more powerful and cleaner to make async/await work with any arbitrary
future type (via a protocol). The alternative (building futures on top of
async/await) requires more code when the two are mixed. I very much prefer
how it's done in C#, where you can freely mix the two models without having
to resort to ad-hoc wrappers, and you can use async/await with any futures
implementation you might already be using.

I really think we should be having more discussion about the tradeoffs
between those two approaches, and I'm concerned that some of the opinions
about how C# does it are not based on a clear and accurate understanding of
how it actually works in that language.

--
Adam Kemp

On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt@gmail.com> >>>> wrote:

The async/await is very similar to the proposed Future (as I posed
earlier) with regard to completion-handler code, they both re-write the
imported completion-handler function using a closure, the relevant sentence
from the Async Proposal is:

"Under the hood, the compiler rewrites this code using nested closures
..."

Unlike the proposed future code the async code is not naturally
parallel, in the running example the following lines from the async code
are run in series, i.e. await blocks:

  let dataResource = await loadWebResource("dataprofile.txt")
  let imageResource = await loadWebResource("imagedata.dat")

The equivalent lines using the proposed Future:

  let dataResource = loadWebResource("dataprofile.txt")
  let imageResource = loadWebResource("imagedata.dat")

Run in parallel and therefore are potentially faster assuming that
resources, like cores and IO, are available.

Therefore you would be better using a Future than an async, so why
provide an async unless you can make a convincing argument that it allows
you to write a better future?

  -- Howard.

On 28 August 2017 at 09:59, Adam Kemp <adam.kemp@apple.com> wrote:

This example still has nested closures (to create a Future), and still
relies on a synchronous get method that will block a thread. Async/await
does not require blocking any threads.

I’m definitely a fan of futures, but this example isn’t even a good
example of using futures. If you’re using a synchronous get method then
you’re not using futures properly. They’re supposed to make it easy to
avoid writing blocking code. This example just does the blocking call on
some other thread.

Doing it properly would show the benefits of async/await because it
would require more nesting and more complex error handling. By simplifying
the code you’ve made a comparison between proper asynchronous code (with
async/await) and improper asynchronous code (your example).

That tendency to want to just block a thread to make it easier is
exactly why async/await is so valuable. You get simple code while still
doing it correctly.

--
Adam Kemp

On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution < >>>>> swift-evolution@swift.org> wrote:

The running example used in the white paper coded using a Future is:

func processImageData1() -> Future<Image> {
    return AsynchronousFuture { _ -> Image in
        let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
        let imageResource = loadWebResource("imagedata.dat")
        let imageTmp = decodeImage(dataResource.get ??
Resource(path: "Default data resource or prompt user"), imageResource.get
?? Resource(path: "Default image resource or prompt user"))
        let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
        return imageResult.get ?? Image(dataPath: "Default image or
prompt user", imagePath: "Default image or prompt user")
    }
}

This also avoids the pyramid of doom; the pyramid is avoided by
converting continuation-handlers into either a sync or future, i.e. it is
the importer that eliminates the nesting by translating the code
automatically.

This example using Future also demonstrates three advantages of
Future: they are naturally parallel (dataResource and imageResource lines
run in parallel), they timeout automatically (get returns nil if the Future
has taken too long), and if there is a failure (for any reason including
timeout) it provides a method of either detecting the failure or providing
a default (get returns nil on failure).

There are a three of other advantages a Future has that this example
doesn’t show: control over which thread the Future runs on, Futures can be
cancelled, and debugging information is available.

You could imagine `async` as a syntax sugar for Future, e.g. the above
Future example could be:

func processImageData1() async -> Image {
    let dataResource = loadWebResource("dataprofile.txt") //
dataResource and imageResource run in parallel.
    let imageResource = loadWebResource("imagedata.dat")
    let imageTmp = decodeImage(dataResource.get ?? Resource(path:
"Default data resource or prompt user"), imageResource.get ??
Resource(path: "Default image resource or prompt user"))
    let imageResult = dewarpAndCleanupImage(imageTmp.get ??
Image(dataPath: "Default image or prompt user", imagePath: "Default image
or prompt user"))
    return imageResult.get ?? Image(dataPath: "Default image or prompt
user", imagePath: "Default image or prompt user")
}

Since an async is sugar for Future the async runs as soon as it is
created (as soon as the underlying Future is created) and get returns an
optional (also cancel and status would be still be present). Then if you
want control over threads and timeout they could be arguments to async:

func processImageData1() async(queue: DispatchQueue.main, timeout:
.seconds(5)) -> Image { ... }

On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart < >>>>> florent@flovilmart.com> wrote:

Howard, with async / await, the code is flat and you don’t have to
unowned/weak self to prevent hideous cycles in the callbacks.
Futures can’t do that

On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution < >>>>>> swift-evolution@swift.org>, wrote:

With both he now built in promises in Node8 as well as libraries like
Bluebird there was ample time to evaluate them and convert/auto convert at
times libraries that loved callback pyramids of doom when the flow grows
complex into promise based chains. Converting to Promises seems magical for
the simple case, but can quickly descend in hard to follow flows and hard
to debug errors when you move to non trivial multi path scenarios. JS is
now solving it with their implementation of async/await, but the point is
that without the full picture any single solution would break horribly in
real life scenarios.

Sent from my iPhone

On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution < >>>>>> swift-evolution@swift.org> wrote:

My argument goes like this:

  1. You don't need async/await to write a powerful future type; you
can use the underlying threads just as well, i.e. future with async/await
is no better than future without.

  2. Since future is more powerful, thread control, cancel, and
timeout, people should be encouraged to use this; instead because
async/await are language features they will be presumed, incorrectly, to be
the best way, consequently people will get into trouble with deadlocks
because they don't have control.

  3. async/await will require some engineering work and will at best
make a mild syntax improvement and at worst lead to deadlocks, therefore
they just don't carry their weight in terms of useful additions to Swift.

Therefore, save some engineering effort and just provide a future
library.

To turn the question round another way, in two forms:

  1. What can async/wait do that a future can't?

  2. How will future be improved if async/await is added?

  -- Howard.

On 26 August 2017 at 02:23, Joe Groff <jgroff@apple.com> wrote:

On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt@gmail.com> >>>>>>> wrote:

In particular a future that is cancellable is more powerful that
the proposed async/await.

It's not more powerful; the features are to some degree disjoint.
You can build a Future abstraction and then use async/await to sugar code
that threads computation through futures. Getting back to Jakob's example,
someone (maybe the Clang importer, maybe Apple's framework developers in an
overlay) will still need to build infrastructure on top of IBActions and
other currently ad-hoc signalling mechanisms to integrate them into a more
expressive coordination framework.

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--

-- Howard.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

So that’s basically a new type with a special syntax. What is the advantage to doing it that way versus just introducing a new protocol?

···

On Sep 11, 2017, at 9:56 PM, Thorsten Seitz <tseitz42@icloud.com> wrote:

My hope would be for something along these lines:

func createDownloadTasks(for urls: [URL]) -> [async Data] {
   return urls.map { url in async downloadResource(url) }
}
func await(all tasks: [async Data]) async -> [Data] {
   return tasks.map { task in await task }
}