Current state of the Swift ecosystem with regards to promises isn't much better than the callback hell. There are multiple implementations, none of which are considered for inclusion in the standard library. None of these libraries are compatible with each other, and even worse, I see a lot of 3rd-party libraries shipping their own promise implementations as well, that's in addition to each one having their own Result type implementation.
Of these libraries the fastest are Google Promises and BrightFutures, but only the latter can be used on Linux (untested though).
I honestly think that deferring or ignoring this issue hurts adoption of Swift significantly. When picking a language for server-side, ease of use and availability of libraries matters more than being able to write frontend and backend code in the same language.
Significant amount of server-side libraries written Swift would benefit from first-class concurrency support, but they aren't being written or not as good as they could be.
If this is fixed only with Swift 10 (i.e. in 5+ years), that would be too late. Consider that a lot of foundational server-side software in recent years have been written in Go (Docker, Kubernetes, CockroachDB etc) and even more would be written during those 5+ years. It could have been written in Swift if there was more focus on concurrency and Linux support.
I think in order to get this going we should have commitment from the core team about when this feature should land. Without this commitment I feel any work on these features is going to be subject to extreme change. Once we have a commitment from the core team that first-class concurrency is ready to be tackled in Swift I think we can start committing resources to the problem.
Chris' PR only covers the parsing for the foundational aspects of first-class concurrency, and doesn't include the SILGen or runtime support needed to support the part 1 proposal. So I believe at the minimum it would need to address that issue.
I agree with your point that the momentum for server-side Swift would need to overcome if this doesn't end up getting tackled until Swift 10 would be great. The issue is that there is a finite number of people with the expertise or willing to tackle the implementation of this. Which is why I think this should be a focused effort of multiple Swift releases. Without a release plan for this (which would require some core team commitments) I have serious doubts that this is the right time to tackle this issue as a community.
That pull request implements parsing and semantic analysis. There’s no plumbing.
You’ll find the goals for Swift 5 are significantly more fundamental: String ergonomics, generics improvements for the standard library, ABI stability. Swift is a very young language, and there’s just a lot to do.
It’s not necessary to make the case that concurrency is important. So too is better reflection, Windows support, and so much more.
The fundamental cause of the stall in progress on async/await was a concern about how it would interact with the GCD "thread hopping" problems. It was deemed unacceptable to have apparently straight-line code like this:
foo()
await bar()
baz()
run baz on a different queue/thread than foo was invoked from, which can and will happen if this is a compiler/language only feature. Fixing this requires integration with GCD, which wasn't something that people had time to consider in this cycle.
Would it be possible to implement just the concurrency part without parallelism and GCD? That is, async functions called with await would always execute on the same thread they were called? My understanding is that this would be quite similar (if not the same conceptually) as generators/yield, is that correct?
@Chris_Lattner3 would this undesired thread-hopping be possible even if bar() doesn't do any internal thread hopping (no GCD or pthread usage at all)?
In other words, does async/await ever hop threads by itself (as a part of its implementation)? Or is the problem only that it doesn't ensure thread-hopping can't happen.
For server-side Swift, (specifically Vapor 3), we are using a multi-reactor event-loop pattern (similar to Node.js/Vert.x) where switching threads is forbidden. We are currently using Promises/Futures as a stop-gap for async/await, and because of this no-thread requirement, we can implement promise completion without any thread safety guarantees. (In other words, if you complete a promise on a different thread, its undefined behavior--likely going to cause a bad access crash later). This has allowed us to completely eschew GCD and any sort for thread-safety checks through the framework, yielding amazing performance results.
My question is, would async/await at just the language/compiler level work in this context? In other words, can async/await at the compiler level be thought of simply as syntax sugar for callbacks?
Controlling concurrent access to state by serializing accesses through a queue is a battle-tested architecture for concurrent programs. I'm concerned that you seem to be describing a global queue, though, because it precludes the presence of any true concurrency in the program.
If you do have true concurrency, then the model where async/await is just sugar for callbacks is quite treacherous, because the set of queues that are serialized with the current context matters a lot for the correctness of code. The async/await sugar is arguably worse than the alternative in this model because it strongly suggests to the reader that there's a consistent context that control has returned to, but in fact the context might be completely unserialized — just whatever thread the callback happened to be called on. In contrast, while explicit closures are painful to both read and write, they do at least create obvious delimiters marking where the execution context might have changed.
So while the basic mechanics of async/await don't involve hopping threads, their interaction with things like callback APIs really do need to. Ideally, we pass down the right set of information to the callback API that tells it where to execute the callback, but in the worst case, we have to pass a callback that actually does the hop explicitly.
I just re-read the async/await proposal again (been a long time). And I think for the most part it answered all of my questions. It seems like async/await as described in that document is exactly what we need.
@Chris_Lattner3 you noted that “ wasn’t something that people had time to consider in this cycle” which I’m guessing meant Swift 4.
Is this something that is being considered for Swift 5 or 6? This would be absolutely HUGE for server-side Swift.
Is there anything I (or the server-side Swift community at large) can do to help push this forward? Even if this were added as an experimental feature at first (maybe behind some sort of compiler flag?) that would be incredible.
We are in the Swift 5 evolution cycle. As @Chris_Lattner3 wrote, concurrency is not being considered for Swift 5, and it is not yet decided what will be considered for Swift 6.