If this would be true, there wouldn't have been any accepted proposals at all...
However, I agree with the gist that things are hurried too much (that did not start with the threading-story; maybe it started even before SwiftUI); the time of the "small steps" seems to be over.
That isn't bad per se, and I often wished for a "bigger picture" — but it is impossible to avoid errors when deciding complex questions, and I have the constant fear that the compatibility concerns will prohibit to correct them :-(
So although (afaik) it breaks with the official process, I think it is a good idea to release "experimental" features; there should be months or even years of evaluation before all those proposals become fully accepted.
6 Likes
Several years ago I came across this fantastic two-part blog post, which might be of interest here: All about Concurrency in Swift - Part 1: The Present explores Swift's current (as of ~2017) async functionality, such as NSLock and GCD, while Part 2 compares various models like coroutines, actors, futures/promises, etc, with code examples from other languages such as Go.
I haven't reread the entire posts recently, but I remember them to be a great introduction to concurrency in general, as well as a good rundown as to how other languages solve these problems.
2 Likes
+1
You're asking for an unbounded amount of unspecified work. That is not and has never been the burden of proof for a proposer, because it is unattainable.
This implementation scheme works when there is only surface-level integration between the syntactic sugar and the underlying feature. Arrays and Dictionaries only intersect with the compiler in fairly limited ways (type and expression sugar; one extra set of implicit conversions) that don't pervade the implementation or type system.
Optional types being syntactic sugar for Optional<Wrapper> is basically a lie: they are special-cased at every level of the compiler and runtime to the point where we would have been better off making them true structural types rather than pretending they are nominal.
Syntactic sugar is nice. It's easy to reason about, easy to show how it improves things, but it doesn't move the model forward to allow you to do something truly different.
This approach might make sense if we'd been shipping a Task or Future API in the standard library for a long time and a huge ecosystem of APIs that worked with that approach. Neither of those holds, and trying to cope with all of the various kinds of future types folks work with would likely make a mess.
That approach also has a large opportunity cost because you don't get to build an efficient implementation of asynchronous functions. In a sense, the lightweight nature of the async/await syntax does too good a job of hiding what's going on under the hood. If it were just closures and captures and state machines, which is the implementation model one does for syntactic sugar on top of Task-returning synchronous functions, we'd have missed out on the ability to purpose-build async functions for efficient suspension.
You aren't happy about throws-vs-Result, but it also proves the point above: if it were layered on top of Result, code size would increase and performance would suffer due to the generic manipulation code around Result and the need to emit "thunks" from non-throwing to throwing functions. Instead, Swift's calling convention for throwing functions is purpose-built to minimize code size and allow conversions from non-throwing to throwing functions to be simple bitwise operations (no thunks), making it efficient enough to use everywhere that needs error handling. The effect with async will be more pronounced because the difference between the "sugar" implementation and a purpose-built implementation is greater.
Some---perhaps many---features make sense as syntactic sugar over something else that you can already express. But it's important to distinguish cases where syntactic sugar suffices from cases where making a feature truly succeed requires deep integration. Concurrency requires deep integration.
Doug
27 Likes
This is a great point; it makes lots of sense and is explained very clearly. Can it be recorded for posterity in the proposal text?
9 Likes
+1
It could be in the Considered Alternative section. A fair number of people also asked for a sugar-based solution
2 Likes
This is a great way of wording something I’ve been trying to get after for a long time:
Many statements by the proposal authors imply that they, the authors, have put a lot of thought into considering the design of this proposal. The authors will, for example, immediately dismiss out-of-hand a possible alternative that somebody raises, for reasons that the author finds trivial or obvious.
But in fact those reasons are not at all obvious to everyone else (as evidenced by the number of such posts), and they only seem trivial to the authors because those authors have been focused so deeply on the async domain for so long that reasoning about it has become second-nature to them.
The authors are (hopefully) experts in the area.
The proposal should be the method by which the authors communicate their expertise to the rest of the community.
Quite the opposite.
Proposals have an “alternatives considered” section, where the authors are to explain what other avenues they explored, and why they ultimately settled on the one being proposed.
You were able to quickly summarize why you see async / await as a better fit for Swift than basing everything on futures. That shows you have already thought about the matter.
In other words, you have already considered a futures-based concurrency system, and you have determined that async / await is preferable. That is the type of information which should be documented in the “alternatives considered” section.
You know why (in your view) async makes a better foundation than futures. But the rest of us, those who have not spent months or years developing a concurrency system alongside you, do not.
This same principle holds for every design decision in the proposal. The authors know why those choices were made. They have already thought about it, reasoned through it, and come to a conclusion. But the rest of us don’t and haven’t.
The authors’ reasons should be recorded for posterity in the proposal text.
Conversely, if the authors of a proposal (any proposal) had not considered a reasonable spread of alternative approaches, then it seems obvious to me that such a proposal ought to be returned until the authors had done their due diligence and explained in the text what other designs they considered, and why they believe the one being proposed is best for Swift.
10 Likes
Thanks for taking the time to respond.
No, that is not what I'm asking for. I'm asking for the "Alternatives Considered" section of the proposal to show actual alternatives, not minor variations on what was proposed. @Nevin describes this very well.
Edited to add: A different way of describing what's missing here is that in order to judge this proposal, I need something to compare it against. The proposal offers nothing in the way of comparison. It starts off with the carte blanche assumption that async/await is the right solution, but does not explain why. If we accept it as-is, then we will forever be left wondering "but what if another thing...".
I'm not asking for treatises on other concurrency systems. I'm asking for the "Alternatives Considered" section to say "we considered futures; here's where they're great; here's where they fall short. because of x, y, and z, we don't think it's a viable future for everything we want to do" and "we considered syntactic manipulation; here's where it's great; here's where it falls short. because of x, y, and z, we don't think it's a viable future for everything we want to do" and so on.
I make no claims about being a compiler engineer. I have no idea how hard or easy any of this is to implement; that sort of thing is well outside my wheelhouse.
What I do know is that the throws-vs-result world has been frustrating for me to use as a developer. I'm getting the same sense from this proposal about async/await and its co-existence with existing concurrency code. That, to me, indicates that this proposal will end up causing me even more frustration as a developer, because I'll be using a language that forces me down a particular syntactic path in certain contexts whether I want that style or not. That's frustrating and inelegant and counter to my productivity.
I want Swift to be the best language to use, not one where new features make it ever-more frustrating.
So you say. But without seeing any alternatives discussed, what basis do I have for accepting this claim?
The point of these review threads is for community members to express their thoughts on a proposal. These are my thoughts:
As the proposal currently stands, it does not sufficiently convince me this is the right path forward, and therefore, I do not believe it should be accepted.
3 Likes
My mental model of a proposal is that they just need to highlight net new changes to swift. If more context is needed then this should be provided as a reference to prior art.
I do agree with you that there may be some concepts that are so different from previous art that we would need all the context to be included in the proposal but I don't believe this specific proposal falls in that category because swift is just adopting an already popular idea from other languages. In the case of async/await the authors point out a previous document that has a section named learning-from-other-concurrency-designs
If swift was the first language to propose async/await then I would expect exactly what you are proposing.
3 Likes
No.
There is currently a proposal to adopt (and adapt) an idea from other languages into Swift.
That proposal does not, in its present form, explain why we should adopt this one particular idea, rather than any of the other existing ideas in the same space, some of which are also already popular in other languages.
Moreover, this is not the garden variety “Let’s add a few things to the standard library” type of proposal.
This is a low-level proposal that would make a fundamental design choice for Swift. In order to evaluate that, it is necessary to consider, in detail, what other designs could be chosen, what the benefits and tradeoffs of different choices are, and how that will affect the language and libraries going forward.
“Alternatives considered” is not simply nice-to-have here. It is absolutely imperative to consider alternatives. And not just in a cursory, “Oh, here are some other ideas, but we like this one better,” sort of manner.
The alternatives should be steel-manned, not straw-manned.
We’re not asking the authors to prove “beyond a reasonable doubt” that no better alternative exists, but the proposal should present clear and convincing evidence that the authors made a good-faith effort to identify the best design they could find.
6 Likes
@Chris_Lattner3's Swift Concurrency Manifesto definitely goes over many alternative designs/approaches, and there are a couple specific blurbs in there about futures too.
Perhaps not all of these alternatives were mentioned in the proposal, but clearly they've been considered, and reasons for Swift to go with async/await (instead of known alternatives) were given in that doc
5 Likes
Yes, but I think what many have been expressing in trend in hard to track decision making documentation (documentation in general) regarding the Swift project.
This has come up in a few proposals that referenced material and important context are missing from proposals that exist, because it's not mentioned or linked to in a centralized official location that is easy to reference: The Swift Evolution Proposal itself.
If these things are considered "background context" to the proposal's frame of reference, it needs to be explicitly mentioned and linked to in order for readers to be in the same evaluation bounds that the author's are in.
EDIT: The mentioned "Concurrency Manifesto" by Chris is linked in the proposal, but the statement stands for any other possible discussions that were written that are considered prior research for the proposal.
In general I agree with this proposal, but I'm also 100% in agreement with @davedelong and others who have expressed concerns over the proposal's effective use of our ability to decide if it's the best choice without at least a basic pros/cons list of other approaches to concurrency that were considered.
@davedelong as a side-note, it was explicitly mentioned through the Result proposal review that the intention is that the type might end up deprecated and was a stop-gap for when we had a first-class concurrency model so that you just annotate async throws on a method and never deal with Result again.
... but I wrote that last statement before finding and re-reading the link, and the quote I remembered and the quote written are different and makes me wonder if Result might have some role to play in the Structured Concurrency conversation (or even concurrency model)
2 Likes
I certainly agree that this is basically a documentation issue, excellent documentation has been demonstrated to be difficult time and time again (and I'd personally despise for anything at all to be held up by a lack of documentation).
Mostly I was just helping connect the dots, because docs are hard :)
1 Like
Sure, I stubbed out some wording here: [SE-0296] Add Alternative Considered for async/await as Futures sugar · DougGregor/swift-evolution@0c1d578 · GitHub
Yes, we had considered it. It's not a good match for Swift. I've now documented why it in Alternatives Considered for posterity.
There is no way for authors to predict what people in a review will think is an idea that "must" be considered as an alternative or else the proposal is incomplete. That's what discussion is for, and some things that come up in discussion get promoted to "Alternatives Considered."
Doug
11 Likes
Part of the reason that Result was kept out of the standard library so long is because it's duplicative of throws, which causes confusion. Result was also primarily motivated by asynchronous code patterns that we knew could be subsumed by an async analogue to throws in time. But we also knew that "in time" was going to be several years, and having numerous Result re-implementations in the ecosystem.
async/await may only have come up for formal review two weeks ago, but this approach to solving the problem has been around in the Swift community for ~4 years.
Doug
4 Likes
But there is a way for the authors to list the alternatives they actually did consider.
I still have not seen a response to this. Perhaps I missed it?
suspend is a little long. Maybe we could shorten it to sus...
I’ll see myself out...
4 Likes
The review period has now ended and the proposal has been accepted.
Thanks to everyone who participated!
6 Likes