Oh, I see that this is addressed in the “ x = await (await foo()).bar()” example. That IS ugly.
-W
Hi Chris,
I love what I’ve read so far. I just have one curiosity from my cursory look over the proposal.
It seems that in transitioning API to an Async Await system, the completion handler’s type becomes the return type. How would you handle bridging in Foundation API that already has a return type, eg URLSession returns you data tasks, and then fires a completion handler. Perhaps I missed something. I’m sure you could always form a tuple as well, just curious the thoughts on that.
Ah, e.g. this one:
extension URLSession {
/*
* data task convenience methods. These methods create tasks that
* bypass the normal delegate calls for response and data delivery,
* and provide a simple cancelable asynchronous interface to receiving
* data. Errors will be returned in the NSURLErrorDomain,
* see <Foundation/NSURLError.h>. The delegate, if any, will still be
* called for authentication challenges.
*/
open func dataTask(with request: URLRequest, completionHandler: @escaping (Data?, URLResponse?, Error?) -> Swift.Void) -> URLSessionDataTask
}
Returning a tuple is an option, but probably wouldn’t provide the right semantics. The URLSessionDataTask is supposed to be available immediately, not just when the completion handler is called. The most conservative thing to do is to not have the importer change these. If they should have async semantics, they can be manually handled with an overlay.
In any case, I wasn’t aware of those APIs, thanks for pointing them out. I added a mention of this to the proposal!
-Chris
···
On Aug 17, 2017, at 4:56 PM, Rod Brown <rodney.brown6@icloud.com> wrote:
Thanks,
Rod
On 18 Aug 2017, at 8:25 am, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Aug 17, 2017, at 3:24 PM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:
Hi all,
As Ted mentioned in his email, it is great to finally kick off discussions for what concurrency should look like in Swift. This will surely be an epic multi-year journey, but it is more important to find the right design than to get there fast.
I’ve been advocating for a specific model involving async/await and actors for many years now. Handwaving only goes so far, so some folks asked me to write them down to make the discussion more helpful and concrete. While I hope these ideas help push the discussion on concurrency forward, this isn’t in any way meant to cut off other directions: in fact I hope it helps give proponents of other designs a model to follow: a discussion giving extensive rationale, combined with the long term story arc to show that the features fit together.
Anyway, here is the document, I hope it is useful, and I’d love to hear comments and suggestions for improvement:
Swift Concurrency Manifesto · GitHubOh, also, one relatively short term piece of this model is a proposal for adding an async/await model to Swift (in the form of general coroutine support). Joe Groff and I wrote up a proposal for this, here:
Concrete proposal for async semantics in Swift · GitHuband I have a PR with the first half of the implementation here:
Async await prototype by lattner · Pull Request #11501 · apple/swift · GitHubThe piece that is missing is code generation support.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
I think the comments on “prettify()” are from an earlier version of the sample code? The way it’s called it doesn’t look like it’d access theList at all.
The body of the function is omitted :-)
The idea of the goofy example is that it walks through theList to see if the existing entries are capitalized. If so, it capitalizes the string, if not it returns the string unmodified.
-Chris
···
On Aug 17, 2017, at 5:47 PM, William Jon Shipley <wjs@delicious-monster.com> wrote:
actor TableModel {
let mainActor : TheMainActor
var theList : [String] = {
didSet {
mainActor.updateTableView(theList)
}
}
init(mainActor: TheMainActor) { self.mainActor = mainActor }// this checks to see if all the entries in the list are capitalized:
// if so, it capitalize the string before returning it to encourage
// capitalization consistency in the list.
func prettify(_ x : String) -> String {
// ... details omitted, it just pokes theList directly ...
}actor func add(entry: String) {
theList.append(prettify(entry))
}
}
Oh, also, one relatively short term piece of this model is a proposal for adding an async/await model to Swift (in the form of general coroutine support). Joe Groff and I wrote up a proposal for this, here:
Concrete proposal for async semantics in Swift · GitHuband I have a PR with the first half of the implementation here:
Async await prototype by lattner · Pull Request #11501 · apple/swift · GitHubThe piece that is missing is code generation support.
Hi Chris et al,
This is definitely a great initial model.
To clarify, are the authors proponents of the syntax shown in the body of the draft--async, throws, async throws--or of the alternative design listed below that is "probably the right set of tradeoffs"--async, throws, async(nonthrowing)?
I believe that Joe is in favor of making async imply throws, but I’ll let him speak to why.
I’m weakly in favor of it, because I believe it will lead to a simpler system in practice. It seems that many async operations should also throw, so a design where async implies throw seems like it would skew to the right thing. Not just on the declaration side, but also on the marker side, because instead of “try await” everywhere, we could make await imply try. Another reason is that this would lead to a simpler actor model, where we’d probably just bake the “reliability” stuff directly into the base actor model, since all the async APIs can throw already.
That said, I don’t feel like I have enough data to have a strong opinion on it. The missing link for me is exactly how pervasive the non-throwing cocoa completion handlers are. If it turns out that people would have to confront "async(nonthrowing)” in practice (as opposed to being an obscure corner that the compiler implementers have to care about) then it is probably the wrong design.
Naively, to me, the observation that in your introduction you've separated `async` and `throws` suggests that keeping the two orthogonal to each other is the more teachable design. I appreciate how there is an intimate connection between `async` and `throws`, but it seems to me that `async(nonthrowing)` is a highly unintuitive result--_especially_ since the rationale for not outright making `async` a subtype of `throws` is that asynchronous non-throwing functions are a key Cocoa idiom.
Agreed.
Other than that, I would hope that the primitive functions `beginAsync`, etc., are indeed exposed outside the standard library; agree that their names could use some light bikeshedding :)
+1!
-Chris
···
On Aug 17, 2017, at 5:51 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
This is fantastic! Thanks for taking the time to write down your thoughts. It’s exciting to get a glimpse at the (possible) road ahead.
Happy to.
In the manifesto you talk about restrictions on passing functions across an actor message. You didn’t discuss pure functions, presumably because Swift doesn’t have them yet. I imagine that if (hopefully when) Swift has compiler support for verifying pure functions these would also be safe to pass across an actor message. Is that correct?
Correct. The proposal is specifically/intentionally designed to be light on type system additions, but there are many that could make it better in various ways. The logic for this approach is that I expect *a lot* of people will be writing mostly straight-forward concurrent code, and that goal is harmed by presenting significant type system hurdles for them to jump over, because that implies a higher learning curve.
This is why the proposal doesn’t focus on a provably memory safe system: If someone slaps “ValueSemantical” on a type that doesn’t obey, they will break the invariants of the system. There are lots of ways to solve that problem (e.g. the capabilities system in Pony) but it introduces a steep learning curve.
I haven’t thought a lot about practically getting pure functions into Swift, because it wasn’t clear what problems it would solve (which couldn’t be solved another way). You’re right though that this could be an interesting motivator.
The async / await proposal looks very nice. One minor syntax question - did you consider `async func` instead of placing `async` in the same syntactic location as `throws`? I can see arguments for both locations and am curious if you and Joe had any discussion about this.
I don’t think that Joe and I discussed that option. We discussed several other designs (including a more C# like model where async functions implicitly return Future), but he convinced me that it is better to focus language support on the coroutine transformation (leaving futures and other APIs to the library), then you pretty quickly want async to work the same way as throws (including marking etc). Once it works the same way, it follows that the syntax should be similar - particularly if async ends up implying throws.
That said, if you have a strong argument for why this is perhaps the wrong choice, lets talk about it!
-Chris
···
On Aug 17, 2017, at 7:39 PM, Matthew Johnson <matthew@anandabits.com> wrote:
Hi Chris,
I love what I’ve read so far. I just have one curiosity from my cursory look over the proposal.
It seems that in transitioning API to an Async Await system, the completion handler’s type becomes the return type. How would you handle bridging in Foundation API that already has a return type, eg URLSession returns you data tasks, and then fires a completion handler. Perhaps I missed something. I’m sure you could always form a tuple as well, just curious the thoughts on that.
Ah, e.g. this one:
extension URLSession {
/*
* data task convenience methods. These methods create tasks that
* bypass the normal delegate calls for response and data delivery,
* and provide a simple cancelable asynchronous interface to receiving
* data. Errors will be returned in the NSURLErrorDomain,
* see <Foundation/NSURLError.h>. The delegate, if any, will still be
* called for authentication challenges.
*/
open func dataTask(with request: URLRequest, completionHandler: @escaping (Data?, URLResponse?, Error?) -> Swift.Void) -> URLSessionDataTask
}Returning a tuple is an option, but probably wouldn’t provide the right semantics. The URLSessionDataTask is supposed to be available immediately, not just when the completion handler is called. The most conservative thing to do is to not have the importer change these. If they should have async semantics, they can be manually handled with an overlay.
In any case, I wasn’t aware of those APIs, thanks for pointing them out. I added a mention of this to the proposal!
Arguably these could be converted to return both the formal return value and a future. That would be a rather heroic rule for automatic import, though.
John.
···
On Aug 17, 2017, at 9:58 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
On Aug 17, 2017, at 4:56 PM, Rod Brown <rodney.brown6@icloud.com <mailto:rodney.brown6@icloud.com>> wrote:
-Chris
Thanks,
Rod
On 18 Aug 2017, at 8:25 am, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Aug 17, 2017, at 3:24 PM, Chris Lattner <clattner@nondot.org <mailto:clattner@nondot.org>> wrote:
Hi all,
As Ted mentioned in his email, it is great to finally kick off discussions for what concurrency should look like in Swift. This will surely be an epic multi-year journey, but it is more important to find the right design than to get there fast.
I’ve been advocating for a specific model involving async/await and actors for many years now. Handwaving only goes so far, so some folks asked me to write them down to make the discussion more helpful and concrete. While I hope these ideas help push the discussion on concurrency forward, this isn’t in any way meant to cut off other directions: in fact I hope it helps give proponents of other designs a model to follow: a discussion giving extensive rationale, combined with the long term story arc to show that the features fit together.
Anyway, here is the document, I hope it is useful, and I’d love to hear comments and suggestions for improvement:
Swift Concurrency Manifesto · GitHubOh, also, one relatively short term piece of this model is a proposal for adding an async/await model to Swift (in the form of general coroutine support). Joe Groff and I wrote up a proposal for this, here:
Concrete proposal for async semantics in Swift · GitHuband I have a PR with the first half of the implementation here:
Async await prototype by lattner · Pull Request #11501 · apple/swift · GitHubThe piece that is missing is code generation support.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
On Thu, Aug 17, 2017, at 10:04 PM, Chris Lattner via swift-evolution wrote:> That said, I don’t feel like I have enough data to have a strong
opinion on it. The missing link for me is exactly how pervasive the
non-throwing cocoa completion handlers are. If it turns out that
people would have to confront "async(nonthrowing)” in practice (as
opposed to being an obscure corner that the compiler implementers have
to care about) then it is probably the wrong design.
It may be helpful to look at data points outside of Cocoa. We use a
Future framework that distinguishes between plain old `Future<T>` and
`Future<Result<T>>`. In a back-of-the-hand calculation, about 10% of our
futures don’t use Result. It’s rarer, but it is useful; it aligns well
with the Error manifesto’s notion of a domain failures.
I think I’m overall in favor of an implicitly throwing model, though.
Not only is it cleaner, but it’ll simplify all sorts of abstractions.
f.ex: In our use of this framework, collecting up errors or
cancellations for an array of tasks is a perennial problem. `Future<T?>`
(and similarly `async -> T?`) can be plenty useful, but I’m not sure
they pull their weight enough to deserve occupying the best syntax.
(Huge, huge +1 to the whole shebang, though. Let’s just do this!!!)
Sincerely,
Zachary Waldowski
zach@waldowski.me
I, too, didn't really get these at first, although the code samples illuminated things some for me.
`beginAsync(_:)` is a sort of poor man's `Future`—it guarantees that the async function will start, but throws away the return value, and *might* throw away the error unless it happens to get thrown early. Given that its ability to return information from the body is so limited, I frankly don't think it's worth making this function rethrow only some errors. I would instead make it accept only a non-throwing `async` function, and if you need to call something that throws, you can pass an async closure with a `do`/`catch` block.
`suspendAsync(_:)` essentially converts the async calling into completion-based calling, as you said. I'm not sure if there's *literally* an implicit completion closure or if they're doing something else—the proposal doesn't lay out the transformations the compiler performs in much detail. I don't want to open this up to bikeshedding, but I sort of think of this function as `runAsyncWithCompletionHandler(_:)` or, more idiomatically, `withAsyncCompletion(do:)`.
···
On Aug 17, 2017, at 11:59 PM, Félix Cloutier via swift-evolution <swift-evolution@swift.org> wrote:
The purpose of `beginAsync` is to start an `async` function while firewalling against the propagation of the `async` keyword to the caller. `beginAsync` does not return a value. By the time `beginAsync` returns, whatever continuation closure was generated for the `body` parameter has been passed off to someone else and some scheduling mechanism has a reference to it and will call it later. That's not giving me too much trouble either.
I'm somewhat bugged by `suspendAsync`. My understanding is that it allows you to take a synchronous function which accepts a callback and turn it into an asynchronous one, where "asynchronous" means "continuation-driven" more that "asynchronously executing", because something like `let foo: Int = await suspendAsync { $0(42) }` looks like it should be legal (although not particularly useful).
The interaction of `suspendAsync` and `await` is that `suspendAsync`, like any other `async` function, accepts a hidden closure parameter, and `await` takes the rest of the function and turns it into a continuation closure to pass as that hidden parameter. The explicit parameters of `suspendAsync` are closures that accept the continuation (either for success or failure), so `suspendAsync` is the primitive that's responsible for translating the rest of an `async` function into something that you can pass as a callback.
That seems to make sense to me (although I might have it wrong), but I'm having trouble with the terminology. I'm not trying to start a bike shed debate here; it's simply not immediately clear to me what "suspendAsync" means for a function that seems more about starting an "old world" asynchronous task than suspending anything (let alone an asynchronous thing).
--
Brent Royal-Gordon
Architechies
Hi Chris,
I love what I’ve read so far. I just have one curiosity from my cursory look over the proposal.
It seems that in transitioning API to an Async Await system, the completion handler’s type becomes the return type. How would you handle bridging in Foundation API that already has a return type, eg URLSession returns you data tasks, and then fires a completion handler. Perhaps I missed something. I’m sure you could always form a tuple as well, just curious the thoughts on that.
Just as a follow up, thinking this through I’m not sure combining it with this completion handler argument tuple is helpful here, because you generally want access to the data task during execution for cancellation, progress handling etc.
The Async await system doesn’t seem to work well with handling setting something and allowing duplicate return types at different times like a lot of the foundation API is written, does it? I’m not sure how this is solved in C#.
···
On 18 Aug 2017, at 9:56 am, Rod Brown via swift-evolution <swift-evolution@swift.org> wrote:
Thanks,
Rod
On 18 Aug 2017, at 8:25 am, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
On Aug 17, 2017, at 3:24 PM, Chris Lattner <clattner@nondot.org> wrote:
Hi all,
As Ted mentioned in his email, it is great to finally kick off discussions for what concurrency should look like in Swift. This will surely be an epic multi-year journey, but it is more important to find the right design than to get there fast.
I’ve been advocating for a specific model involving async/await and actors for many years now. Handwaving only goes so far, so some folks asked me to write them down to make the discussion more helpful and concrete. While I hope these ideas help push the discussion on concurrency forward, this isn’t in any way meant to cut off other directions: in fact I hope it helps give proponents of other designs a model to follow: a discussion giving extensive rationale, combined with the long term story arc to show that the features fit together.
Anyway, here is the document, I hope it is useful, and I’d love to hear comments and suggestions for improvement:
Swift Concurrency Manifesto · GitHubOh, also, one relatively short term piece of this model is a proposal for adding an async/await model to Swift (in the form of general coroutine support). Joe Groff and I wrote up a proposal for this, here:
Concrete proposal for async semantics in Swift · GitHuband I have a PR with the first half of the implementation here:
Async await prototype by lattner · Pull Request #11501 · apple/swift · GitHubThe piece that is missing is code generation support.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
Hi Chris & swift-evo,
(Given the already lengthy thread I tried to separate my points and keep them reasonably short to allow people to skip points they don't care about. I'm very happy to expand on the points.)
Thanks very much for writing up your thoughts/proposal, I've been waiting to see the official kick-off for the concurrency discussions :).
I) Let's start with the async/await proposal. Personally I think this is the right direction for Swift given the reality that we need to interface with incredibly large existing code-bases and APIs. Further thoughts:
I think these points have been covered upthread, I’ll focus on the remaining topics:
II) the actor model part
-
Erlang runtime and the actor model go hand in hand
I really like the Erlang actor model but I don't think it can be separated from Erlang's runtime. The runtime offers green threads (which allow an actor to block without blocking an OS thread) and prevents you from sharing memory (which makes it possible to kill any actor at any point and still have a reliable system). I don't see these two things happening in Swift. To a lesser extend these issues are also present in Scala/Akka, the mitigate some of the problems by having Akka Streams. Akka Streams are important to establish back-pressure if you have faster producers than consumers. Note that we often can't control the producer, they might be on the other side of a network connection. So it's often very important to not read the available bytes to communicate to the kernel that we can't consumes bytes that fast. If we're networking with TCP the kernel can then use the TCP flow-control to signal to the other side that they better slow down (or else packets will be dropped and then need to be resent later).
Makes sense. The design I outline does talk about how to get to effective elimination of shared mutable state, so the reliability piece should work. As per greenthreads, I’m intentionally not getting into the runtime design, because there are lots of options and I’m not the best person to drive that discussion.
-
regarding fatal failure in actors
in the server world we need to be able to accept hundreds of thousands (millions) of connections at the same time. There are quite a few cases where these connections are long-lived and paused for most of the the time. So I don't really see the value in introducing a 'reliable' actor model where the system stops accepting new connections if one actor fatalError'd and then 'just' finishes up serving the existing connections.
This was one example of how to handle errors, not meant to be the only possibility.
So I believe there are only two possible routes: 1) treat it like C/C++ and make sure your code doesn't fatalError or the whole process blows up (what we have right now) 2) treat it like Erlang and let things die. IMHO Erlang wouldn't be successful if actors couldn't just die or couldn't be linked. Linking propagates failures to all linked processes. A common thing to do is to 1) spawn a new actor 2) link yourself to the newly spawned actor 3) send a message to that actor and at some point eventually await a reply message sent by the actor spawned earlier. As you mentioned in the writeup it is a problem if the actor doesn't actually reply which is why in Erlang you'd link them. The effect is that if the actor we spawned dies, any linked actor will die too which will the propagate the error to an appropriate place. That allows the programmer to control where an error should propagate too. I realise I'm doing a poor job in explaining what is best explained by documentation around Erlang: supervision [1] and the relationship between what Erlang calls a process (read 'actor') and errors [2].
Agreed. I’m a fan of the “just let them die” approach, along with providing notification out when it happens (so something out of band can handle it). That said, there is the problem of cleanup of the actor. If failure is frequent, then you will want to be able to restart the process at some point in time.
-
OS threads and actors
as you mention, the actor model only really works if you can spawn lots of them, so it's very important to be able to run hundreds of thousands of them on a number of OS threads pretty much equal to your number of cores. That's only straightforward if there are no (OS thread) blocking operations or at least it's really obvious what blocks and what doesn't. And that's not the case in Swift today and with GCD you really feel that pain. GCD does spawn threads for you and has a rather arbitrary limit of 64 OS threads (by default on macOS). That is too many for a very scalable server application but too few to just tolerate blocking APIs.
Agreed.
-Chris
···
On Aug 18, 2017, at 8:13 AM, Johannes Weiß <johannesweiss@apple.com> wrote:
## Conversion of imported Objective-C APIs
One issue I see with the importer is that the conventions for callbacks aren’t as strict as NSError ** methods were. For instance, URLSession completion blocks include response, data and error, all of which are optionals. The response is almost always present, even if there was an error. But there is no way to know that from the ObjC type system, and no way to represent a throwing function that also returns metadata on error in Swift.
Sure, the default would be to import these as just a bunch of optionals. Something better could be done on a case by case basis (in overlays) though.
How should we represent dataTask(with:completion:)?
Someone else brought this up. The most conservative thing is to leave it as a completion handler, and only transform void-returning ObjC methods.
## Actors
I fail to see the benefit of adding an entirely new construct for the actor model. It seems like access control, value semantics, and dispatch queues largely fill this need already, and the implimentation of an actor wouldn't be so complicated currently that it needs a compiler feature to ensure a correct implimentation.
It really comes down to whether you think elimination of shared mutable state is worth it, and if you think that having a higher level answer for concurrency is worth it. Reasonable people can differ on this, and I expect opinions to vary, because GCD and other existing things are pretty good. I think we can aim higher though.
-Chris
···
On Aug 18, 2017, at 11:56 AM, David Beck <david@davidbeck.co> wrote:
For instance, say you’re handling a button click, and you need to do a network request and then update the UI. In C# (using Xamarin.iOS as an example) you might write some code like this:
private async void HandleButtonClick(object sender, EventArgs e) {
var results = await GetStuffFromNetwork();
UpdateUI(results);
}This event handler is called on the UI thread, and the UpdateUI call must be done on the UI thread. The way async/await works in C# (by default) is that when your continuation is called it will be on the same synchronization context you started with. That means if you started on the UI thread you will resume on the UI thread. If you started on some thread pool then you will resume on that same thread pool.
I completely agree, I would love to see this because it is the most easy to reason about, and is implied by the syntax. I consider this to be a follow-on to the basic async/await proposal - part of the Objective-C importer work, as described here:
Another difference between the C# implementation and this proposal is the lack of futures. While I think it’s fair to be cautious about tying this proposal to any specific futures implementation or design, I feel like the value of tying it to some concept of futures was somewhat overlooked. For instance, in C# you could write a method with this signature:
...
The benefit of connecting the async/await feature to the concept of futures is that you can mix and match this code freely. The current proposal doesn’t seem to allow this.
The current proposal provides an underlying mechanism that you can build futures on, and gives an example. As shown, the experience using that futures API would work quite naturally and fit into Swift IMO.
-Chris
···
On Aug 18, 2017, at 12:34 PM, Adam Kemp <adam.kemp@apple.com> wrote:
Great writeup. Here's a few comments about the manifesto, actors and value semantics specifically.
# About actors and data isolation
Can I call global functions like sin(x) in an actor? Surely you need to be able to do that. But how can the compiler know which functions are safe to use and which are out of bound for an actor?
The design I outline allows you to call it, and punts on the problem of safety:
Swift Concurrency Manifesto · GitHubActors shouldn't access the global mutable state and thus should only call functions that do not access the global mutable state. We probably need a concept of a functions being pure (as in not accessing the global mutable state) for that.
I’m not aware of a practical model to achieve that, but if there were, more safety would provide more goodness :-)
# About the part "Does a type provide proper value semantics?"
I would argue that this is the wrong question. Most types are a hybrid form of value and reference semantics. All you need is to limit what you do to what is proper value semantics and forbid the parts that use reference semantics. Otherwise you are pessimising the capabilities of the types.
For instance, has Array<UIView> value semantics?
By the commonly accepted definition, Array<UIView> does not provide value semantics.
Dave Abrahams and I had an extremely long exchange at one point where he was taking the opposite position (if I understood it correctly). It would be nice to have the Swift community adopt a standard definition of value semantics and document it on swift.org <http://swift.org/>\. :)
···
On Aug 18, 2017, at 2:15 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
On Aug 18, 2017, at 7:19 AM, Michel Fortin <michel.fortin@michelf.ca <mailto:michel.fortin@michelf.ca>> wrote:
You might be tempted to say that it does not because it contains class references, but in reality that depends on what you do with those UIViews.
An aspect of the type (“does it have value semantics or not”) should not depend on the clients. By your definition, every type has value semantics if none of the mutating operations are called :-)
If you treat the class references as opaque pointers (never dereferencing them), you preserve value semantics. You can count the elements, shuffle them, all without dereferencing the UIViews it contains. Value semantics only end when you dereference the class references. And even then, there are some exceptions.
I agree with you that the model could permit all values to be sent in actor messages, but doing so would give up the principle advantages of mutable state separation. You’d have to do synchronization, you’d have the same bugs that have always existed, etc.
-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
The compiler could rewrite this:
print(await dataModel.getNumberOfEntries())
actor func getNumberOfEntries() -> Int
{
return theList.count
}
as this:
dataModel.getNumberOfEntries(_internalQueue) { count in
print(count)
}
actor func getNumberOfEntries(queue: DispatchQueue, handler: Int -> Void) -> Void
{
_internalQueue.async {
let count = theList.count
queue.async {
handler(count)
}
}
}
There is another problem that bothers me, if the function were to await on another actor (and therefore dispatch away from its _internalQueue), how would you guarantee that "only one message is processed by the actor at a time". You would need to somehow prevent the internal queue from processing other messages.
Thomas
···
On 18 Aug 2017, at 17:13, Johannes Weiß via swift-evolution <swift-evolution@swift.org> wrote:
GCD doesn't actually allow you to dispatch back to the original queue, so I find it unclear how you'd achieve that. IMHO the main reason is that conceptually at a given time you can be on more than one queue (nested q.sync{}/target queues). So which is 'the' current queue?
It’s hard to predict. IMO, ABI stability and concurrency are the most important things the community faces in the short term, so you’re looking at Swift 6 at the earliest, possibly 7 or 8….
-Chris
···
On Aug 18, 2017, at 8:18 PM, Andre <pyunpyun@mac.com> wrote:
If this was to be a feature at some point, when do you foresee a possible inclusion for it?
Swift 5,or 6 timeframe?
(Also, I notice that a fire-and-forget message can be thought of as an `async` method returning `Never`, even though the computation *does* terminate eventually. I'm not sure how to handle that, though)
Yeah, I think that actor methods deserve a bit of magic:
- Their bodies should be implicitly async, so they can call async methods without blocking their current queue or have to use beginAsync.
- However, if they are void “fire and forget” messages, I think the caller side should *not* have to use await on them, since enqueuing the message will not block.
I think we need to be a little careful here—the mere fact that a message returns `Void` doesn't mean the caller shouldn't wait until it's done to continue. For instance:
listActor.delete(at: index) // Void, so it doesn't wait
let count = await listActor.getCount() // But we want the count *after* the deletion!
Perhaps we should depend on the caller to use a future (or a `beginAsync(_:)` call) when they want to fire-and-forget? And yet sometimes a message truly *can't* tell you when it's finished, and we don't want APIs to over-promise on when they tell you they're done. I don't know.
I agree. That is one reason that I think it is important for it to have a (non-defaulted) protocol requirement. Requiring someone to implement some code is a good way to get them to think about the operation… at least a little bit.
I wondered if that might have been your reasoning.
That said, the design does not try to *guarantee* memory safety, so there will always be an opportunity for error.
True, but I think we could mitigate that by giving this protocol a relatively narrow purpose. If we eventually build three different features on `ValueSemantical`, we don't want all three of those features to break when someone abuses the protocol to gain access to actors.
I also worry that the type behavior of a protocol is a bad fit for `ValueSemantical`. Retroactive conformance to `ValueSemantical` is almost certain to be an unprincipled hack; subclasses can very easily lose the value-semantic behavior of their superclasses, but almost certainly can't have value semantics unless their superclasses do. And yet having `ValueSemantical` conformance somehow be uninherited would destroy Liskov substitutability.
Indeed. See NSArray vs NSMutableArray.
OTOH, I tend to think that retroactive conformance is really a good thing, particularly in the transition period where you’d be dealing with other people’s packages who haven’t adopted the model. You may be adopting it for their structs afterall.
An alternate approach would be to just say “no, you can’t do that. If you want to work around someone else’s problem, define a wrapper struct and mark it as ValueSemantical”. That design could also work.
Yeah, I think wrapper structs are a workable alternative to retroactive conformance.
What I basically envision (if we want to go with a general `ValueSemantical`-type solution) is that, rather than being a protocol, we would have a `value` keyword that went before the `enum`, `struct`, `class`, or `protocol` keyword. (This is somewhat similar to the proposed `moveonly` keyword.) It would not be valid before `extension`, except perhaps on a conditional extension that only applied when a generic or associated type was `value`, so retroactive conformance wouldn't really be possible. You could also use `value` in a generic constraint list just as you can use `class` there.
I'm not totally sure how to reconcile this with mutable subclasses, but I have a very vague sense it might be possible if `value` required some kind of *non*-inheritable initializer, and passing to a `value`-constrained parameter implicitly passed the value through that initializer. That is, if you had:
// As imported--in reality this would be an NS_SWIFT_VALUE_TYPE annotation on the Objective-C definition
value class NSArray: NSObject {
init(_ array: NSArray) { self = array.copy() as! NSArray }
}
Then Swift would implicitly add some code to an actor method like this:
actor Foo {
actor func bar(_ array: NSArray) {
let array = NSArray(array) // Note that this is always `NSArray`, not the dynamic subclass of it
}
}
Since Swift would always rely on the static (compile-time) type to decide which initializer to use, I *think* having `value` be non-inheritable wouldn't be a problem here.
It would be a perfectly valid design approach to implement actors as a framework or design pattern instead of as a first class language feature. You’d end up with something very close to Akka, which has provides a lot of the high level abstractions, but doesn’t nudge coders to do the right thing w.r.t. shared mutable state enough (IMO).
I agree that the language should nudge people into doing the right thing; I'm just not sure it shouldn't do the same for *all* async calls. But that's the next topic.
However, this would move the design of the magic protocol forward in the schedule, and might delay the deployment of async/await. If we *want* these restrictions on all async calls, that might be worth it, but if not, that's a problem.
I’m not sure it make sense either given the extensive completion handler based APIs, which take lots of non value type parameters.
Ah, interesting. For some reason I wasn't thinking that return values would be restricted like parameters, but I guess a return value is just a parameter to the continuation.
I guess what I'd say to that is:
1. I suspect that most completion handlers *do* take types with value semantics, even if they're classes.
2. I suspect that most completion handlers which *do* take non-value types are transferred, not shared, between the actors. If the ownership system allowed us to express that, we could carve out an exception for it.
3. As I've said, I also think there should be a way to disable the safety rules in other situations. This could be used in exceptional cases.
But are these three escape valves enough to make safe-types-only the default on all `async` calls? Maybe not.
To that end, I think failure handlers are the right approach. I also think we should make it clear that, once a failure handler is called, there is no saving the process—it is *going* to crash eventually. Maybe failure handlers are `Never`-returning functions, or maybe we simply make it clear that we're going to call `fatalError` after the failure handler runs, but in either case, a failure handler is a point of no return.
(In theory, a failure handler could keep things going by pulling some ridiculous shenanigans, like re-entering the runloop. We could try to prevent that with a time limit on failure handlers, but that seems like overengineering.)
I have a few points of confusion about failure handlers, though:
1. Who sets up a failure handler? The actor that might fail, or the actor which owns that actor?
I imagine it being something set up by the actor’s init method. That way the actor failure behavior is part of the contract the actor provides. Parameters to the init can be used by clients to customize that behavior.
Okay, so you imagine something vaguely like this (using a strawman syntax):
actor WebSupervisor {
var workers: [WebWorker] =
func addWorker() -> WebWorker {
let worker = WebWorker(supervisor: self)
workers.append(worker)
return worker
}
actor func restart(afterFailureIn failedWorker: WebWorker) {
stopListening()
launchNewProcess()
for worker in workers where worker !== failedWorker {
await worker.stop()
}
}
…
}
actor WebWorker {
actor init(supervisor: WebSupervisor) {
…
beforeFatalError { _self in
await _self.supervisor.restart(afterFailureIn: self)
}
}
…
}
I was thinking about something where `WebSupervisor.addWorker()` would register itself to be notified if the `WebResponder` crashed, but this way might be better.
···
On Aug 18, 2017, at 12:35 PM, Chris Lattner <clattner@nondot.org> wrote:
--
Brent Royal-Gordon
Architechies
For instance, has Array<UIView> value semantics?
By the commonly accepted definition, Array<UIView> does not provide value semantics.
You might be tempted to say that it does not because it contains class references, but in reality that depends on what you do with those UIViews.
An aspect of the type (“does it have value semantics or not”) should not depend on the clients. By your definition, every type has value semantics if none of the mutating operations are called :-)
No, not mutating operations. Access to mutable memory shared by multiple "values" is what breaks value semantics. You can get into this situation using pointers, object references, or global variables. It's all the same thing in the end: shared memory that can mutate.
For demonstration's sake, here's a silly example of how you can give Array<Int> literally the same semantics as Array<UIView>:
// shared UIView instances in global memory
var instances: [UIView] =
extension Array where Element == Int {
// append a new integer to the array pointing to our UIView instance
func append(view: UIView) {
self.append(instances.count)
instances.append(newValue)
}
// access views pointed to by the integers in the array
subscript(viewAt index: Int) -> UIView {
get {
return instances[self[index]]
}
set {
self[index] = instances.count
instances.append(newValue)
}
}
}
And now you need to worry about passing Array<Int> to other thread. ;-)
It does not really matter whether the array contains pointers or wether it contains indices into a global table: in both cases access to the same mutable memory is accessible through multiple copies of an array, and this is what breaks value semantics.
Types cannot enforce value semantics. Its the functions you choose to call that matters. This is especially important to realize in a language with extensions where you can't restrict what functions gets attached to a type.
If you treat the class references as opaque pointers (never dereferencing them), you preserve value semantics. You can count the elements, shuffle them, all without dereferencing the UIViews it contains. Value semantics only end when you dereference the class references. And even then, there are some exceptions.
I agree with you that the model could permit all values to be sent in actor messages, but doing so would give up the principle advantages of mutable state separation. You’d have to do synchronization, you’d have the same bugs that have always existed, etc.
What the compiler should aim at is enforcing useful rules when it comes to accessing shared mutable state.
···
--
Michel Fortin
https://michelf.ca
I have a question here. If the body calls an async method "without blocking their current queue", does that mean the queue gets to process other messages, even if the current message is not done with? Or should the queue somehow be suspended so new messages will not be processed?
Thomas
···
On 18 Aug 2017, at 21:35, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:
Yeah, I think that actor methods deserve a bit of magic:
- Their bodies should be implicitly async, so they can call async methods without blocking their current queue or have to use beginAsync.
H29/08/19 12:41、Chris Lattner <clattner@nondot.org>のメール:
If this was to be a feature at some point, when do you foresee a possible inclusion for it?
Swift 5,or 6 timeframe?It’s hard to predict. IMO, ABI stability and concurrency are the most important things the community faces in the short term, so you’re looking at Swift 6 at the earliest, possibly 7 or 8….
Got it, well I definitely look forward to it then.
Thanks!
Andre
···
On Aug 18, 2017, at 8:18 PM, Andre <pyunpyun@mac.com> wrote:
-Chris
what's important is that the code may pause for a while during a given expression and run other stuff in the meantime.
I think that’s what `yield` actually means. In you sentence there is nothing about (a)waiting, only about pausing and ‘yielding’ the cpu time to ‘run other stuff’.
-g.