Swift project focus areas in 2023

Announcements
1

There is a new blog post on Swift.org that highlights focus areas in the various workgroups and areas of development in 2023:

Please feel free to discuss the blog post here.

52 Likes
2

A couple of questions:

  1. Concurrency

    Does this include custom executors? I remember there was also talk about adding locking primitives and such; is any of that in scope for 2023?

  2. "basic support for non-copyable types"

    Does this include standard library changes? Or just the language support for declaring such types? How basic is basic?

  3. Reflection

    I think better reflection APIs are a pretty major feature that are worth drawing attention to. Presumably they are in-scope since there's already a pitch. Some developers have been asking for things like runtime-discoverable attributes as well, and it seems there may be some early exploration about that. It would be nice to know what the scope of the plans are.

    Whenever you're ready. I'm just surprised it wasn't mentioned.

  4. Side-effect and escape analysis.

    I've noticed some of this land already and I think it's really exciting - I've always wondered why the compiler didn't do this. Is there some way to experiment with it?

10 Likes
4

We're still figuring that out. :slightly_smiling_face: Expect to see more pitches and discussion of this in the very near future.

To elaborate a bit on the sketch I posted here, Joe's roadmap vision, and John's "Ownership Manifesto" there are a bunch of things we need to sort out:

  1. New language features to provide ways to manage data without copying. (SE-0366) (SE-0377) (@noImplicitCopy) (others coming soon)
  2. Concrete move-only types that don't involve generics
  3. Support for move-only types as generic type arguments
  4. New standard library features (such as BufferView)
  5. Retrofitting existing standard library support (which rests heavily on generic support, of course)

And of course, we need to ensure that the optimizer is fully aware of all this new machinery as we proceed.

The first item above has proven to be larger than I initially expected. In retrospect, it's pretty obvious that "non-copyable types" aren't all that useful if the basic language constructs expect to copy everything. :thinking: We may be able to change a few core constructs to eliminate the copying assumptions, but we'll also need to extend the language with new operations such as the ones linked above.

As those fundamentals come together, we should soon be able to show prototypes of actual working move-only types (#2). And we think there are a few places in the standard library where we can use move-only types without generics (parts of #4).

But I am not optimistic that we'll get everything above done this year. In particular, the generic support (#3 and #5) is going to require some delicate surgery to Swift's type system and we want to take time to make sure we get that right.

Tim

10 Likes
5

We're interested in adding the ability for actors to customize their executor in about this timeframe. Other things covered by my old custom executors pitch are not currently planned.

We'll probably want locks to be a move-only type, so we need to resolve that dependency first.

More generally, as a project I think we want to take a year or so to get a better sense of how people are using the current concurrency features and what problems they're running into before we start planning any specific feature work.

7 Likes
6

I did want to drop by to say congratulations (and condolences, for the increased workload) to the esteemed @mishal_shah!

17 Likes
7

Thanks for this, very helpful.

Only one question: does Concurrency include reasync?

2 Likes
8

The Core Team is investigating creating several more workgroups, including one dedicated to improving Swift’s usability across platforms.

I think the cross platform aspect is (or should be) a very important aspect even from the perspective of Apple who is — for obvious reasons — the largest incubator for Swift (sorry for the quite simplified language in what follows):

  • A programming language has to work properly on “the important platforms” to be “taken seriously”,
  • and Apple should have interest in using a programming language that is indeed “taken seriously” (even outside the Apple world).

It seems that this is well understood and I think Swift has indeed a great future even outside the Apple world (server-side Swift is a first indicator for this).

5 Likes
9

Would this include primary associated types for asynchronous sequences? I know this would potentially open the ‘typed throws’ can of worms, but think it would greatly improve the ergonomics and help adoption of structured concurrency more generally. I think it’s something people come up against relatively quickly as they attempt to use the feature.

(FWIW I’m -1 on typed throws as feel the additional complexity outweighs the control flow benefits, but I realise others feel differently.)

2 Likes
10

Could this part of custom executors possibly address the lack of FIFO-ness in some async areas?

11

Custom actor executors can absolutely be made FIFO. That said, I think it’s important to understand what that means. One of the main arguments that ultimately convinced us to not make Swift actors FIFO by default is that it’s really hard to actually turn that guarantee into anything useful at a higher level. That’s still going to be true with custom executors.

When you’re doing explicit enqueuing, having FIFO serial queues allows you to propagate the well-orderedness of a stream of events across an arbitrary sequence of queues. Suppose that serial queue A processes an event and ends by enqueuing on serial queue B. FIFO queues will ensure that both queues process the events in the same order, and by induction this works across any number of queues. But this relies on going directly between an identical series of serial queues; any transit through non-serial execution, or skipping an intermediate queue, completely fouls up that semantic property.

The way that Swift async functions are scheduled is really not conducive to maintaining that property. Swift async functions really like to briefly transit through non-serial executors, and optimization really wants to avoid doing jumps to actors where you’re not going to do anything. We could try to harden those rules and give more direct control, e.g. to allow new Tasks to be explicitly enqueued on a specific actor. But that feels like it would be a very brittle way to solve that problem; you’d have to understand a lot of implementation details to even try it, and it would probably be an exercise in frustration in practice.

We’re aware that the current language tools for maintaining event order aren’t great, and that’s definitely something we want to improve. But I don’t think FIFO actors actually get you there very well.

15 Likes
12

Thanks for clarifying!

One thing I'd like to ask about specifically is generator functions. They were mentioned briefly in the ownership manifesto, but in all the discussions we've had in the years since then, about move-only types and extra control of ARC, etc, they've never come up again.

I hope they are still on the agenda - it doesn't seem like IteratorProtocol's next() -> Element requirement would work for a non-copyable Element type.

But more than that, I think generators are just plain better than iterators, even for copyable types. As your processing grows in complexity and branchy-ness, it becomes really difficult to manage state in the iterator's instance variables. The fact that you need to return after each element becomes a burden, because you'll need to somehow restore to exactly where you were on the subsequent call to next(). All of your local variables, all of the branches you took, all gone after every element.

Generators don't have that problem (because coroutines), and they're especially good at defining async streams because you do often want to perform complex processing in an async function, yielding elements occasionally, while allowing others to compose those streams, and for the result to be consumable with a simple for-loop. It's possible to do all that with iterators, it's just really, really awkward.

Is that a discussion I should look forward to in the not-incredibly-distant future? Or is it unlikely any time soon?

10 Likes
13

We’ve been looking at iteration over move-only collections (and the challenges of optimizing it even for copyable collections) and thinking more-or-less the same thing, so yes, I don’t think generators are terribly far away on the roadmap. One thing at a time, though.

13 Likes
14

It’s a pity more reflection capabilities aren't on that roadmap. Or are they already prioritized somewhere else?

15

Would love to see Generators as well! I brought up some unavoidable use of unstructured concurrency in async-algos recently AsyncSequence algorithms + structured Concurrency. These kind of algos could be implemented completely structured with generators as far as I can see.
Though I agree, one thing at a time :grinning:

4 Likes
16

There is actually an effort to provide better reflection APIs, but it didn’t make the list. Here’s a recent pitch.

2 Likes
17

Oh yes, I'm familiar. I just wish it was on the Big List :wink:

1 Like
18

When I use Swift, I tend to go with protocol-oriented, functional-style programming. (I've been writing a bunch since retiring.) Although I love the localized reasoning about runtime causality that obtains, every so often I lose a lot of time to a type refactoring and consequent compile-time-error debugging. I often end up just backing out and duplicating code after trying hard for days. For example, protocols support for separation of concerns, but when there are conflicting definitions found in protocol extensions, Xcode provides very poor support. (In this case I want a lattice view of a slice through the protocols.)
To whatever extent my experience is typical, I would want effort spent on a better compile-time error experience. Even though it has improved a bunch--thanks, team!--I think there is so much opportunity out there, and the work will have to span compiler and tooling. Thank you for your consideration.

18 Likes
19

One thing I am curious about, as Swift, Concurrency, Combine, and SwiftUI all mature, is back-deployment of of features and Frameworks being a focus with more and more work being done to make it easier and easier reducing fragmentation for developers?

This seems like something that is a bit absent than from the roadmap, but perhaps it is a strong part of it. If so could you please expand on it?

3 Likes
20

I completely agreeAnd currently, there is an abundance of embedded devices, and for such devices, writing secure and reliable code is crucial. Swift has the capability and has proven itself to be up to the task. I genuinely hope that Swift can see more development and application in the cross-platform domain.

3 Likes
21

One thing we’ve seen performance issues with is keypaths which was called out here:

  • Implement performance improvements to KeyPaths. While not strictly part of Differentiable Swift, key paths become extremely important when optimizing strongly-typed models for introspection. As a first step, there is an effort to add a robust set of key path benchmarks to the compiler suite.

Are there any pitches or PRs related to this that someone knows of?

2 Likes