A roadmap for improving Swift performance predictability: ARC improvements and ownership control

Yeah that's kind of what I'm worried about; that nobody is going to remember these rules (which might even change in the future as new cases are discovered and features added), and just think:

(I left out the part about caring about uniqueness; it's not always clear if you care about uniqueness - or at least, it's not always clear that you don't need to care).

So then users will think "Hey, I have this great idea for a new collection!", then they'll look at swift-collections (or package X), see a bunch of moves in almost every function across the entire library, and think "gosh, there's something really intricate going on here and I don't understand it; I'm out of my depth and shouldn't contribute to this package".

It may be an unavoidable consequence of achieving predictable performance, but it's still unfortunate.

Anyway, I'll wait for the lexical lifetimes proposal. From the brief description here, I really like it.


I wonder -- and this is a totally oddball idea that I absolutely have not thought through -- if the goal is just to communicate some performance assertions to the compiler that it could diagnose, why not just write those as some kind of assertion? In as close to real English as possible.

I guess this ultimately becomes a question of syntax and how well concepts are communicated in source code. The ideas floated in the move proposal about adding a drop(x) or assert(no longer using x) to mark the end of a lifetime is a lot more attractive in light of this manifesto.

// Instead of writing:
init(_ values: [String]) {
  // OMG what does 'move' do? This is some low-level performance intrinsic - eek!
  self.values = move(values)
  self.values.sort()
}

// You'd write something more like:
init(_ values: [String]) {
  self.values = values
  // Oh, ok, they want to check that 'values' is no longer being used.
  // It isn't required for correctness. It's just a check.
  #assert(no longer using 'values')
  self.values.sort()
}

This would be a lot more familiar to Swift developers who aren't ready to get in to the weeds about move semantics and ownership. It's just regular Swift, with some additional assertions to the compiler which are very approachable -- the reason for them being there might be a bit opaque, but no more opaque than having 'move' dotted about everywhere.

6 Likes

Overall, I'm thrilled to see such a detailed roadmap for performance-critical Swift! It's going to be a significant positive leap for the language.

On the topic of memory model and performance predictability, one thing that I wish could be added to this discussion is immortal data, such as data stored in static data segments. Currently, the only way to achieve this in Swift—aside from some specific types like StaticString that explicitly produce it—is to do an optimized build, but the ObjectOutliner SILOptimizer pass is entirely unpredictable to the average Swift developer.

I'm also very interested in hearing more about lexical lifetimes, because I agree with @Karl here; in the values sorting example, it seems like the compiler could detect that the local argument values is no longer used after the assignment and perform the move automatically, but that might run counter to the desired change of having lifetimes controlled by their lexical scope. While the more flexible lifetime does leave open the possibility of a performance hit if someone later adds code that uses the local values without thinking, it's also consistent with a philosophy of progressive disclosure: "get the best or near-best results writing the most straightforward code, then be explicit if you need to be later".

Essentially, I hope we can avoid some of the pitfalls that C++ has, where trying to do what you think is "the right thing" ends up making things worse. For example, a C++ user new to move-semantics might think "I have a function that returns a large collection, but I'll return std::move(...) it so it doesn't make a copy!", but then they've gone and undone the return-value optimization. So while this philosophy is something I agree with:

...it would be even better if Swift can do the performant thing by default without any explicit expression of intent; that's what would set its memory model apart from other languages like C++ and Rust.

Whenever the compiler is able to infer something about memory lifetime automatically that could be expressed explicitly, maybe there could be a way for users to enable "educational notes" as optional diagnostics, and the compiler would flag all those inferences to let the user know "by the way, if you want to ensure that you don't make a change that could cause problems later, you can add move here" or something to that effect. Memory models can be tricky to master, and giving authors guided diagnostics about how they could be more explicit, even if they don't need to be with the code written as it is at the moment, is just as important as giving them error diagnostics when they do something wrong.

18 Likes

One thing to keep in mind, it isn't so much that we are saying that the lifetime is controlled by the scope. What we are instead saying is that the value's lifetime will be at least the end of scope, but modulo specific deinitializer side-effects, the optimizer is allowed to shrink the lifetime. Deinit side-effects are the only side-effects that can cause this problem and act as a barrier against lifetime shrinking.

That is what assembly vision is: PSA: Compiler optimisation remarks - #6 by Michael_Gottesman. I am basically adding to this over time and hope to make it handle more and more cases over time.

9 Likes

My mental model thus far has been that a value’s lifetime may end starting when it is last referenced. Is it correct to say that that is still true, but has been joined by a guarantee that it will end when the scope is exited?

2 Likes

To me, immortal data seems intimately tied to compile-time evaluation. That probably means it is going to need more work yet, unless you just want to add more special cases like StaticString.

Yeah, move shouldn't have any runtime effect at all on its own; it should only have the compile-time effect of taking the moved variable out of scope, and as a side effect of that, allowing the bound value's lifetime to shrink to the new end of scope. That compile-time-only transparency should avoid pitfalls like the one @allevato mentioned of std::move in C++ occasionally defeating last-use optimizations if used improperly.

That's a fair concern. Our hope is that the lexical lifetime rules strike a balance where obvious code like this still does the right thing without explicit moves, so move (or however we end up spelling the operation) will still be something most Swift developers don't normally have to think about. But at the same time, performance-constrained developers who must get hard guarantees of their code's behavior really need something that will make the compiler raise diagnostics if those guarantees can't be met, for the reasons Andy laid out.

I think we will need move-only types to be able to fully compose types with move-only properties on the level of Rust. These proposals will get us closer to that point, at least, and many of them would be required to work with values and parts of structs and classes without requiring copies anyway, but the full composability in the type system won't be there yet.

I think this capability will be a natural outgrowth of compile-time constant values. Once we've established what a compile-time-constant expression is, then it would be a natural next step to be able to annotate an initializer expression as being required to be a constant expression.

It is my hope that Swift will still be able to continue to "do the right thing" by default in most situations. As I noted above, our design for move should hopefully avoid the pitfalls of move in C++ being able to accidentally defeat the desired optimization. It could well be in practice, people end up spraying move everywhere out of an abundance of caution, but I hope that they won't need to.

In practice, the guarantee that it will end before or at the point the scope exits has always been there. We are planning to introduce new barriers that limit how far back a value's lifetime may end, so that it not only includes the final use in source code, but also things like weak reference loads and memory accesses through UnsafePointer that tend to rely on the lifetime of other variables to get correct behavior.

13 Likes

I actually have been working on a new collection for that package, and I’ve run into that sort of issue with all the unstable features I’m expected to use. I’ve mostly copied existing uses and winged it, which is not a good feeling given the circumstances.

It is going to be especially important that these new tools are well-documented, such that anyone who stumbles upon them will come away with a concrete understanding of when to use them and why.

For the sake of people encountering them for the first time, I still think they need to have more descriptive names. I don’t think MemoryLayout.size is particularly intimidating despite being fairly low-level, simply because the meaning is obvious at the call site. You read the documentation of MemoryLayout and you’re pretty much good to go. That’s what we need for move, copy, and other low-level operations.

I’d contrast that with existing top-level methods like assert, precondition, and fatalError, none of which quite explain when you should use them and why. They’re also fairly hard to discover, as they have no semantic relation to each other. Newcomers to Swift tend to litter fatalError everywhere, which is not ideal.

7 Likes

This is a stunningly elegant idea, and I look forward to applying it whenever applicable. It seems like a potent tool in the compiler optimization toolbox, especially for non-inlinable function calls.

For the sake of orthogonality, will we be able to mark closures as @nonescaping and non-closures as @escaping? I’m thinking that would act basically the same way internal does: making a default explicit for the sake of readability at the programmer’s discretion.

6 Likes

Would @nonescaping be usable for Iterators? It doesn't sound like it, but it'd be nice if it could be. One optimization we've wanted to make in some places is to reuse a single object which is mutated each time rather than allocating new objects that are then immediately disposed. For obj-c types (where we can't use isKnownUniquelyReferenced) it'd be very error-prone to do this, but @nonescaping could maybe address that.

Optimization often won't be fully effective in situations like the one above without the programmer taking some step to opt-out of lexical variable lifetimes. There are just too many ways that side effects can be hidden from the compiler.

However, the compiler can tell you when a copy is a good optimization candidate via performance diagnostics (see discussion of AssemblyVision). Those diagnostics can point you to one of a few solutions. High-performance libraries will use any one of these opt-out mechanisms:

  • @nonescaping
  • @noImplicitCopy
  • move function/operator

I expect @nonescaping to be the most common over time.

@noImplicitCopy would be the next option, for variables that do explicitly escape. This has the effect of a lifetime assertion.

move is only useful if neither of those attributes are already used. It is an important primitive. But I don't expect to see widespread use of move given that the other attributes should be available soon after.

With move-only types, @nonescaping will be the only relevant annotation. And move will obviously be a no-op.

[EDIT] In rereading some responses I see where confusion is creeping in. This proposal does not make our current ARC optimization less effective. It simply says that the current approach to optimization is sometimes insufficient for predictable performance.

2 Likes

Wouldn’t that be a use for inout? IteratorProtocol doesn’t require anything with parameters to begin with.

I suspect optimizer effects will be formally established by the time Swift 7 rolls around. They seem pretty much nonnegotiable for extreme optimizations like compile-time evaluation or even automatic parallelization. Sendable already inches toward that space.

I agree: that seems like the sort of thing that you’d always want to use, except when you are reluctant to commit to that in your API. A bit like @inlinable.

1 Like

This is not part of the roadmap or any current proposal. But we do expect to order deinitialization with respect to "external function calls". And to me it only makes sense for that ordering to be bidirectional. So, a print statement in a deinitializer should be ordered relative to print statements before and after the end of the scope.

I'm not really sure that @nonescaping will be able to scale to any meaningful extent unless we go for something more like Rust's lifetime system, even with the extensions described here.

A simple example, LazyMapCollection:

struct LazyMapCollection<Source, Result> where Source ... {
  var source: Source
  var transform: (Source.Element) -> Result
}

As I understand it, even with these extensions, I won't be able to use a @nonescaping closure in this type. There's no way to propagate that the overall type becomes @nonescaping if the closure is.

Now, I'm using a closure here, but it could be a class, or refcounted COW storage, or a pointer to a stack buffer. The things described in this proposal are nice (don't get me wrong - these are some very welcome improvements, and I'm excited about them!)... but fundamentally, it is still quite limited. The demonstrations are quite basic, and I'm not sure how many issues it will solve in more complex, real-world code.

EDIT: Oh, and presumably you wouldn't be able to use this with a @nonescaping source collection either. So even if I could tag my function's argument as not escaping and have it stack-allocated, lots of important features wouldn't work with it -- because the compiler doesn't necessarily know that array.lazy.map { ... } won't escape the array.

Also, I'm not suggesting this is a trivial problem to solve. I only know of Rust that has managed it, and it comes with some severe usability costs.

3 Likes

This is absolutely fantastic!

Are arguments to case constructors also consumed?

@Joe_Groff may I propose one possible alternative name to ref. If we want a keyword that is as short as let and var while it should enforce some special behavior, how about "use"?

use is somewhat already fairly close to borrow, so it seems to me that it could actually fit naturally.

Yet another alternative names could be "cut" or "own".

4 Likes

Could variables explicitly marked as consuming be moved automatically and require explicit copy? That could help remove a lot of move() cluttering.

Can @noImplicitCopy be used with property variables as well?

What an awesome roadmap, very exciting!

Just for my understanding, doesn't using the consuming keyword mean that you won't need to copy the marked argument? And that you therefore don't need to use move within the scope of the function.

I don't know if I have it right but this pitch doesn't seem to have consuming variables be moved automatically.

I don’t think anyone is ever going to be using move regularly. In fact, I think it should be moved inside a caseless enumeration with a name like Binding, á la MemoryLayout, so it is easier to understand when seen for the first time.

1 Like