SE-0446: Nonescapable Types

Evolution Proposal Reviews
21

The flag is currently "experimental." Once the proposal is accepted, it will be promoted to "upcoming" and eventually be enabled by default.

2 Likes
22
  • What is your evaluation of the proposal?

This is a +1 from me.

  • Is the problem being addressed significant enough to warrant a change to Swift?

While I don't expect to actively write ~Escapable types myself anytime soon, having the targeted Span types is something that even a lot of non-library authors will benefit from.

  • Does this proposal fit well with the feel and direction of Swift?

IMO yes, as it follows the ~Copyable concept neatly. The only, minor, thing is perhaps the similarities-yet-slightly-different-spelling with @escaping closures. On the other hand, since developers encounter those way earlier, this perhaps makes understanding it a little easier than ~Copyable. The latter is admittedly often difficult to understand ("it's suppressing the invisible default Copyable conformance, not adding a NonCopyable protocol!").

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I can only compare it to my super-old, super inexperiences C days, when I had to deal with any form of ownership and "what scope deallocs this?" questions by hand. Obviously this beats that in terms of safety.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I read it carefully and even refreshed the related ~Copyable proposals when needed.

1 Like
23

I'm sorry if this was discussed before but why are we making "escapability" a property of a type rather than a property of a binding (with the @nonescaping attribute or some other type qualifier)?
The escapability property makes sense when we pass a value into a function, so the caller can reason about the lifetime of the passed value and this reasoning doesn't depend on the type itself.
Once we have explicit lifetime dependencies, couldn't we model the Span type as the existing UnsafeBufferPointer<T> type with an additional qualifier restricting its lifetime (aka dependsOn(xxx))?
By specifying the escapability next to the function argument, we can use it for any ref-types, allowing the optimizer to allocate such objects on the stack in cross-module calls(without @inlinable). We could also use the familiar withoutActuallyEscaping to pass such values into functions that don't support non-escaping arguments yet but don't escape them actually. And also this will allow us to generalize the @escaping attribute over all types, solving some issues with macros.

I'm not against ~Escapable, I'm just wondering if the generalized @nonescaping attribute would be better.

3 Likes
24

This is a big +1 from me. I couldn't get it to work in my use cases yet though, I think because the feature implementation needs some more tweaking.

25

This would be an extremely useful feature. We would be able work with visitor patterns, passing a closure that can access 'inout' variables and other ~Escapable data. It's not in the proposal because it isn't implemented.

I should point out that this does have implications for exclusivity checking: the NRR rule will need to cover parameters of ~Escapable type that may contain a closure:
Restrictions on recursive uses of non-escaping closures

I don't think it's a serious restriction. Here's a little example of what would break without that restriction:

struct NEClosure: ~Escapable {
  var closure: @nonescaping () -> ()
}

func recursiveApply(arg: NEClosure, apply: (() -> ()) -> ()) {
  apply(arg.closure)
}

func violateCapturedInoutExclusivity(x: inout Int) {
  recursiveApply(arg: NEClosure(closure: { x = 0 }),
                 apply: { (f: ()->()) in
                   x = 1
                   f()
                   assert(x == 1)
                 })
}
5 Likes
26

If there are no source compatibilities introduced by this proposal, it should not need an upcoming feature flag at all.

The upcoming feature flag is to enable syntax or behavior that is source-breaking when the proposal is implemented in a release.

Having the 'Upcoming Feature Flag' header field in the proposal means you intend there to be a flag that exists in released versions of Swift once this proposal is implemented. It shows up on the Evolution Dashboard and it is confusing for developers to see a proposal has a flag when none is really intended.

If you are updating the proposal text, please remove the upcoming feature flag heading.

2 Likes
27

I have to say I feel the opposite of vns: I don’t feel like I can properly evaluate this new feature without seeing what the lifetime part looks like. There are a number of different forms that could take, and there’s a chance—if a small one—we / the Core Team might not find any of them acceptable. To speak frankly, as much as I want this control, I don’t think I want to accept this big chunk of additional complexity in the language, especially for new learners, without seeing the full size of the feature. And returning these values is part of that.

As a small addendum, Andy invented a syntax above for @nonescaping closures stored in structs (or enums). That seems like a useful thing, which will need a formally-approved syntax…but that doesn’t need to go in this proposal.

5 Likes
28

We did have a discussion with the proposal authors and the Language Steering Group about what interactions this subset might have with a full lifetime dependency model before beginning this review, and what behaviors it locks in as the "default":

  • When a function receives an nonescaping value as a parameter, that value has a lifetime dependency that at least covers the lifetime of the function's execution. That feels like a safe default behavior, and one could argue it's "obvious" and the only way things could possibly work, though it's imaginable that a flexible enough model could tie the lifetime of a value to the scope of running another function parameter received as a parameter, as a way to say "you can't use this value, except to pass it over here".
  • There's a subtler question about the lifetime behavior of higher-order functions that pass nonescaping value(s) to their function parameters: if the function is invoked more than once, does each invocation run under a common scope, or does each invocation run independently? The former would be akin to a Rust declaration
     fn foo<'a>(body: impl Fn(Param<'a>) -> ())
    and this could allow for the body closure to pass nonescaping information between invocations of itself, since each invocation would be understood to occur in the same scope:
    let mut carry_over: Option<Param> = None;
foo(|param| {
  use(carry_over, param);
  carry_over = Some(param);
});
    This might be useful for something like a forEachSpan operation on an in-memory data structure, where the caller may want to track multiple spans during the computation, but on the other hand, the model would rule out a function that transiently produced spans and only kept each span alive for the duration of a single body invocation. That would be better modeled by the latter case, which would correspond to a Rust declaration like:
    fn foo(body: impl for<'a> Fn(Param<'a>) -> ())
    Since this is the more conservative API guarantee, it feels like the right default, and having this behavior now doesn't foreclose on a more general lifetime feature allowing the same-lifetime-for-each-invocation model to be expressed in the future.

If we never planned to support lifetime dependencies at all, there's technically still a way within the bounds of these strictly-downward-scoped nonescaping types to make a complete feature out of it, since initializers could primitively produce nonescaping values by passing the initialized value down and then unconditionally failing:

struct Span: ~Escapable {
  var start: UnsafeRawPointer, count: Int

  private init(_unsafeStart: UnsafeRawPointer, count: Int, body: (Span) -> Void) throws {
    self.start = _unsafeStart
    self.count = count
    body(self)
    throw Sike()
  }
}

That's awful, of course, and we would never actually leave the feature at a point where you have to do that since we know we do want to support proper lifetime dependencies (and if we didn't, there would hopefully at least be some accommodation to have these downward-passing initializers exist as more than just a fluke of existing behavior), but I think it serves as an example that it is possible to consider only strictly-downward-scoped nonescaping types as a feature on its own.

4 Likes
29

I am in favor of this proposal.

It adds a very useful capability to Swift that will allow creating better and safer APIs. I look forward to making use of this in my own code in various places.

On its own this proposal isn't sufficient of course but I have no doubts that the various follow-up proposals will flesh this out, similar to what was previously done with e.g. concurrency or noncopyable types.
This proposal lays out a direction I am in favor of and much of the mechanics straightforwardly mirror ~Copyable.

I have previously read pitches in this area and read this proposal and the Span proposal for this review.
I wanted to try the toolchain with an existing usecase for ~Escapable in a personal project but unfortunately ran into an unrelated bug.

30

I share this sentiment. For example, Rust does not need the concept of non-escapable types, because it is always clear from the lifetime annotations what can escape and what cannot. If Swift adopted a similar approach (lifetime parameters are the part of the type), that would potentially make ~Escapable redundant. Committing to this proposal definitely suggests what direction Swift lifetimes are going in and cannot be completely decoupled.

That being said, if there is a consensus for the general directions for lifetime annotations, we might not need every detail (like syntax) to be worked out before discussing ~Escapable.

31

Rust does have the concept of 'static types. If you declare a generic requirement T: 'static, that indicates that T cannot have any lifetime variables and is lifetime independent. This is more or less also what T: Escapable indicates in Swift (though the default polarity is opposite from Rust, as with Copyable), that values of type T are never subject to lifetime constraints, and that's independent of how lifetime constraints ultimately get expressed.

8 Likes
32

While this is true, I don't think this is comparable to Escapability in Swift for a couple of reasons:

  1. I rarely see T : 'static in real world Rust code, in fact I never had spell this out despite writing non-trivial amount of Rust in the past. On the other hand, most code interacting with lifetimes would refer to escapability in Swift.
  2. I think this only reinforces my point. Rust does not need the concept of escapability, because they can express it with lifetime annotations. In case we have the same expressivity for lifetimes in Swift, we do not need the concept of escapability.
33

Nonetheless, there are implicit constraints on what you can do with an unadorned type variable in Rust because of the need to preserve any potential lifetime constraints in the substituted type, and Swift as it exists today does not impose those constraints on type variables by default. So there does need to be some way to indicate whether a type variable may be bound to a type with lifetime constraints. The "concept of escapability" is merely requiring the lack of lifetime constraints and exists in both languages even if Rust doesn't call it that.

3 Likes
34

Thanks! This sounds like a strong justification for this concept. Are these implicit constraints spelled out somewhere?

Edit:

It sounds like this feature is the result of prioritizing source compatibility and progressive disclosure. We do not want to require annotations when something is escaping, we want people to opt in into lifetime annotations.

35

One example where you'd need to specify an explicit lifetime requirement on a type variable is when a value whose type involves the variable gets converted to a type that doesn't, such as when using dyn Trait types. This will raise an error that "T may not live long enough":

pub trait Foo {}

pub fn foo<T: Foo>(x: T) -> Box<dyn Foo> {
    Box::new(x) // error
}

To solve that, you can indicate that either T is required to be 'static:

pub trait Foo {}

pub fn foo<T: Foo + 'static>(x: T) -> Box<dyn Foo> {
    Box::new(x) // OK
}

or that Foo only admits 'static impls:

pub trait Foo: 'static {}

pub fn foo<T: Foo>(x: T) -> Box<dyn Foo> {
    Box::new(x) // OK
}

or explicitly carry whatever lifetime dependency from the type variable to the dyn:

pub trait Foo {}

pub fn foo<'a, T: Foo + 'a>(x: T) -> Box<dyn Foo + 'a> {
    Box::new(x)
}

Anecdotally, I've heard that explicit 'static requirements come up often in async code, since Future implementations often need to be enqueued in global data structures.

2 Likes
36

It is possible, though, that if we had a system of named lifetimes on par with Rust's in the future, that we would also want a way to specify an arbitrary lifetime as a lower bound for a type to satisfy. The 'static requirement in Rust is a specific case of the more general ability to require any lifetime to be a lower bound for the lifetime of a type by T: 'a requirements. However, since Escapable in Swift has no runtime manifestation, we could still transition to such a model by turning Escapable into an alias like:

typealias Escapable = any 'static

or however we chose to spell it.

7 Likes
37

Thanks for your feedback, everyone! The Language Steering Group has decided to accept this proposal with modifications.

4 Likes