[Pitch #3] Compile-time lifetime dependency annotations

Evolution Pitches
1

Hi Swift Evolution. Lifetime dependencies were introduced as an experimental feature for the development of safe APIs such as Span and MutableSpan. Since the previous pitch, the feature has improved, and several language improvements have been built directly on top of lifetime dependencies:

This revised proposal serves both to document the state of the experimental feature and as a place to continue discussion about its use and design:

The @_lifetime attribute is currently available under the Lifetimes experimental feature.

5 Likes
2

I understand this feature is still in the development/pitch phase, but I think this is an important question to ask early. Have the authors of the pitch thought about how they intend to teach Swift developers about this feature?

Lifetimes are easily one of the most tricky parts of Rust to understand. However in practice I've found that teaching them to both inexperienced and experienced developers to be easier than teaching many Swift concepts — largely because of how well the Rust Book approaches the topic.

I would urge the authors to consider what supplemental material must be made alongside the implementation in the compiler and adoption in the standard library. Its likely docc could use improvements to aid writing great documentation on this topic — for example, visualizing how ownership flows between parameters and return values across an API surface. Without that, even well-written documentation will struggle to communicate these contracts to API consumers.

16 Likes
3

I have some questions about @_lifetime(borrow xx).

I fully understand that the dependency source xx must have borrowing semantics, it cannot be consuming or inout. At the same time -- If I remember it correctly -- the default ownership semantics of parameters of methods and getters are already borrowing, and for the initializer parameters they are consuming.

Question 1:
I surveyed the standard library and found that @_lifetime(borrow xx) usually comes with an explicit borrowing specifier, like this:

extension Array {
  public var span: Span<Element> {
    @lifetime(borrow self)
    borrowing get {
      // ...
    }
}

To my understanding, Array is copyable, and self is already borrowing here, so the only effect of specifying the getter to be borrowing is to avoid any accident copy self in the body of the getter. But the ownership rules already ban people from deliberately writing this:

@lifetime(borrow self)
get {
  let copied = self
  return /* some expression that references copied.xxx */    // ❌ error
}

So my questions is, is adding borrowing here just a coding style thing, or does it actually help to avoid programming errors in practice?

Question 2:
In the standard library I also found some initializer parameters do not follow the borrowing-semantics rule, for example:

public struct Span {
  @lifetime(borrow pointer)
  init(
    _unchecked pointer: UnsafeRawPointer?,
    count: Int
  ) {
    _pointer = pointer
    _count = count
  }
}

the pointer parameter has consuming semantics by default, but the compiler is happy. I though maybe its because UnsafeRawPointer is Escapable & BitwiseCopyable, but that suspicion quickly proved to be wrong, because I can write this

struct A: ~Escapable {
    let value: Int
}

extension A {
  @_lifetime(borrow value)
  init(_ value: consuming Int) {   // ✅
    self.value = value
  }
}

But not this:

extension A {
  @_lifetime(borrow value)
  init(_ value: consuming Int) {  // ❌ lifetime-dependent variable 'self' escapes its scope
    self.init(value: value)
  }
}

Does is mean there exist a special rule for designated initializers?

4 
extension Array {
  public var span: Span<Element> {
    @_lifetime(borrow self)
    borrowing get {...}
  }
}

Note that the @_lifetime(borrow self) annotation is redundant on two levels. borrow is the only kind of dependency you can have here, so the keyword isn't required. And borrow self is already the default for an accessor (or any method with no parameters).

So my questions is, is adding borrowing here just a coding style thing, or does it actually help to avoid programming errors in practice?

Your question is really about the borrowing ownership specifier, which is mostly unrelated to the lifetime. You're right, it's also the default. But specifying it explicitly does enable implicit-copy diagnostics. So if you accidentally did this:

    borrowing get {    // error: 'self' is borrowed and cannot be consumed
      self.dropFirst() // note: consumed here
      ...
    }

You get an error telling you that the consuming method is actually an implicit copy. Basically, if you're being explicit about ownership, you also need to be explicit about the copies.

In short, the standard library authors are just being as explicit as possible.

4 Likes
5

Thanks, this clears up many things for me. Can you have a look at my Question 2?

6

I also found some initializer parameters do not follow the borrowing-semantics rule

These initializers are the unsafe part of Span's implementation. Span's Escapable fields can be initialized without any lifetime dependency.

Similarly, with this example:

struct A: ~Escapable {
    let value: Int
}

Direct assignment to A.value doesn't require any dependency because Int is Escapable. It's the same reason that this works:

extension A {
  @_lifetime(immortal)
  init() {
    self.value = 3
  }
}

The initializer of a type whose fields are all Escapable only specifies a lifetime requirement to enforce safety on the caller side. The non-Escapability of a data type like Span is first enforced in the code that calls into initializer.

In your next example, you are calling into an initializer with a borrowed value:

  @_lifetime(borrow value)
  init(_ value: consuming Int) {  // ❌ lifetime-dependent variable 'self' escapes its scope
                                  // note: it depends on the lifetime of variable 'value'
    self.init(value: value)
  }

Here, you're running into the strangeness of depending on a BitwiseCopyable value. The semantics of an integer value doesn't specify any ownership or lifetime. So, when you ask for that dependency, the compiler creates local borrow scope for the variable in the caller, giving it a sort of temporary lifetime. This makes it reasonably easy to build non-Escapable types from UnsafePointers. But it also tends to expose the compiler implementation in the form of overly strict diagnostics.

The diagnostic above is an artifact of the compiler. Since you're allowed to reassign value, it's internally modeled as a separate local variable that happens to be initialized with a copy of the argument. The call to self.init temporarily borrows that local variable. We could teach the compiler to look through the copied variable assignment in this case, it just doesn't fall out of the current implementation.

EDIT: I think borrowing dependency on a consuming value should be illegal:

@_lifetime(borrow value) // error: invalid use of borrow dependence with consuming ownership
func foo(_ value: consuming Int)

That defines away the confusing diagnostic above. This is already illegal for nontrivial types, but we seem to have a special handling for BitwiseCopyable, which probably isn't actually desirable.

EDIT #2: We allow this...

    @_lifetime(borrow value)
    func foo<T: BitwiseCopyable(_ value: consuming T)

As a special case because it's common to pass an UnsafePointer into a Span-like initializer. consuming is the default ownership for initializer parameters, and we don't want to force APIs to explicitly borrow a pointer (that wouldn't make sense). The fix here is to cleanup our implementation of ownership in SIL before performing lifetime diagnostics, then you won't end up with the above error 'self' escapes its scope.

2 Likes
7

For an enum such as:

@frozen
public enum Optional<Wrapped: ~Copyable & ~Escapable>: ~Copyable, ~Escapable {
  case none
  case some(Wrapped)
}

extension Optional: ExpressibleByNilLiteral where Wrapped: ~Copyable & ~Escapable {
  @lifetime(immortal)
  public init(nilLiteral: ()) {
    self = .none
  }
}

extension Optional where Wrapped: ~Copyable & ~Escapable {
  @lifetime(copy value)
  public init(_ value: consuming Wrapped) {
    self = .some(value)
  }
}

  • Do the none and some cases have implicit lifetime dependencies?

  • Is the @lifetime(immortal) attribute required, or can it be inherited from the protocol requirement?

  • Is the @lifetime(copy value) attribute required, or can it be the default when consuming a parameter?

  • Why is the standard library already using non-underscored @lifetime attributes?

8
1 Like
9

Belated pitch feedback, cosmetic but (I think) with real implications for user experience—

I remember that it felt like a clear improvement when the pitch moved on from spelling the annotation as dependsOn. At the time, it seemed to make a lot of sense to have @lifetime as an attribute on the overall member declaration.

But seeing it fleshed out now, I get the sense that a good chunk of the design would be made more ergonomic if we moved @lifetime back to decorating the inout parameter or return type the lifetime applies to.

Having @lifetime(...) and @lifetime(self: ...) written in the same position, but referring to the lifetime of the return value and the mutating self, respectively, feels less than ideal. Particularly when, in the alternative, the former could unambiguously sit next to the return type and the latter could remain in the spot where by convention we place other modifiers that refer to self, such as mutating itself.

(We'd have to keep the spelling as @lifetime(self:) for compatibility, but the extra clarity is arguably good to have even if we didn't have compatibility reasons for it.)

And a separate spelling for inout parameters, @lifetime(target: ...), that needs to name rather than just decorate the parameter also feels less than ideal, particularly since the use of the parameter's name has to precede the declaration of it. Since it is avoidable, and since we have precedent for annotating parameters directly (e.g.: property wrappers, isolated), I think it merits reconsideration here.

7 Likes
10

This is really exciting! One of the biggest things I'd request to help with clarity is a section about how this is different from Rust (because it's pretty different in cool ways!). I'd be happy to make an attempt. For me, and I imagine other readers, Rust is the touchstone for languages with lifetimes; but Rust has pervasive and infectious lifetimes to track first class borrows, and this proposal is about tracking the flow of second class non-Escapable values, which cannot be stored in Escapable containers / collections. My first read was very confusing until I realized this. A section that lays this intention out, either early in the document or linked early in the document, would be really helpful in my opinion for front loading this distinction.

Maybe just trivia: something I find interesting is the "evolutionary branch" where Rust could have been a language where borrows were second class, and lifetimes were not explicitly spelled. This proposal reads to me in a similar vein, and I think in conjunction with Swift already having ARC to manage long lived data, that's extremely compelling. More about "Rust with no explicit lifetimes" here. The most notable part is that trying to support external iterators and collections is what pushed Rust towards first class explicit lifetimes... Is there any thinking on if that's something we can swear off?

1 Like