[Pitch] Non-Escapable Types and Lifetime Dependency

My question is whether allowing the full return type syntax is necessary or even advisable just to be able to specify return-side effects. Initializers can already throw, why doesn’t init() throws require a return type of Self?

Initializers in Swift have always been defined as returning the constructed value

The Swift Programming Language has this rather clear statement to the contrary:

Unlike Objective-C initializers, Swift initializers don’t return a value. Their primary role is to ensure that new instances of a type are correctly initialized before they’re used for the first time.

5 Likes

Our first syntax was

init(arg: Arg) -> borrow(arg) Self

We wanted to make it clear that 'arg' is borrowed over the lifetime of the returned 'Self'. The alternative syntax would seem to indicate that 'arg' is only borrowed for the duration of the initializer:

init(arg: Arg) borrow(arg)

That's actually a meaningful syntax, because borrowing the caller's copy of arg is not actually implied by ownership modifiers (init(arg: borrowing Arg)), which still forces the caller to copy arg by default.

Now that we we have dependsOn syntax, that source of confusion is gone:

init(arg: Arg) dependsOn(arg)

I don't have a strong opinion yet on continuing to require -> Self for dependsOn. I have found it slightly annoying to remember to add -> Self.

Eventually we may end up requiring it for other features, like borrowed return values:

init(arg: Arg) -> borrowing Self
3 Likes

Another aspect of Compile-time Lifetime Dependency Annotations that adopters tend to encounter early on, but isn't directly confronted in the proposal, is their interaction with
BitwiseCopyable. BitwiseCopyable values have neither ownership nor lifetimes. The compiler is free to create temporary copies as needed and keep those temporary copies around as long as it likes--it has no observable effect on the program. It follows that a lifetime dependence on a BitwiseCopyable value has no meaning. The compiler has no way to enforce such a dependence, and ignoring the dependence is always valid. To avoid giving programers the wrong impression, the compiler issues an error in code that asks for lifetime dependence on a BitwiseCopyable type:

func f(arg: Int) -> dependsOn(scoped arg) NonEscapableType // ERROR: invalid lifetime dependence on BitwiseCopyable type

The programmer has asked to "borrow" an integer over the lifetime of NonEscapableType, which is meaningless because an integer is nothing more than its bitwise representation.

The interaction between BitwiseCopyable and conditionally escapable types leads to conditionally ignored dependence annotations. Conditionally escapable types are introduced in Non-Escapable Types:

struct Box<T: ~Escapable>: ~Escapable {
  var t: T
}
 
// Box gains the ability to escape whenever its
// generic argument is Escapable
extension Box: Escapable where T: Escapable { }

Returning an escapable or conditionally escapable type requires lifetime dependence:

func transfer<T: ~Escapable>(arg: Box<T>) -> dependsOn(arg) Box<T> // 'dependsOn' may be inferred

The compiler ignores this lifetime dependence when it applies to an escapable type like Int:

func escapingInt() -> Box<Int> {
  let box = Box(t: 3)
  return transfer(arg: box) // OK: Box<Int> is escapable
}

Adopters do often need to prevent BitwiseCopyable values from escaping. This requires wrapping the BitwiseCopyable value in unconditionally ~Escapable types. @Joe_Groff shows how to do that in Values of ~Escapable type with unlimited, indefinite, and/or unknown lifetime, where UnsafePointer is BitwiseCopyable.

public struct Ref<T: ~Copyable>: ~Escapable {
  private var address: UnsafePointer<T>
 
  private init<X>(unsafePointer: UnsafePointer<T>, dependingOn dummy: borrowing X) -> dependsOn(dummy) Ref<T> { ... }
 
  public init(borrowing value: borrowing T) -> dependsOn(value) Ref<T> {
    // This initializer call is only safe if the programmer passes an `unsafePointer` that correctly
    // depends on the lifetime of `value`.
    self.init(unsafePointer: _someWayToGetALimitedScopePointer(to: value), dependingOn: value)
  }
}

Creating a dependence on a BitwiseCopyable value is inherently unsafe. But this approach limits unsafety to a single point at which the programmer declares that the BitwiseCopyable value (unsfePointer) depends on value. We could consider formalizing this step with a standard "DependentValue" wrapper.

1 Like

Can you clarify this? If init(arg: borrowing Arg) forces caller to copy by default, what does it actually do different to init(arg: Arg)?

The ownership qualifiers on arguments are from the callee's perspective.

func foo(arg: borrowing Arg)
The implementation of foo receives a borrowed value. If it needs to persist arg, then it must create its own copy.

func foo(arg: consuming Arg)
The implementation of foo consumes its value. It may persist arg without creating a copy.

But from the caller's perspective, changing the ownership of a copyable argument is a source compatible change, so in either case, it defensively copies any variable passed to the argument:

var value = ...
foo(arg: value) // Pass a copy of `value` into foo

The difference is that, with borrowing, the caller is responsible for destroying the temporary copy of value. So, with a borrowing convention, it's quite likely that the caller can be optimized to eliminate that copy. The compiler only needs to prove that nothing modifies value or releases its reference during the call to foo.

3 Likes

The two points of view are equivalent, because the type of a Foo.init() call is Foo so Foo.init has type () -> Foo. We just spell “returning an arbitrary value” as `self = …; return” instead.

3 Likes

To tie this back to the proposal in question, when a function is declared with a scoped lifetime dependence, the caller no longer passes a temporary copy:

func foo(arg: borrowing Arg) -> dependsOn(arg) NonescapableType

Unlike changing the parameter's ownership modifier, adding a lifetime dependence is not source compatible.

var value = ...
// caller borrows `value`
let depenent = foo(arg: value)
use(dependent)
// caller restores mutability of `value`

Here, instead of copying value, the caller now borrows it for the duration of the call, and it extends that borrow scope at least up to the last use of dependent. Now, if anything tries to mutate or release value, we get a compiler error or runtime trap.

To me it makes no difference whether it's dependsOn or borrow, in terms of clarity. Both could be interpreted (perhaps cantankerously) as applying only to the function itself, not the 'returned' instance. Both should be correctly understood given even a little thought, given the existing of the borrowing parameter modifier.

Perhaps consider:

init(arg: Arg) -> Self which borrows(arg)

It doesn't really change anything regarding the above nominal ambiguity - it's still not technically clear whether that applies to Self or the function overall. On the one hand, it's marginally clearer since now Self is closer to the clause than any other part of the declaration. But on the other, by virtue of the kinship with where it loses clarity because where applies to the whole declaration, not the return value. So, a wash overall. But I think still just plain nicer to read.

Note that the latter could be addressed by:

init(arg: Arg) -> Self where Self: borrows(arg)

I'm partial to that - it's more verbose, yes, but it's clearer. And it fits more neatly into the existing grammar.

3 Likes

Has there been any discussion on Rust-style lifetime annotations? I feel like Rudy’s approach is more flexible concerning future directions and already has a lot of existing documentation. It would be nice to see the Alternatives Considered section address why this type of syntax wasn’t chosen.

7 Likes

I don't think it's particularly cantankerous. Nothing about the phrase "depends on" implies lifetimes. It would be nice if we can think of a clearer phrase.


The thing that I've been waiting for from lifetime dependencies in the language is the ability to have an array of non-escaping closures, with the array having a lifetime constrained by its contents.

I'm not sure if that is possible with the current proposal. Do lifetime dependencies compose at all? For instance, is it possible to write something like:

struct NonEscapingArray<Element: ~Escaping>: ~Escaping {

  consuming func append(
    _ newElement: Element
  ) -> NonEscapingArray dependsOn(self, newElement)
}

Where the lifetime of the result of append is the intersection of the array's existing lifetime and the lifetime of the element being added to it?

Also, I wrote this as a consuming function returning a value because I think that's the only way to add a lifetime dependency, but it would lead to usage like this:

var arr = NonEscapingArray<() -> Void> { ... }
arr = arr.append { ... }
arr = arr.append { ... }
arr = arr.append { ... }

(I think? Is it allowed to reassign a variable with a value of the same type but different lifetime?)

Another possibility would be if we allowed mutating functions to modify the lifetime of self. Then you wouldn't need to keep reassigning to the same variable.

6 Likes

Alternatively, if we don't think about having lifetime dependencies on other values but instead about enforcing lifetime constraints, the above could be modelled as:

struct NonEscapingArray<Element: ~Escapable>: ~Escapable {

  mutating func append(
    _ newElement: Element
  ) where lifetime(newElement) > lifetime(self)
}

It's kind of a different way of looking at things. Rather than saying the array depends on elements X/Y/Z, the array has its own lifetime (it's non-escaping), and the elements are required to satisfy those bounds. The array's lifetime doesn't depend on its contents; it sets a lower-bound for them.

I think it's more elegant (at least for this case), but it's also more of a Rust-like model.

Is the idea that we would maybe have both in Swift?

6 Likes

I've added several new sections to the Compile-time Lifetime Dependency Annotations Proposal. I copied the new sections into a separate proposed updates document in case you don't want to reread the proposal.

The new sections are:

  • Dependent parameters
  • Dependent properties
  • Conditional dependencies
  • Immortal lifetimes
  • Depending on immutable global variables
  • Depending on an escapable BitwiseCopyable value
  • Standard library extensions
  • unsafeLifetime helper functions
  • Dependency semantics by example
  • Future directions
    • Value component lifetime
    • Abstract lifetime components
    • Protocol lifetime requirements
    • Structural lifetime dependencies

Please take a look!

13 Likes

I’m not deeply familiar with Rust, but the proposed syntax feels significantly more complex than that of Rust. Is that because it is more capable (or will be), or are there other reasons?

The forum is a good place to start constrasting examples of Rust lifetime variables vs. Swift lifetime dependence. I also imagine interesting blog posts on this topic in the future. But it's not the job of the official Swift language docs to teach Rust syntax to Swift devs.

The intention is that Swift's syntax will eventually be at least as powerful as Rust. We do want to make sure people can translate between both syntax forms without losing semantics.

I'd like to state up front that the primary goal for Swift is to design clear and obvious APIs for the majority of programmers who do not know what a lifetime variable is, and simply need to call into Swift libraries. I don't think most programmers will ever need to know what a lifetime variable is because it's an artificial abstraction, rather than a fundamental property of the API.

2 Likes

Consider that it may be the reverse:

func foo<'a>(arg: &'a Arg) -> 'a NonescapableType // (fake syntax)

is actually kind of syntax sugar for:

func foo(arg: borrowing Arg) -> dependsOn(arg) NonescapableType

Since, conceptually, and fundamentally, all we have is one value that needs to be destroyed before another.

If lifetime variables are indeed only syntax "sugar", then would you want to force it on anyone?

That's the key question. I haven't seen any examples yet where lifetime variables can be more expressive.

I disagree, anything written as part of the API makes it a pretty fundamental property of the API. Expressing the lifetime relationship between a parameter and the return value with dependsOn(parameter) at the source syntax level for the API makes it fundamental to how you interact with this API regardless of if the compiler can implicitly determine this relationship for you.

6 Likes

I'm not sure I follow. I only mean that if you express that dependence using a lifetime variable
func foo<'a>(arg: &'a Arg) -> 'a NonescapableType, then both the programmer and the compiler need to perform an extra level of analysis (solving a system of type constraints) to understand its meaning: the result depends on its argument.

Yes, which makes it a fundamental property of the API itself.

1 Like

Is it meaningful for a struct or a tuple with e.g. lots of Integer properties? There cases when creating copies is expensive in comparison to CPU consumption of the algorithm itself.
One more situation is when a large function is decomposed to a bunch of smaller ones, and those can be called thousands of time. Borrowing such a BitwiseCopyable tuple / struct / enum instead of copying it every time can help to achieve performance constraints. Am I missing something?

I also take issue with this; why is it up to Swift to determine the meaning of a particular integer to my program? Many integers have important lifetimes which the programmer may absolutely want to base another value's lifetime on.

For example: a function which returns a writeable buffer for a thunderbolt ring whose lifetime is limited to however long the descriptor is owned by caller.

func bufferForThunderboltDMARing(
  descriptor: UInt32
) -> dependsOn(descriptor) BufferWriter {
  // Yield back a writable buffer to fill thunderbolt buffer with usb4 packets.
}

Also (IIRC) the Pointer types are now BitwiseCopyable. This statement implies you cannot write a function with a lifetime based on that of a pointer argument which seems like a huge miss.

1 Like