Non-`Copyable` `Optional` types

Evolution Pitches
21

I guess I didn't a good job explaining behaviour I want to express. maybeValue shouldn't be in an indeterminate state. It's either .some or .none depending on path, but is not "consumed".
If I consume the value I want the maybeValue to become .none. If I don't consume the value I want the maybeValue to keep .some(value)

1 Like
22

To my eyes, you describe exactly what take() is for.

23

You could take and conditionally take back, no?

2 Likes
24

Yep, but take takes the value unconditionally.

let tmpValue = maybeValue.take()
let result: Int?
switch consume tmpValue {
  case .none:
    break
  case let .some(value):
    if value.id % 2 == 0 {
      result = (consume value).id
    } else {
      maybeValue = consume value
    }
}

@xwu Yep, I understand this approach, but is this really a desired way to do it?

25

I see. Right now we have the problem of not having the borrow switch. Even if you could borrow switch, you'd be in a borrow scope, and you wouldn't be able to mutate the optional without an exclusivity violation. I think you're left with having to mutate twice, unless the borrow analysis ends up being really smart (and it might be!)

1 Like
26

Ok, I think its a viable, yet a bit error prone approach. Too easy to forget to restore the state of the original source of the value. I find my _mapMaybeConsuming a bit more rigid, though the signature of the closure is kinda ugly.

27

How about:

func iWantThis(_ optional: borrow Value?) -> Bool { ... }

func conditionalTake(_ optional: inout Value?) -> Value? {
  if iWantThis(optional) { return optional.take() }
  return nil
}

(of course this still needs the borrow bindings to be nice.)

28

Good for filter-like operations, but not for map-like ones.
We could improve readability by introducing something like:

enum ConsumingMapResult<Wrapped, Result> {
  case take(Result)
  case leave(Result, Wrapped)
}

func mapConsuming<Result>(_ body: (consuming Wrapped) -> ConsumingMapResult<Wrapped, Result>) -> Result

let result: Int? = maybeValue.mapConsuming {
  if $0.id % 2 == 0 {
    .take($0.id)
  } else {
    .leave(nil, $0)
  }
}

But it seems this is really a question about supplementary API design, not something that the language itself can address.

1 Like
29

If you get an inout binding to the optional itself, then within the inout access, you can either grab the value out (and put a new one in) or leave the existing one alone. So you could do something like

inout maybeValue = path.to.optional
if let id = maybeValue?.id, id % 2 == 0 {
  // take the current value away, put nil in its place
  someOtherValue = maybeValue.take()
} else {
  // put nil in the other destination, leave the current value here
  someOtherValue = nil
}
1 Like
30

I don't see any reason why those have to be different, strictly speaking. If Optional were to have this behaviour of resetting to nil when consumed from, I would expect it to do that in all cases.

But, they could be different, and perhaps should. In the latter case a compiler warning might be wise, if x is read again before being explicitly reassigned, because that seems unlikely to be the author's intent (but could be, technically - maybe it's only consumed on some code paths, by design). Whereas for member variables the compiler has to assume they will be read again in future - since it can't prove they won't - and the implicit 'reset' value of nil is part of the programmer's intent.

(an explicit take() is barely any more explicit, since it could easily be cargo-culted by the majority of people as just "Swift's weird name" for what other languages would call get())

Consider if x were an Array<File> instead, and these consuming assignments were transfers of the contents. Nobody would question that x still exists after the consumption - it's merely empty. But they probably would appreciate a compiler warning saying "hey, you emptied this array here, and I know for a fact it's still empty when you tried to index into / enumerate it / whatever over here, so, are you sure that's what you want?". And they could either fix it (if appropriate) or ignore the warning or insert an explicit assignment of nil to satisfy the compiler and eliminate the warning.

I'd love it if the compiler could warn me about stuff like that even when it is a member variable, but it's just really hard for the compiler to reason about code at that level.

1 Like
31

Perhaps a new overload of swap can be made (once we get noncopyable generics) where one argument is passed by value and the old value is returned? Swapping different values in general would be more difficult with noncopyable types, and this would cover all of those cases, not just swapping an optional value with nil.

func swap<T: ~Copyable>(_ a: inout T, with replacement: consuming T) -> T {
    var consumedValue = replacement
    swap(&a, &consumedValue)
    return consumedValue
}

let consumedOptional = swap(&object.optionalValue, with: nil)
let consumedArray = swap(&object.arrayValue, with: [])

Another idea I have is that consuming a mutable stored property could implicitly set that property to its default value, if it has one, so that for example, consuming a stored property with a default value of nil would reset it back to nil again. This would mirror how uninitialized stored properties are implicitly set to their default values in initializers, and give consume a secondary meaning of "restoring a stored property to its uninitialized state/default value".

1 Like
32

1. Swap

I don't like the name take for this operation because it creates two near-synonym operations (take and consume). But I really like the idea of using swap for this because its meaning is more obvious.

extension Optional where Wrapped: ~Copyable {
    mutating func swap(with replacement: Wrapped?) -> Wrapped? {
        let result = consume self
        self = replacement
        return result
    }
}

let a: File? = File(...)
let b = a.swap(with: nil) // `a` becomes `nil` here, obviously

Of course this method will become redundant (and perhaps undesirable) if we later add a generic standalone swap function for this:

let b = swap(&a, with: nil)

What we avoid with swap is adding a new operation meaning "consume and set a default empty value" when types in Swift are never implicitly set to a default empty value. Note that while T? does have a default initialization to nil, this does not work when spelled Optional<T>, so it's not really part of the type.

2. Reset or Clear

If we really want to add a "consume and set a default empty value" operation, I think it should be named something other than take. I'd suggest reset or clear to emphasize the value has been set back to some kind of default (the main difference with consume). Just like remove on Array it would return the old value as a @discardableResult. This could be added to all sort of types, as long as they have a default initial value to return to.

But I don't think taking this route with a "consume and set a default empty value" operation is necessary. Having a default initial value is not a concept we need to have to solve the problem at hand. So I'd rather just add a swap method.

Whatever happens in Optional, other types in different code bases will follow in its footsteps. So it's important to choose wisely.

2 Likes
33

There are three distinct stages I think:

  1. Generic nominal types can declare their generic parameters to be ~Copyable, and you can write extensions like extension Optional where Wrapped: ~Copyable, but you can't conform such types to protocols yet.
  2. Non-copyable types can conform to protocols without associated types, if those protocols are declared ~Copyable.
  3. Non-copyable types can conform to protocols with associated types, so those associated types can themselves be ~Copyable.

Implementing take() within the language would only need 1).

I think for 1) and 2) we pretty much understand the design and implementation, for 3) there are still a few open questions about associated types and copyability.

6 Likes
34

Idea: the inconsuming qualifier. It's like the inout qualifier, but instead of the ability to write a value back, it allows to consume or not to consume a value. If the value is consumed, the source of the value is also considered consumed, but when it's not - a special restore block is invoked, in which the source is accessible again.
A rough sketch:

var file: File
foo(file: consume file restore {
  // this block will be executed when `foo` will return
  // and `file` wasn't consumed.
  // `file` is accessible here.
})
// `file` isn't accessible here

func foo(file: inconsuming File) {
  if .random() {
    _ = consume x
  }
}

Basically, an inconsuming parameter is a parameter that allows for backpropagation of whether it was consumed or not.
The closure of the map function of Optional is an example of where this type of parameter passing is suitable. Returning an inconsuming value from a function can be modelled as a coroutine.

Examples for Optional:

  • take function that yields the optional:
    mutating func take() -> inconsuming Wrapped? {
  let result = consume self
  self = .none
  yield_inconsuming consume result restore {
    self = result
  }
}

func usageExample() {
  var file1: File? = File(...)
  do { // just a lexical scope
    inconsuming file2 = maybeFile.take()
    // file1 is nil (accessible)
    if inconsuming file3 = file2 {
      // enters here is file2 wasn't nil
      // file3 isn't optional
      // file2 isn't accessible
      if .random() {
        _ = consume file3
      }
    } else {
      // enters here is file2 was optional
      // file2 is accessible
    }
  } // end of lifetime of file2
  // if `file3` was consumed, `file1` stays `nil`, otherwise it restores its value
}
  • map function that yields the unwrapped value to the body closure, and then yields the result to the caller
    mutating func map<Result>(_ body: (inconsuming Wrapped) throws -> inconsuming Result) rethrows -> inconsuming Result? {
  let s = consume self
  self = .none
  switch consume s {
  case .none:
    yield_inconsuming nil // return nil
  case let .some(wrapped):
    inconsuming w = consume wrapped restore {
      self = .some(consume wrapped)
    }
    inconsuming result = try body(w)
    yield_inconsuming result
  }
}

func usageExample2() {
  var file: File? = File(...)
  incosuming fd = file.map {
    yield_inconsuming $0.fd
  }
  if incosuming fd2 = fd { // optional unwrap
    do {
      try closeFileDescriptor(fd2) // pass by borrowing
      _ = consume fd2
    } catch {
      // can't close file
      // releasing `fd` back to `file`
    }
  }
  // if file was closed, `file` stays `nil`, otherwise it restores its value
}
  • consume foo restore { ... } expression produces an inconsuming binding. If this binding wasn't consumed by the end of its lifetime the restore block will be invoked with accessible foo.
  • yield_inconsuming statement is like a regular yield but yield an inconsuming binding, instead of var binding.
35

Having a swap that works with noncopyable types would indeed serve the same need as take() more generically. We have the public func swap<T>(inout T, inout T) already, and its existing implementation is hardcoded not to copy, so we could retrofit it to accept noncopyable values. It might still make sense to have a value-returning variation, swap<T>(inout T, consuming T) -> T, and something like the proposed take() method on Optional to handle the case of wanting to swap in nil. swap alone might not be easily discoverable for developers in this situation (though that could also be aided by teaching people swap as part of learning how to work with noncopyable values).

11 Likes
36

FWIW, Rust has swap, replace (the one with a consuming T argument), and take, and I've found places where each of the three would be clearest in context. We don't have Rust's Default for "the thing you leave behind for take for arbitrary types", though it certainly comes up every now and then, but they too considered Option(al) important enough to give it a dedicated take method in addition to the top-level function. It's most relevant when you have shared data that's being consumed by one thread/task.

EDIT: Rust doesn’t have type-based overloading, though, so we do have the option of using swap for both variations. Having these operations available matters more to me than what they’re called, not that the name is completely unimportant.

8 Likes
37

I think in Rust it's more idiomatic for types to have a huge number of "helper methods" than in Swift. For example, their Option type has methods like expect, and, and_then, get_or_insert and a bunch more. (Rust also seems to heavily favor postfix syntax, such as .await being written like a field access.) In Swift I feel like it's be more idiomatic to use general-purpose features to accomplish those things, like guard let or if let or top-level functions, and potentially user-defined extensions.

Edited to add: I think the best example is that Rust's Option type has a replace method, which is almost identical to std::mem::replace (with the minor difference that the method version doesn't take None).

4 Likes
38

I think we should have a separate name for "make this the new value and give me the old one", it's not quite the same thing as swap semantically, and replace is definitely a nicer name than C++'s std::exchange.

3 Likes
39

Part of the reason I went with swap was so that someone who knows about one swap function can easily discover the other. They are slightly different but achieve very similar things. It's an easy jump to go from swap(_:with:) to swap(_:_:) when the need arises, such as when you need to swap two noncopyable properties of a reference type. But replace(_:with:) would be a good name too.

Edit: Now that I think about it, I do wonder if swap would better communicate that the old value is returned and not simply discarded.

1 Like