Hi Evolution, I have a short and sweet Optional proposal that adds some new methods when working with noncopyable wrapped types as well as some generalizations to existing API to work with noncopyables.
Please let me know what you think!
Optional noncopyable improvements and generalizations
- Proposal: SE-NNNN
- Authors: Alejandro Alonso
- Review Manager: TBD
- Status: Awaiting implementation
- Implementation: swiftlang/swift#NNNNN
- Review: (pitch)
Summary of changes
Introduces three new methods on Optional, borrow(), mutate(), and insert().
As well as generalizing map, flatMap, and unsafelyUnwrapped.
Motivation
Since Swift 6.0 where we generalized parts of the standard library to support
storing noncopyable values, working with
noncopyable optionals has been quite cumbersome. It's very common to want to
inspect the contents within the optional, maybe pass it to a function that wants
to borrow the payload, but you don't want to consume the optional. Perhaps you
need to continue using the optional, or you simply don't have an owned optional
in the first place (you were passed borrowing T? for example). Consider the
following example trying to peek at the optional's contents:
if let payload = optional {
}
foo(optional) // error: use after consume!
This has been a constant pitfall with no clear workaround. Fortunately with
noncopyable switches,
we can make control flow borrow the optional's contents:
switch optional {
case .some(let wrapped):
// wrapped is borrowed!
default:
break
}
foo(optional) // ok
However, writing that switch is not very intuitive if you're used to if let or
even guard let.
If you wanted to mutate the optional's payload without consuming it, there's
only a handful of ways to achieve this. For simple property mutations or calling
mutating methods, the ?. chaining is sufficient, but if you needed to
conditionally pass the payload to some function taking it inout you could
write:
if someStruct.x != nil {
foo(&someStruct.x!)
}
Or the more verbose and unintuitive way by consuming the optional in a switch
and reinitializing it:
func foo(_: inout NoncopyableString) {}
func bar(_ x: inout NoncopyableString?) {
switch consume x {
case .some(var string):
foo(&string)
x = consume string
default:
x = nil
}
}
Similarly, there are a number of API on Optional that are not available for
noncopyable wrapped values such as map, flatMap and unsafelyUnwrapped.
Missing out on map and flatMap has lead to many workarounds needing to
manually stamp out switch statements everywhere and rewriting code to work with
noncopyable wrapped values being more verbose than its predecessor.
Proposed solution
We introduce two new methods on Optional: borrow() and mutate(). These will
return references to the inner wrapped payload if there is one and nil
otherwise.
func bar(_ x: borrowing SomeNoncopyable) {
...
}
if let payload = optional.borrow() {
bar(payload.value) // 'bar' gets passed a 'borrowing Wrapped'
}
foo(optional) // ok
func baz(_ x: inout SomeNoncopyable) {
...
}
if var payload = optional.mutate() {
baz(&payload.value) // 'baz' can mutate the payload in place!
}
foo(optional) // ok
These two methods allow for a very concise way to conditionally access the
payload value of an optional without having ownership of the optional.
We also propose generalizing the following Optional methods to support
noncopyable and nonescapable wrapped types:
mapflatMapunsafelyUnwrapped
let optAtomicInt: Optional<Atomic<Int>> = ...
let optInt: Optional<Int> = optAtomicInt.map {
$0.load(ordering: .relaxed)
}
let optInlineArray: Optional<[0 of Atomic<Int>]> = []
let optInt2: Optional<Int> = optInlineArray.flatMap {
$0.isEmpty ? nil : $0[0].load(ordering: .relaxed)
}
let optMutex: Optional<Mutex<Int>> = ...
let mutex: Mutex<Int> = optMutex.unsafelyUnwrapped
optMutex?.withLock { ... } // error: use of 'optMutex' after consume
A quality of life API we're also proposing is Optional.insert. Consider the
following pattern:
struct Cache: ~Copyable {
var opt: UniqueArray<Int>?
}
var cache = Cache()
// do some computation
var items: UniqueArray<Int> = fooBar()
cache.opt = items
// more calculations, maybe some
// API calls
let newItem = await retrieveNewItem()
cache.opt!.append(newItem)
The use of ! here is really unnecessary because we know there's a value that
exists in the optional. In some cases, this ! may not get optimized away if
you're calling into a non-inlined function taking Cache because it can't make
any assumptions about the values that exist in it. We can safely model an insert
method that returns a direct mutable reference to the payload of an optional
given a new item to put into the optional:
struct Cache: ~Copyable {
var opt: UniqueArray<Int>?
}
var cache = Cache()
// do some computation
let items: UniqueArray<Int> = fooBar()
var itemsRef = cache.opt.insert(items)
// more calculations, maybe some
// API calls
let newItem = await retrieveNewItem()
itemsRef.value.append(newItem)
Detailed design
borrow() and mutate()
extension Optional where Wrapped: ~Copyable {
/// Returns a borrowed reference to the payload within the optional, if there
/// is one.
@lifetime(borrow self)
public func borrow() -> Ref<Wrapped>?
/// Returns the mutable reference to the payload within the optional, if there
/// is one.
@lifetime(&self)
public mutating func mutate() -> MutableRef<Wrapped>?
}
Conceptually, these methods effectively move the ownership of the optional and
its payload. In order to call borrow(), for example, you must have at least
a borrowing Optional<T> which, if you squint hard enough, is Ref<Optional<T>>.
borrow() takes this Ref<Optional<T>> and produces Optional<Ref<T>> which
moved the reference inside the optional meaning we get back an owned value we
can mutate, consume, or simply have exclusive access to.
The same is true for mutate(). It takes at least inout Optional<T> to be able to
perform the call which can look like MutableRef<Optional<T>> into an owned
Optional<MutableRef<T>> value.
insert()
extension Optional where Wrapped: ~Copyable {
/// Sets the value of the optional to the passed in new value while returning
/// a mutable reference to that value inside the optional.
///
/// If there's already a value within the optional, that value is destroyed.
///
/// - Parameter new: The new payload value to put into the optional.
/// - Returns: A mutable reference inside the optional to its newly inserted
/// payload.
@lifetime(&self)
public mutating func insert(_ new: consuming Wrapped) -> MutableRef<Wrapped>
}
unsafelyUnwrapped generalization
extension Optional where Wrapped: ~Copyable & ~Escapable {
public var unsafelyUnwrapped: Wrapped {
consuming get
}
}
map() and flatMap() generalizations
extension Optional where Wrapped: ~Copyable & ~Escapable {
@lifetime(copy self)
public consuming func map<Result: ~Copyable & ~Escapable, E: Error>(
_ transform: (consuming Wrapped) throws(E) -> Result
) throws(E) -> Result?
@lifetime(copy self)
public consuming func flatMap<Result: ~Copyable & ~Escapable, E: Error>(
_ transform: (consuming Wrapped) throws(E) -> Result?
) throws(E) -> Result?
}
We were hesitant to eagerly generalize these back in SE-0437
because there are technically three forms of map and flatMap that can occur
with noncopyable wrapped values. One can choose to consume the optional entirely
being passed the owned value of the payload in the closure, borrow the optional
and get passed a borrowing reference to the payload, or mutate the optional
and get passed an inout reference to the payload. All three variants are all
equally useful, but we don't currently have a way to distinguish between them
if we named them all map due to overloading rules/limitations. By making these
generalizations always consuming by default, borrow() and mutate() actually
help us achieve the other variations by giving us owned optionals values:
// Not sure why you'd wrap an atomic in an optional,
// but for the sake of an example.
func foo(x: borrowing Optional<Atomic<Int>>) -> Optional<Int> {
x.map { // error: 'x' is borrowed and can't be consumed
$0.load(ordering: .relaxed) &+ 1
}
}
func bar(x: borrowing Optional<Atomic<Int>>) -> Optional<Int> {
x.borrow().map { // ok!
$0.value.load(ordering: .relaxed) &+ 1
}
}
func baz(x: inout Optional<UniqueArray<Int>>) -> Optional<Int> {
x.mutate().map {
// Update the array while mapping over it
$0.value.append(123)
return $0.value.count
}
}
Source compatibility
Optional.borrow(), Optional.mutate(), and Optional.insert() are new
methods so they shouldn't introduce any source compatibility issues. The
generalizations of map and flatMap on the other hand share a name with
the existing Optional.map and Optional.flatMap. This actually isn't an issue in
practice because while they share the same name, they have very different
signatures. Existing callers of Optional.map, for example, are all
required to call it where Optional is Copyable. With this new
overload, the existing method will become "more specialized" which
actually becomes favored for all current callers due to how overload
resolution works. Thus, there is no source compatibility issues with these
specific new generalizations.
unsafelyUnwrapped is purely a generalization and cause no source compatibility
issues.
ABI compatibility
The new methods on Optional are new API to the standard library that also don't
come with any ABI. The generalizations of map, flatMap, and unsafelyUnwrapped
don't come with any new ABI nor break any old ABI.
Implications on adoption
The Optional.borrow(), Optional.mutate(), and Optional.insert()
methods will have availability equal to the availability of the new
Ref and MutableRef types. The rest of the generalizatons will be marked as
always available.
Future directions
Borrow and inout bindings
A potential future if we decide that borrow/inout bindings makes sense is to
augment the compiler to recognize if borrow/if inout patterns for optionals
to provide conditionally scoped access to the payload:
if borrow x = optional {
}
foo(optional) // ok
If we decided this was a better direction than the borrow() and mutate()
story, then we'd need to rethink how we want to generalize map, flatMap, and
unsafelyUnwrapped because they can no longer be always consuming. We can provide
consumingMap, borrowingMap, mutatingMap, etc. which is one solution, but
not a pleasant one because we would see this multiplication of API for map,
flatMap, etc.
Generalize Optional: Equatable and Optional: Hashable
Now that SE-0499
generalized protocols like Equatable and Hashable to support noncopyable
and nonescapable conformers, it seems obvious that we should just generalize
Optional's conformances to support those suppressed wrapped types.
Unfortunately, we don't have a way to say that a particular conformance was
generalized at some availability. We need to prevent the scenario where accidentally
calling into the generic == or hash(into:) for Optional on an older ABI
stable OS copies the wrapped payload.
Automatic dereferencing for Ref and MutableRef
In the proposed solution, the examples using borrow() and mutate() to do a
map needed to explicitly access the .value property on these reference
types. This isn't quite as ergonomic as the existing map on Optional that
let's you interact with the passed parameter as the type itself. We could give
Ref and MutableRef a special behavior in the future to automatically
dereference themselves when accessing member properties or methods on them:
func bar(x: borrowing Optional<Atomic<Int>>) -> Optional<Int> {
x.borrow().map { // ok!
// No more '.value.'
$0.load(ordering: .relaxed) &+ 1
}
}
Automatic dereferences like this greatly improve working with these types. If we
had this for Ref/MutableRef, we could extend this functionality generically
to other types through a protocol based solution like Deref
which would be useful for UniqueBox as well.
Alternatives considered
Change the default ownership of if let bindings
In the motivation for some of the methods of this proposal, it's stated that we
need methods like borrow() and mutate() to allow for borrowing versions of
control flow. We could instead change the default ownership of these if let
scenarios to be borrowing by default. However, this has the opposite effect
because now it becomes a source breaking change for folks actually utilziing the
consuming nature of these operations.
Optional.ref and Optional.mutableRef names
Instead of the borrow() and mutate() method names, we could have properties
that returned the same value like the various .span and .mutableSpan
properties.
However, we feel that the nature of the verbs borrow and mutate fit quite
well in API usage especially for things like opt.borrow().map { ... }. It also
mimics the recently accepted Borrow Accessors SE-0507
names.