[Pre-Pitch] Disconnected Properties and Variables, and Opaque enums

Currently in Swift 6.2, you cannot send an object out of an actor or Task if it has been bound to a property or captured. Also you cannot send a non-Sendable object from a synchronous context into an asynchronous context.

SE-0414 Region Based Isolation and SE-0430 Sending Parameters and Return Values mention a Disconnected type in the Future Directions sections. I would like to discuss a simple implementation that doesn't require adding it to the language at a low level.

For non-Copyable types, if you store a ~Copyable object in a property, you can never get it out of the property. You can replace it, but never retrieve it. In order to allow ~Copyable types to be stored in properties and later retrieved, Optional has been extended to match the copyability of its wrapped object, and a take() method consumes the Wrapped value and sets the Optional to nil.

The Disconnected type must suppress copyability always, and have a sending parameter/return on its init(_:) and take() methods. Otherwise it is the same as Optional.

public enum Disconnected<Wrapped: ~Copyable & ~Escapable>: ~Copyable, ~Escapable {
	case some(Wrapped)
	case none
}

extension Disconnected: ExpressibleByNilLiteral
where Wrapped: ~Copyable & ~Escapable {
  /// Do not call this initializer directly. It is used by the compiler when you initialize with nil.
  public init(nilLiteral: ()) { self = .none }
}

extension Disconnected: Sendable where Wrapped: ~Copyable & ~Escapable & Sendable {}

extension Disconnected: Escapable where Wrapped: ~Copyable & Escapable {
	public init(_ value: consuming sending Wrapped) { self = .some( value ) }

	public mutating func send() -> sending Wrapped? {
		let result = consume self
		self = .none
		if case let .some(result) = result { return result }
		else { return nil }
	}

	public mutating func trySend() throws(DisconnectedError) -> sending Wrapped {
		let result = consume self
		self = .none
		if case let .some(result) = result { return result }
		else { throw .empty }
	}
}
public enum DisconnectedError: Error {case empty}

As long as you only use the init method to create this object, it will prevent data races, but allow you to move an object in and out of Actor or Task isolated regions.

class Nonsendable { var name: String = "nonsendable" }
actor A {
	var i = Nonsendable()
	// func test() { var d: Disconnected<Nonsendable> = .init(i) } // error: Sending 'self.i' risks causing data races. 'self'-isolated 'self.i' is passed as a 'sending' parameter; Uses in callee may race with later 'self'-isolated uses.
	func useNS(_ ns: sending Nonsendable) { print("actor A used:", ns.name) }
}

actor Traveler {
	var value: Disconnected<Nonsendable>
	var otherValue: Disconnected<Nonsendable> = nil
	init(_ value: sending Nonsendable) {
		self.value = .init(value)
	}
	func trySend() throws(DisconnectedError) -> sending Nonsendable {
		try self.value.trySend()
	}
}

// A bad actor
extension Traveler {
	/*
	func doBadThings() {
		otherValue = value // error: Noncopyable 'unknown' cannot be consumed when captured by an escaping closure or borrowed by a non-Escapable type
	}
	
	
	func uhOh() {
		if case let .some(otherValue) = otherValue { // error: Noncopyable 'unknown' cannot be consumed when captured by an escaping closure or borrowed by a non-Escapable type
			otherValue.name += "-uh oh!"
		}
	}
	*/
}

However, this implementation allows one to initialize Disconnected like var d: Disconnected<Nonsendable> = .some(value) which bypasses the sending parameter in the initializer. This allows data races.

If Disconnected is extended with copyability, then it allows data races if you copy one.
//extension Disconnected: Copyable where Wrapped: Copyable & ~Escapable {}

Similarly, if Optional is extended with this send() method, it will allow data races.

actor Traveler2 {
	var value: Nonsendable?
	var otherValue: Nonsendable? = nil
	init(_ value: sending Nonsendable) { self.value = .init(value) }
	func trySend() throws(DisconnectedError) -> sending Nonsendable { try self.value.trySend() }
}
// ... same extension
func test() {
	let a = A()
	let x = Nonsendable()
	let t = Traveler2(x)
	Task {
		await t.doBadThings()
		do {
			let sentX = try await t.trySend() // This is a second reference to x
			Task { t.uhOh() } // Modifies the reference to x stored in otherValue
			Task { a.useNS(sentX) } // Data race! This can read before or after x is modified.
		} catch {
			print(error)
		}
	}
}

If Disconnected could have an attribute which suppresses case initializers and requires the enum to use init methods, then I think this would satisfy the requirements of a 'Disconnected' type as described in the SE proposals.

Because the init(_:) and send() methods both have sending values, the type system ensures the Wrapped object is in a disconnected region at both input and output. Because Disconnected is a non-Copyable enum with only one stored value, we know that Disconnected cannot do anything internally that would break exclusivity rules (like copy Wrapped's internal state). Therefore well-typedness implies data race safety.

This would allow us to safely move non-Sendable objects from a synchronous context into an asynchronous context as well as to make data structures where the internal structure is protected by a region.

The attribute would:

  • Make case initializers private
  • either: synthesize an init method for each case (but allow you to override them) or require you to provide an init method corresponding to each case's associated type pattern
  • Require that each case have a unique associated type pattern
  • Require there is no more than one case without an associated type
    The attribute could be called @opaqueEnum or @opaque. This makes sense because if you declare one that is ~Copyable, then it also can't be destructured externally, making it a type you cannot see inside of, neither at initialization, nor by destructuring.

In my opinion, it would be swifty to change the name of this type from Disconnected to Capability. This changes the focus to the way programmers use the type rather than the internal math.

I don't know how to make attributes, so I hope to discuss this with someone who does know.
Also I would appreciate if someone could point me to more thorough data race tests or an explanation of exactly what must be proved for this to provide safety.

Could you not say:

public struct Disconnected<Wrapped: ~Copyable & ~Escapable>: ~Copyable, ~Escapable {
  public enum UnderlyingType {
    case some(Wrapped)
    case none
  }

  public internal(set) var value: UnderlyingType
}

?

Ya, I thought about that, if the attribute isn't accepted, that is how I would do this. But it seems like a complication. I was having trouble getting it to work with ~Escapable. But if I make it always Escapable, it is very simple.

When I think about it more, it doesn't really make sense for Disconnected to be non-Escapable. Thanks for the suggestion, it simplifies things tremendously if the language can support this already.

This should work:

public struct Disconnected<Wrapped: ~Copyable>: ~Copyable {
	private enum UnderlyingType : ~Copyable {
		case some(Wrapped), none
		mutating func send() -> Wrapped? {
			switch consume self {
			case .some(let value):
				self = .none
				return value
			case .none:
				self = .none
				return nil
			}
		}
	}
	private var value: UnderlyingType

	public init(_ value: consuming sending Wrapped) { self.value = .some(value) }
	public init() { self.value = .none }
	
	public mutating func send() -> sending Wrapped? {
		switch self.value.send() {
		case .some(let value):
			self.value = .none
			return value
		case .none:
			return nil
		}
	}
}
extension Disconnected where Wrapped: ~Copyable {
	public mutating func receive(_ value: consuming sending Wrapped) {
		self.value = .some(value)
	}
}

I don't know why you want to merge optionality with disconnectedness; I don't think it's necessary.

My version of Disconnected looks like this:

public struct Disconnected<Value: ~Copyable>: ~Copyable, @unchecked Sendable {
    private nonisolated(unsafe) var value: Value

    public init(_ value: consuming sending Value) {
        self.value = value
    }

    public consuming func consume() -> sending Value {
        value
    }

    public mutating func swap(_ other: consuming sending Value) -> sending Value {
        let result = self.value
        self = Disconnected(other)
        return result
    }

    public mutating func take() -> sending Value where Value: ExpressibleByNilLiteral & SendableMetatype { // I think the need for `SendableMetatype` here is a bug, https://github.com/swiftlang/swift/issues/83253
        return swap(nil)
    }

    public mutating func withValue<R: ~Copyable, E: Error>(
        _ work: (inout sending Value) throws(E) -> sending R
    ) throws(E) -> sending R {
        try work(&self.value)
    }
}

(definitely not saying we shouldn't support ~Escapable too; that's just newer than I've had to deal with in practice)

2 Likes

Unfortunately, one of the things you really want to do with this, is Mutex<[Disconnected<T>]>, but Array still doesn't support ~Copyable elements :( So you end up having to write your own non-Collection collections.

Thanks for the alternative code. I wasn't intending to merge disconnectedness with optionality, this was just the solution I came to.

Yes you would have to make your own non-Collection collections, that certainly is a downside.

I was actually hoping that this would allow us to make non-blocking lock-free structures. I'm not quite sure how it would work yet, just a vague idea. I think it would involve a Span or similar object.

I tested your code some, and found that I like your design more than my previous ones. I like how it separates disconnectedness from optionality and uses Optional for optionality rather than needing a non-copyable version of Optional.
I was wondering if there is any particular reason you made it @unchecked Sendable and made the value nonisolated(unsafe). It sounds like you've used this in practice, so I was wondering if you have encountered a situation that requires it? Also, is there a particular reason you use conformance to ExpressibleByNilLiteral & SendableMetatype instead of generic unwrapping where Value == U? to represent Optional?
After testing it a bit, I found that the withValue(_:) method is unsafe. The inout parameter in the closure should not be sending. sending tells the compiler that it is ok to retain the value or its internal state, which is not ok in this case.

struct Ref {
	var nonsendable: Nonsendable
}

func test() {
	let x = Ref(nonsendable: .init(name: "x"))
	var d = Disconnected2(x)
	let a = A()
	Task {
		let wrapped = d.withValue{
			$0 // it shouldn't let me return this
		}
		await a.useNS(wrapped.nonsendable)
		let ns = d.withValue{
			$0.nonsendable // it shouldn't let me return this
		}
		await a.useNS(ns)
	}
}

If I remove @unchecked Sendable, the compiler shows two errors. The one on withValue(_:), and one on the consume() method. I'm not sure why the compiler complains about the consume() method causing a data race, maybe it has to do with closures. I couldn't figure out a way to get it to allow consume() when Value is non-Sendable. If I only allow consume() when Value is Sendable, then it is ok.

This is what I came up with after removing @unchecked Sendable, (then I put it back):

public struct Disconnected<Value: ~Copyable>: ~Copyable, @unchecked Sendable {
	private var value: Value

	public init(_ value: consuming sending Value) {
		self.value = value
	}

	public mutating func swap(_ other: consuming sending Value) -> sending Value {
		let result = self.value
		self = Disconnected(other)
		return result
	}

	public mutating func take<U: ~Copyable>() -> sending Value where Value == U? {
		return swap(nil)
	}
	
	public mutating func withValue<R: ~Copyable, E: Error>(
		_ work: (inout Value) throws(E) -> sending R
	) throws(E) -> sending R {
		try work(&self.value)
	}

}

extension Disconnected where Value: Sendable & ~Copyable {
	public consuming func consume() -> Value {
		value
	}
}

In the Xcode 16.4 toolchain, I have issues with unwrapping an Optional after sending. It says that the wrapped value is isolated to the Traveler actor that it came from. When I use a actor Traveler ... func trySend() throws -> Nonsendable method on the Traveler instead, I have no issues. I don't have this issue in the Swift 6.2 development toolchain. This happens regardless of @unchecked Sendable on the Disconnected type.

Once again, thanks for sharing the code! I had set out to model a disconnected Optional, and incidentally ended up merging disconnectedness with optionality when that wasn't necessary. Approaching the problem from a different perspective, you came up with a cleaner solution. I'm glad to hear other people are thinking about this too. I think adding this feature to Swift will make actors and async functions more approachable for beginners.

Having worked on this more and seen alternative implementations, I feel reaffirmed in my belief that Disconnected should be provided by Swift. It is an important feature to be able to move non-Copyable objects from a synchronous context into an asynchronous context, and to move objects in and out of actors. It is difficult to design the Disconnected type correctly, and requires a considerable amount of expertise. There is little chance every beginner will get it right. So I think it would be best if a well designed version was provided by Swift.

You're right that withValue appears to be quite unsafe. Which surprises me, given Mutex uses the same formulation. Not sure what's going on there, but it feels like a compiler bug — the supplied closure is not sending R.