SE-0427: Noncopyable Generics

Right - if the caller doesn't need the value anymore, it can just pass it to the function without any copying.

I guess I'm not certain, but I thought the final decision on consuming was that it does not require the caller to relinquish the original [copy] if it doesn't want to (unless the type is non-copyable). I don't think a copy keyword is required at the callsite - I think it's implicit - but I could be wrong. I remember that being discussed.

I work a lot with state machines. A pattern with statically typed states become available with move-only types, here are some details and example: Typestate - the new Design Pattern in Swift 5.9 | Swiftology

I want to create library which helps to work with states, and I want library to guide users in a right way: use move only types, not Copyable types or classes.
Another one reason is exit action – the action which is performed on state exit. It can be done in dinit which can be declared in move-only types and classes, but classes should be discouraged.

I encourage everyone to give this proposal a test drive with a nightly toolchain and -enable-experimental-feature NoncopyableGenerics. Perhaps try to write a generic Maybe<T> type like Optional that is conditionally copyable and post your feedback! :slight_smile:

Here's what you'll get:

test.swift:1:31: error: parameter of noncopyable type 'T' must specify ownership
1 │ func foo<T: ~Copyable>(_ bar: T) {}
  │                               ├─ error: parameter of noncopyable type 'T' must specify ownership
  │                               ├─ note: add 'borrowing' for an immutable reference
  │                               ├─ note: add 'inout' for a mutable reference
  │                               ╰─ note: add 'consuming' to take the value from the caller
5 Likes

I'd like to try and address some of the syntactic confusion I've seen in this discussion from a different angle. We have the expectation that the following assertion is always true:

func comparing<T: Equatable>(_ t: T) {
  assert(t is any Equatable)
}

What trips people up is reading ~Copyable as "not Copyable". Because it means thinking this assertion is also always true:

func moving<T: ~Copyable>(_ t: T) {
  assert(!(t is any Copyable))
}

But it is not. You can substitute both Copyable and non-Copyable types into moving. The assertion that is always true inside moving is assert(t is any ~Copyable), but it's trivally true for every type. It's like the new version of t is Any in the post-NoncopyableGenerics universe.

The ~ makes people think "not", but it's actually a suppression, or "subtraction", of a requirement that is there by default.

I wouldn't be opposed to moving towards -Copyable instead, to help nudge people to think subtraction instead of negation. It'd be intuitively read "minus Copyable". Yes, we'd have to probably deprecate the syntax from SE-390, which did allow ~Copyable in an inheritance clause. But that's probably for the better.

Another alternative is to keep ~ but avoid using :, which people typically read as "is", and instead something like where T ~ Copyable to subtract or remove the Copyable requirement from generic parameter T.

Thoughts?

3 Likes

To your prior point, wouldn't that need a consuming / borrowing / inout in there?

For constraint clauses, I like the direction but I fear that's a lateral move at best - it could still be read as "minus Copyable, so not Copyable, right?". ~ is arguably better there because it's intuitively closer to the maybe aspect that is key in those clauses.

But for conformances, I think it is significantly clearer. "struct S minus [the normally included by default] Copyable conformance" as opposed to "struct S and… maybe Copyable?" or "struct S is… approximately Copyable?!" etc.

I'd previously suggested using ! in conformance statements specifically, for the same reason. It and - are stronger statements and more accurate to the intent.

Using different prefixes for clauses vs conformances helps emphasise that the very nature of those two contexts is different ("what the type actually conforms to" vs "what conformances a type must have to fit into a given generic").

2 Likes

Yes. I'm not a compiler so I didn't catch it. :slight_smile:

That's the goal. Make people think of it as a suppression / removal / absence of Copyable and then have to take that second step to consider what that means. "T minus Copyable means it doesn't have a Copyable requirement. So that means I can't copy whatever I get from callers, whether I get something Copyable or not".

1 Like

Regarding use-cases for "where T is not Copyable", I can imagine that being useful for something like e.g.:

struct Vault<Element> where Element: !Copyable {
    func deposit(_ item: consuming Element) { … }
    func withdrawAny() -> consuming Element? { … }
}

One could argue that semantically this 'Vault' type doesn't make any sense with copyable elements, because its whole purpose is to sequester away the elements, and if the elements can be trivially & implicitly copied then what's the point?

One might argue that it's harmless if copyable types can be used with it too, even though that's not the intent, but I'm not sure that is harmless - it allows logic errors.

This relates to…

The question seems to be whether there's a valuable distinction between language mechanics and program semantics.

It's a bit like how you can make a protocol tied to a global actor - the whole protocol, not individual members - and therefore require that global actor isolation on all conformers of the protocol. Even though mechanically that's unnecessary, since technically the Swift language only cares whether individual members have isolation requirements.

1 Like

A type such as struct Vault<Element: ~Copyable> { ... } according to this proposal only promises it will not copy the Elements you give it. I don't see a problem with there being both Vault<Int> and a Vault<FileDescriptor>.

My guess is that you're scratching at a Vault<Element> that rejects deposits of Elements that are already stored, but want to rely on noncopyable types to guarantee uniqueness.

A type being noncopyable doesn't necessarily imply that all values of that type are unique. You still need Equatable to check if two noncopyable values are semantically equal.

1 Like

That's why I used a hypothetical ! prefix to denote not Copyable, not merely maybe Copyable.

And what's the point of Vault<Int>? You can deposit the same value endlessly, and now the vault has to semantically define what it even means to deposit the same thing repeatedly - e.g. does that have no effect, or does it mean you can withdraw the element multiple times too? If not the latter, it also now has to check for duplicates, which hurts its implementation (no more simple array, it has to be a hash table or somesuch - and now the Element has to be Equatable and Hashable as well, in order to do that).

To be clear, in this Vault example the idea is that it lets you know that any elements it contains cannot be in use anywhere else in the program. It's not merely a 'dumb' collection, it's a way to reason about the state of the program and enforce invariants at compile time.

You can try to enforce uniqueness that way, but in a way it's simpler to just use non-copyable types. Then you know that intrinsically they are unique, without relying on some kind of internal identifier and runtime equality check. And they can then be value types too, not reference, which can be beneficial for performance.

As of last night's (the 19th) nightly toolchain, the standard library now has support for noncopyable generics on its pointer types and on Optional (gated under the -enable-experimental-feature NoncopyableGenerics of this proposal – though there is also another step needed which is to run a review for that addition, pitch tk).

This is very relevant for folks wanting to try out this feature, since it's very hard to do anything useful with noncopyable generics without these basic building blocks.

To see a simple example of this adoption, here is a PR I put up on the Swift Playdate sample code that was recently released. This adds a very basic non-copyable array type, and then uses that to store the brick sprites.

Previously these sprites had to be represented as an enum (indicating ownership) plus a class (to handle deallocation). The noncopyable version of that pattern is significantly simpler: it just holds the underlying opaque pointer (the playdate SDK's sprite handle) plus a bool to track ownership, then uses a struct with a deinit to free it if necessary.

The PR also adds a basic array type to hold the brick sprites. This is partly because Swift.Array does not yet support non-copyable elements (doing so requires more discussion of what it means for the collection protocol hierarchy) but also because adopting a much simpler type that does not need to worry about ref counting or growing has meaningful benefits to binary size in an embedded context.

Note, please direct discussion of this change itself to either the PR or the thread on the playdate example rather than this review thread. But of course, feedback on the proposal itself citing examples from that code are welcome here.

10 Likes

Thanks for the fresh ideas and writeup Kavon!

Since this came up from a chat I had with Kavon I wanted to also publicly share that I think that’s a very nice rephrasing because it nicely denotes that we’re “removing” (with ~, close enough to a minus sign) the Codable here — so yeah “this is a T without the Copyable even if the T had it”. This reads much more logical to me than trying to smoosh in the : “conforms to” with the “~SomeType” “but without that”.

It makes sense to me much more than using the conformance syntax. Type declaration sites can stay as they are but the use in type aliases or generic types as the “T ~ Someting” would be quite interesting to entertain as the “without”.

I wonder if people on the thread who were confused about the <T:~Copyable> (and I count myself as one of such, having been confused in other proposal reviews a few times when this spelling had come up, eg in the recent Mutex) would see this as an improvement?

In types there is an interesting parity with & as well: “T & Copyable” and “T ~ Copyable” seem interesting to be able to express :thinking:

1 Like

What would that look like for non-trivial cases? e.g.

func foo() -> consuming some Thingie & Hashable ~ Copyable

Is that right?

It feels a bit weird to me… maybe just because it's new… but it sort of breaks the simple structure of using '&' as a delimiter. It requires ~ be read as "and not necessarily", which is a slightly bigger interpretative leap than is required for & ~ (& is literally "and", so ~ only has to carry "not necessarily").

It also seems to imply a restricted ordering on protocol names…? Or else inconsistent syntax?

func foo() -> consuming some ~ Copyable & Thingie & Hashable

I can't quite elucidate it, but something about that feels a bit off to me. Maybe because visually it kinda looks like the ~ is just a visual separator between the keywords and the protocol names? It's not completely clear (intuitively) that it applies specifically and only to Copyable.

1 Like

Yeah in the original discussion around ~Copyable the idea of "apply the operation to : rather than to the protocol name" came up and while I also found it somewhat compelling I think it doesn't compose as well, and the question of any ~Copyable still comes up. If we end up needing to write ~Copyable anyway, I think it is nice to use the same syntax everywhere.

5 Likes

Is this not just a property of using, say, a private value of a concrete type which isn't Copyable? Justifying Vault on this basis feels somewhat circular.

You're assuming pointer equality for all noncopyable types. But that is not the only notion of equality. So even if your !Copyable really did only permit noncopyable values, your set-like Vault data structure still doesn't make sense for all noncopyable types without Equatable to establish a consistent notion of equality:

struct FileDescriptor: ~Copyable, Equatable {
  let id: Int

  static func ==(_ a: borrowing Self, 
                 _ b: borrowing Self) -> Bool {
    // using structural equality!
    return a.id == b.id
  }  
}

let a = FileDescriptor(id: 0)
let b = FileDescriptor(id: 0)
assert(a == b)
let v = Vault<FileDescriptor>()
v.insert(a)
v.insert(b)
assert(v.size() == 1) // fails
4 Likes

At the risk of further confusion by adding yet another sigil, aping Rust's syntax and using ?Copyable to indicate "Copyable can't be assumed here" could help reduce confusion: struct FixedArray<Element: ?Copyable>: ?Copyable {} rather than struct FixedArray<Element: ~Copyable>: ~Copyable {}. It would be applicable to functions too: func transmogrify<T: ?Copyable>(_ t: T) -> T

The suppression syntax ~Copyable is still IMO better for unconditionally uncopyable entities, like struct FileDescriptor: ~Copyable {}.

3 Likes

In the spirit of trying out the build with basic building blocks:

struct PerfectBinaryTree<T: ~Copyable>: ~Copyable {
    var left: T
    var right: T

    init(left: consuming T, right: consuming T) {
        self.left = left
        self.right = right
    }

    consuming func map<U: ~Copyable>(_ f: (consuming T) async -> U) async -> PerfectBinaryTree<U> {
        await .init(left: f(left), right: f(right))
    }
}

yields:

     consuming func map<U: ~Copyable>(_ f: (consuming T) async -> U) async -> PerfectBinaryTree<U> {
         await .init(left: f(left), right: f(right))
                           ╰─ error: cannot partially consume 'self'
     }

     consuming func map<U: ~Copyable>(_ f: (consuming T) async -> U) async -> PerfectBinaryTree<U> {
         await .init(left: f(left), right: f(right))
                                           ╰─ error: cannot partially consume 'self'
     }

And I can't figure out how to use copy or discard in a way that avoids the partial consumption. what am I doing wrong?

1 Like

You can also -enable-experimental-feature MoveOnlyPartialConsumption to allow it. That proposal is also under review now.

6 Likes

ah... thx! will try that out.

Also reccomended is -enable-experimental-feature BorrowingSwitch which was pitched here

6 Likes