[Pitch] Noncopyable Generics

[localhost]

Given the broad implications, perhaps we should have a dedicated discussion for programmer-defined defaults.

In particular, the syntax default extension where Value: Copyable feels like it's tailored specifically to Copyable

[simonomi (alice pellerin)]

First, I would like to throw my own 'english translation' of the ~Copyable syntax into the ring: "not necessarily Copyable".

I too find that reading ~Copyable as "not Copyable" is a very easy mistake to make, but it's been pretty easy to fix that by amending it to "not necessarily Copyable".

---

@Karl

But even with that, personally I don't think it's that bad to write those constraints explicitly. I wonder if it is truly worth the added complexity of yet another magic inference rule.

- struct Pair<Elm: ~Copyable> { ... }
+ struct Pair<Elm: ~Copyable>: ~Copyable { ... }

+ extension Pair: Copyable where Elm: Copyable {}

I strongly support removing this inference rule, especially when looking at other protocols such as Equatable or Hashable. The standard library, for instance, already has conditional conformances to these for Optional:

extension Optional: Equatable where Wrapped: Equatable {/* ... */}

extension Optional: Hashable where Wrapped: Hashable {/* ... */}

and even

extension Optional: Sendable where Wrapped: Sendable {}

As such, it seems consistent and reasonable to require

extension Optional: Copyable where Wrapped: Copyable {}

It is a bit boilerplatey, but I think @xwu's suggestion of adding a macro is much less confusing than adding unexpected and confusing (at least to me) language-level inference.

[Karl]

It's also worth remembering that if you're using non-copyable types, your code is already required to be much more explicit with regard to things like ownership.

The proposal even calls this out:

As with a concrete noncopyable type, any generic type parameter that does not conform Copyable must use one of the ownership modifiers borrowing , consuming , or inout , when it appears as the type of a function's parameter. For details on these parameter ownership modifiers, see SE-377.

And provides examples, which I'm abbreviating:

// T is Copyable, things are easy.

func genericFn<T>(_ t: T) -> T

// T is ~Copyable. Now you also need to specify ownership.

func identity<T: ~Copyable>(_ t: borrowing T) -> T 
//                               ^^^^^^^^^

struct Pair<Elm: ~Copyable> {
  func exchangeFirst(_ new: consuming Elm) -> Elm? { /* ... */ }
  //                        ^^^^^^^^^
}

I think this - scaled across all of a type's member functions - is likely to be the biggest source of "syntactical noise", so to speak. What I mean by that is that I expect it will be the biggest difference between code which requires Copyable types and that which doesn't, requiring developers to adjust their habits to account for these details that used to be implicit.

Weighed against that, I don't think the need to write a conditional Copyable conformance is a big deal; I just don't expect anybody to come back and say that is the thing that was difficult for them to adjust to.

[kavon (Kavon Farvardin)]

@wadetregaskis

~ means literally the inverse, mathematically, which means if something was copyable then ~copyable means not copyable. For our purposes here it's a synonym with ! because there's no two's complement of a concept.

If it were meant to mean "maybe" then it should have used ?, following the established pattern for that symbol in Swift.

I personally would like to push-back on folks referring to ~Copyable as "maybe copyable" because it does not make sense with respect to the rest of the generics system.

When you define a type variable, you're specifying what it's capabilities are within the scope of that type variable via conformance or same-type requirements. To fully spell that out in prose, we say that "T is some type that conforms to Codable, within the body of f" in the following example:

func f<T>(_ t: T) where T: Codable {
  // T is Codable and Copyable here, and nothing else!
}

In fact, Codable is the only thing you can statically assume of T, outside of just being Copyable. Thus, it does not make sense to talk about what other conformances a type that might be substituted for T could have. All f has required is that T supports Codable (and implicitly, Copyable). There's no static proof it is "maybe" anything else.

For example, consider this caller:

public func caller<Arg>(_ arg: Arg, _ b: Bool)
  where Arg: AnyObject, Arg: SetAlgebra, Arg: Codable {
  if b {
    f(arg)
  }
}

We don't say that, because caller has a call to f and its Arg conforms to AnyObject & SetAlgebra & Codable, that T is "maybe AnyObject and maybe SetAlgebra". Dynamically, T "maybe" conforms to every protocol under the sun! But you can't statically prove that. Perhaps f is never called by caller. All you know about T is that it's Codable & Copyable, because the type checker has ensured that every call-site at least provides a value that supports it.

This is why I think it is fine to refer to ~Copyable as "not copyable", because a type constrained to it is in fact not Copyable within the scope of the constrained type:

func id<V: Codable & ~Copyable>(_ x: borrowing V) -> V {
  return x   // error: 'x' is borrowed and cannot be consumed
}

id(10)
id(FileDescriptor())

Whether you pass a copyable value or noncopyable one into id, that function cannot copy any values of type V; there's no maybe about it! Thus, "V is Codable and not Copyable" is an accurate way to think about it.

[Jon_Shier]

If I recall the evolution discussion correctly, the "maybe" terminology was given both as a post hoc justification for using ~ rather than something like ! that would more naturally fit "not", and also to distinguish it from the true negative constraints that people have asked for in Swift for years. If you communicate to users that ~Copyable just means not Copyable, they will naturally expect to use it any time they want to express a negative constraint. Since the language won't be offering that capability, using different language helps distinguish the features. If you dislike "maybe", you can also use "unknown" or "can't assume" or other phrases. But I still think it's important not to simply call it "not" so that some explanation is required when users encounter it. Perhaps "inverse" or "opposite", given that there are additional capabilities unlocked, unlike basic negative constraints?

[Nevin]

“~Copyable” is properly read as “implicn’t Copyable”

[Slava_Pestov]

@xwu

Also, if I understand correctly, given protocol P: ~Copyable, then for a conforming type:

struct S: P { }

...S is intended by this pitch to be Copyable using the same rationale, but it is not demonstrated explicitly in the pitch examples.

S is always copyable independent of its conformance to P. To make S non-Copyable you must write S: ~Copyable.

@xwu

Supposing that we take the pitch as stated such that S is implicitly Copyable just as Q is, what of the following?

struct T<U: P> { } // `U` is constrained to `P: ~Copyable`

By the logic of constraints, U must not be implicitly Copyable; rather, U: P implies U: ~Copyable even though Q: P above implies Q: P, Copyable. Tricky, but learnable.

The generic parameter U of T here is implicitly Copyable because all generic parameters are. If you want to opt-out you have to write

struct T<U: P & ~Copyable> {}

T: ~Copyable does not constrain T to be non-Copyable. It is sugar that elides the default behavior of T: Copyable and it must be written at the declaration of T. The inference rules for generic requirements do not change with this proposal because T: ~Copyable is not a real requirement, it is desugared very early.

[kavon (Kavon Farvardin)]

@xwu

Since we only allow one sort of inference but not the other, I expect this would lead to surprises for users if two structs, generic Foo<T> and nongeneric Bar, have essentially identical declarations and constraints, but one "just works" and the other doesn't. Pair that with different rules for classes, and I think this rightly gets quite confusing.

@Karl

Even if a developer understands that structs and generic types and such are assumed to be Copyable, I don't think it's going to be immediately obvious why Pair isn't Copyable here. Especially when things seem to work when they use a class instead of a struct.

I think it's worth thinking about these senarios a bit with the proposed rule.

First, A cannot contain NC:

struct NC: ~Copyable {}

struct A {
  let nc: NC // error: 'A' cannot conform to Copyable with noncopyable storage 'nc'
}

This naturally comes from SE-390, and you're required to add ~Copyable to A. The same thing happens if it were generic in B with a fixed noncopyable value:

struct B<T> {
  let t: T
  let nc: NC // error: 'S<T>' cannot conform to Copyable with noncopyable storage 'nc'
}

There was nothing conditional to infer in either case: A and B cannot be Copyable at all because it has a fixed type NC that is noncopyable.

What's new is that T can have have it's Copyable requirement removed. So now whether D here could be Copyable depends on what types you substitute for it due to the containment rule:

struct D<T: ~Copyable> {
  var t: T
}

If someone wrote just the above but without the proposed inference rule, they will get an error about D not conforming to Copyable because of its storage t. The only remedy would be to write ~Copyable on the type as well:

struct E<T: ~Copyable>: ~Copyable {
  var t: T
}

Because nothing's been explicitly said about D's ability to be copied, I think it makes sense to be smarter there. There's no reason to force people to write out E unless they want to override the conditional conformance with something else (or just omit it). They can be explicit about both to make it clear:

struct F<T: ~Copyable>: Copyable {
  var t: T  // error: 'F' does not conform to Copyable due to noncopyable storage 't'
}

struct G<T: ~Copyable>: ~Copyable {
  var t: T
}

let x: G<Int> // 'x' is not copyable

@xwu

If that's found to be too repetitive down the line, it seems this would be an ideal use case for a macro, something like:

@ConditionallyCopyable
struct Pair<T: ~Copyable> { ... }

// expands to:
struct Pair<T: ~Copyable>: ~Copyable { ... }
extension Pair: Copyable where T: Copyable { }

The main issue I have with this is its discoverability. You will often write this Pair type correctly the first time if you have the proposed inference rule. Whereas with this macro approach, the compiler will give an error with this type:

struct Pair<T: ~Copyable>  { 
  let x: T
  let y: T
  let token: Token<T>
 }

struct Token<V: ~Copyable>: ~Copyable {}

If the suggestion to users is to add @ConditionallyCopyable, they get into more trouble, because that won't necessarily fix it either. My understanding is that a macro cannot determine what protocols a type conforms to, so it's hard in general to determine if Token<T> is Copyable.

With the proposed inference rule, now users only have to write ~Copyable on Pair when it's truly required.

@Karl

I think this really needs diagnostics which explain the inferred conditional-copyability.

Here's what that looks like for the Pair-with-a-Token-in-it example, in a branch I'm working on:

test.swift:4:7: error: stored property 'token' of 'Copyable'-conforming generic struct 'Pair' has noncopyable type 'Token<T>'
  let token: Token<T>
      ^
test.swift:1:16: note: generic struct 'Pair' has '~Copyable' constraint on a generic parameter, making its 'Copyable' conformance conditional
struct Pair<T: ~Copyable>  { 
               ^
test.swift:1:8: note: consider adding '~Copyable' to generic struct 'Pair'
struct Pair<T: ~Copyable>  { 
       ^
                           : ~Copyable
test.swift:7:29: note: generic struct 'Token' has '~Copyable' constraint preventing 'Copyable' conformance
struct Token<V: ~Copyable>: ~Copyable {}
                            ^

[xwu (Xiaodi Wu)]

@Slava_Pestov

The generic parameter U of T here is implicitly Copyable because all generic parameters are. If you want to opt-out you have to write

struct T<U: P & ~Copyable> {}

Thanks, I had misunderstood this point after reading the proposal—it is an important one.

I can see why one would settle on that design, and there's a nice consistency to it where any constraint that's not explicitly noncopyable is implicitly copyable. To my mind, this makes implicit noncopyable-inference-from-constraints even more of an odd duckling.

That said, I do worry that this design will lend itself to counterintuitive results, particularly when P: ~Copyable is a protocol whose conforming types are never or seldom semantically copyable:

There is an expectation—I think justifiable, and currently one that's met—that given a protocol Q one can pass any concrete value of a conforming type as an argument to func f(_: some Q). Indeed, until we had implicit existential opening, one of the most common questions on these forums was why one couldn't pass an instance of any Q as an argument. It is nice that we have finally smoothed over that rough edge.

The result of the rule pitched here is that we're introducing another flavor of the same surprise: a user can't pass a value of just any conforming type as an argument to func g(_: some P). Indeed, if it doesn't make sense for P-conforming types to be copyable, then users can't pass any value of any type at all!

[dmt (Dima Galimzianov)]

@kavon

If someone wrote just the above but without the proposed inference rule, they will get an error about D not conforming to Copyable because of its storage t. The only remedy would be to write ~Copyable on the type as well

But that's the point, right? Explicit vs implicit. Without the rule compiler will emit error

struct D<T: ~Copyable> { // Struct 'D' cannot contain a noncopyable type without also being noncopyable
  var t: T
}

And the programmer will take responsibility to suppress Copyable aware "why". Or will make the property computable. Anyway it will be their decision.
But when type of a stored property is copyable the compiler won't complain.

struct J<T: P & ~Copyable> { // ok
  var u: T.U // T.U is known to be copyable
}

With the rule however J won't be Copyable for no reason, and it's impossible to say was it a conscious decision by the programmer or it was just overlooked.
IMO, the compiler shouldn't make decisions behind programmer's back when it can't be 100% sure of correctness of those decisions. And in this case it can't. Diagnostics are much better way even if there's only one way to fix an issue.

[Slava_Pestov]

@xwu

The result of the rule pitched here is that we're introducing another flavor of the same surprise: a user can't pass a value of just any conforming type as an argument to func g(_: some P). Indeed, if it doesn't make sense for P-conforming types to be copyable, then users can't pass any value of any type at all!

Yeah, some P would actually mean some P & Copyable, just like named generic parameters default to Copyable, any P is actually any P & Copyable, and so on. I believe this is the only way we can retroactively adopt ~Copyable in existing protocols like Equatable though without changing the meaning of existing code.

[ksluder (Kyle Sluder)]

@xwu

The result of the rule pitched here is that we're introducing another flavor of the same surprise: a user can't pass a value of just any conforming type as an argument to func g(_: some P). Indeed, if it doesn't make sense for P-conforming types to be copyable, then users can't pass any value of any type at all!

If it doesn’t make sense for P-conforming types to be Copyable, then P should be declared ~Copyable and this issue will not arise, because the parameter will need to be declared borrowing or consuming.

[xwu (Xiaodi Wu)]

@ksluder

If it doesn’t make sense for P-conforming types to be Copyable, then P should be declared ~Copyable and this issue will not arise, because the parameter will need to be declared borrowing or consuming.

I'm not sure I understand. Given the design pitched here and a protocol P: ~Copyable to which no copyable types conform, a function func f<T: P>(_: borrowing T) would be uncallable because T: Copyable is an inferred constraint.

[ksluder (Kyle Sluder)]

You’re right, this goes back to the subtlety that you picked up on from @Slava_Pestov’s post.

[dmt (Dima Galimzianov)]

Is it valid at all in theory for a protocol to require a type to have no conformance to Copyable? Clearly it wouldn't be a physical requirement (as protocols can't require stored properties), and I can't think of any semantical use of such constraint.

[Jason_Gregori]

It took reading all the comments here but I think I “get” it now.

protocol P: ~Copyable {}

really doesn’t constrain any uses of P anywhere to allow non copyable types. It ONLY allows non copyable types to conform to it. Then anywhere you want to allow non copyable Ps, you must add ~Copyable which will allow non copyable Ps and make the compiler enforce things.

The above makes it so we can make protocols like Comparable allow non copyable conformance without breaking backwards compatibility and requiring changes to all use sites of Comparable. Once Comparable conforms to ~Copyable, all the previously created extensions and functions will continue working without supporting non copyable types. Helper extensions that take on some of the requirements will need new ~Copyable versions OR the adopting non copyable type will have to provide its own version.

Without these rules every extension or function anyone has ever written would have to be rewritten to support non copyable types which would break abi stability AND is not feasible.

The one case I really didn’t get until it finally clicked was:

protocol U: P {} // U is copyable even though P is not. 

function f(x: P) {}

How can I pass a U into f?? It’s a P but it is copyable and P is not copyable! Aha: f defaults to Copyable and doesn’t support non copyable x’s. If we wanted to support non copyable x’s it would have to add ~Copyable. It makes a lot of sense if I think of Comparable and wanting to write a function that takes Comparable but NOT have to deal with non copyable types.

Did I get all that right?

[Karl]

@Jason_Gregori

protocol P: ~Copyable {}

really doesn’t constrain any uses of P anywhere to allow non copyable types. It ONLY allows non copyable types to conform to it. Then anywhere you want to allow non copyable Ps, you must add ~Copyable which will allow non copyable Ps and make the compiler enforce things.

Yeah, I'm not sure it's really meaningful or necessary to have protocols which refine ~Copyable, since as you say it doesn't carry through to any actual uses of the protocol, which all need to restate that they support non-copyable types.

From the proposal:

Protocols and their associated types default to carrying an implicit Copyable conformance requirement:

// signature <Self where Self: Copyable, Self.T: Copyable>
protocol Foo /* : Copyable */ {
 associatedtype T /* : Copyable */

I wonder if we could drop this, so that protocols are formally copying-agnostic, and instead we shift the Copyable constraint to every use of the protocol (which is actually already how the proposal works)?

I don't think we would want this kind of opt-in approach for non-escaping types, for instance. If a function accepts a value which conforms to Foo and promises not to escape it, then the function should be free to accept a non-escaping conformance to Foo. I'm not sure I agree that it is the protocol author's business to prohibit non-escaping uses from accepting non-escaping values; the thing using the value decides which capabilities it requires.

[Slava_Pestov]

@Karl

I wonder if we could drop this, so that protocols are formally copying-agnostic, and instead we shift the Copyable constraint to every use of the protocol (which is actually already how the proposal works)?

I agree this seems like an appealing simplification at first glance but unfortunately the protocol’s Self: Copyable requirement (or absence thereof) can be observed within the protocol body by using Self as a generic argument, eg

struct G<T> {} // where T: Copyable

protocol P { func f() -> G<Self> } // only valid if Self: Copyable

For this reason we cannot just allow (without breaking source) a non-Copyable type to conform to an arbitrary protocol unless the protocol’s author opted out of the default Self: Copyable.

[Slava_Pestov]

@xwu

Given the design pitched here and a protocol P: ~Copyable to which no copyable types conform, a function func f<T: P>(_: borrowing T) would be uncallable because T: Copyable is an inferred constraint.

Is this a new issue though? func f(_ foo: some Sequence & ExpressibleByIntegerLiteral) is similarly technically satisfiable but perhaps uninhabited in practice.

[Jason_Gregori]

@Karl

Yeah, I'm not sure it's really meaningful or necessary to have protocols which refine ~Copyable, since as you say it doesn't carry through to any actual uses of the protocol, which all need to restate that they support non-copyable types.

Ah, good point!

@Slava_Pestov

struct G<T> {} // where T: Copyable

protocol P { func f() -> G<Self> } // only valid if Self: Copyable

For this reason we cannot just allow (without breaking source) a non-Copyable type to conform to an arbitrary protocol unless the protocol’s author opted out of the default Self: Copyable.

Interesting, so this is the only reason the base protocol must conform to ~Copyable?

Was that mentioned in the proposal? (I might have missed it or it could have gone over my head)

In any case, that makes sense to me. So +1 for me!