SE-0427: Noncopyable Generics

[Jumhyn (Frederick Kellison-Linn)]

@jrose replied above pointing out how that simplification is not entirely correct.

(Though I do think this is how many users will end up thinking about ~Copyable on a non-generic concrete type because I expect exceptions to be very rare/unusual).

[carlynorama]

I'll admit when I first saw ~Copyable, I was skeptical. I have since come to appreciate value-type-as-lego while still preserving safety potential here, and I'm very excited.

Both ~ and Copyable will remain really hard to talk about for a long time though. Because of really simple sentences like:

Classes have additional capabilities that structures don’t have...

from the documentation Documentation

This is how pretty much everyone teaches this. Value types are the base and reference types are more. The problem comes that the average reader will parse this as reference types have been given extra powers. That classes have more moving parts to them than structs.

Those of us who learned or spend a lot time in environments that have no value types maybe don't remember what it's like to really not know that value types are a gift. They were made to make a lot of things easier and safer by putting in guards rails and adding default behaviors. In environments without them the programmer always has to keep track of this stuff and it's a relief to work in a environment where that gets taken care of for you when you want it to.

Most modern languages have value types, and they sensibly get taught first. So now for a huge swath of people who program, they experience value types as "the base" and reference types as a value types that have had features bolted on to the them. The idea that value types are just as much of a construct and that reference types have "more abilities" because they have fewer/different apparatuses bolted on to them is totally foreign. (simplification, true)

Past code does need to work; value types will still prefer to have copying. I understand both the benefit and the need for the implicit behind the scenes :Copyable addition. So there needs to be a mechanism to stop it from happening. Enter ~.

That copying is a choice that value types make is going to be a surprise. ~Copyable will absolutely feel like it's putting a parking boot on the Copyable wheel, not removing the Copyable wheel, because most don't have an internalized sense that the wheel was invented, that it is an added thing, that it can be removed.

Swift, understandably, adds lots of different parts at the factory, but invisible parts. So in the case for this Copyable wheel any talk about "It's not a boot, its saying you don't want that wheel. It's like working with any other wheel." makes no sense because essentially the buyer thinks they've ordered a hovercraft because they don't see the wheels... "What wheels? It just moves?"

This is complicated by the fact that there are two needs, potentially. The ability to blanket decline particular factory installed parts and the ability to prevent a specific type of part from being installed by anyone.

The "Car Engineers" on this forum understandably want to talk about the new apparatus around deciding about the Copyable wheel and the mechanism around declining factory installed parts and declining parts in general (it's very cool). But it's super confusing to so many people because most of the people still don't understand that wheels exist, that factory installed parts are parts not molded in, and that some of them are invisible.

For many things it's fine to treat the documentation as secondary. As something that comes after the pitch, after the implementation, but this is such a new concept to so many people in the Swift user base. So deeply fundamental. And it wasn't even treated as full number release (because it didn't have to be, I get semantically why). It's totally not even hinted at anywhere in the documentation even though ~Copyable is actively already being used. Copyabilty has been factored out of value types. That's "the earth is not the center of the universe" level in this context. Is it right to say that most programers day to day experience in Swift won't change? Yeah. But it's more than just "a detail that can be explained later".

And the ~ thing is also hard in its own right, even if one gets about the wheels, because it's difficult to hold in mind where all the places the invisible parts get added. To know what you don't know. (An IDE feature to "show implicit conformances" kinda like was VSCode does in some cases might be really helpful.)

I probably sound like a broken record and I'll stop now. But I think for this particular feature the documentation absolutely cannot be an after thought, and what might feel like "progressive disclosure" to some will feel like falling off a cliff to many others.

Edited: So many typos! I caught some of them.

[Jumhyn (Frederick Kellison-Linn)]

@wadetregaskis

If S cannot be made Copyable even by an extension, then that is completely novel behaviour in the language (and IMO a much more complicated protocol conformance model than what we have now).

It is worth emphasizing that while a Copyable conformance may be introduced by extension, it cannot be done in an unrestricted manner:

Conditional Copyable conformance must be declared in the same source file as the struct or enum itself. Unlike conformance to other protocols, copyability is a deep, inherent property of the type itself.

So it is not possible, for instance, for a user to define a retroactive conformance to Copyable for a type they do not control.

[Ben_Cohen]

Note that this isn't entirely without precedent, e.g.

b.swift:2:1: error: extension outside of file declaring struct 'S' prevents automatic synthesis of '==' for protocol 'Equatable'
1 │ 
2 │ extension S: Equatable { }
  │ ╰─ error: extension outside of file declaring struct 'S' prevents automatic synthesis of '==' for protocol 'Equatable'

[Jumhyn (Frederick Kellison-Linn)]

Yeah, and even though for many protocols the compiler will (validly) accept a retroactive conformance which simply implements the syntactic requirements, it’s always been the case that one must ensure their implementations obey the documented formal semantics of the protocol (which someone other than the type author may not be able to do!).

Copyable is special in that the compiler has built-in knowledge about how those non-syntactic semantic requirements behave and how they are able to be satisfied, rather than just taking a retroactive conformance’s word for it.

[taylorswift]

@Jumhyn

It is worth emphasizing that while a Copyable conformance may be introduced by extension, it cannot be done in an unrestricted manner:

this doesn’t work for me at all, although the diagnostic could be improved

struct S:~Copyable
{
}
extension S:Copyable
//  cannot find type 'Copyable' in scope
{
}

to me, if it were possible to make something noncopyable copyable again, that would sort of defeat the purpose of marking a (concrete) type ~Copyable in the first place. i understand the concept is more nuanced for generic types, but for concrete types, it would undermine a lot of use cases if it were possible to backdoor a Copyable conformance to something ~Copyable.

[Alejandro (Alejandro Alonso)]

<source>:3:1: error: noncopyable struct 'A' cannot conform to 'Copyable'
1 │ struct A: ~Copyable {}
2 │ 
3 │ extension A: Copyable {}
  │ ╰─ error: noncopyable struct 'A' cannot conform to 'Copyable'

This is what I get at least on main

[wadetregaskis]

@taylorswift

to me, if it were possible to make something noncopyable copyable again, that would sort of defeat the purpose of marking a (concrete) type ~Copyable in the first place. i understand the concept is more nuanced for generic types, but for concrete types, it would undermine a lot of use cases if it were possible to backdoor a Copyable conformance to something ~Copyable.

Yeah, but it's also conceptually simpler if it behaves like any other protocol in this regard, e.g. Codable. It just so happens that instead of:

struct S {}
extension S: Codable {}

…the syntax is:

struct S: ~Copyable {}
extension S: Copyable {}

Limiting the difference to only whether it's on or off by default helps keep it tractable, especially for beginners or more casual users.

And if you do write extension S: Copyable - and it has to be in the same file, even - then, well, what were you expecting?

I think pragmatically there's not actually much room for confusion or error. And what room there is likely requires relatively complex or convoluted logistics (e.g. interplays with macros), in which case you kinda already have to be an expert in all the relevant parts of the language & compiler, in order to make it work correctly anyway.

[taylorswift]

@wadetregaskis

Limiting the difference to only whether it's on or off by default helps keep it tractable, especially for beginners or more casual users.

i appreciate there is mathematical elegance in having a dichotomy centered around abstract assumptions about types, and making this dichotomy fit into existing thought patterns about protocol conformances. but when it comes to everyday practical coding, i would much prefer a “dumb” trichotomy that might take the form of:

1. never Copyable
2. sometimes Copyable, depending on generics
3. always Copyable

[Jumhyn (Frederick Kellison-Linn)]

@taylorswift

this doesn’t work for me at all, although the diagnostic could be improved

struct S:~Copyable
{
}
extension S:Copyable
//  cannot find type 'Copyable' in scope
{
}

Ah i had thought the previous discussion indicated that this was already accepted, I should have checked. I tend to agree that this is confusing and undesirable, though I find Jordan’s point about macros compelling as well. There are a few design goals that end up conflicting:

1. When possible we should design syntax that is more consistent/composable to make it easier to write correct macros that work in all relevant cases.
2. Macro productions copy-pasted into actual source should continue compiling without any new diagnostics.
3. Composable syntactic rules can sometimes admit more pathological cases which are confusing in practice to human readers, and we should try to selectively disallow those cases when they arise.

I don’t see a way to reconcile these points—something has to give.

I certainly would never suggest anyone write code like your snippet above, but does the (3) downside outweigh the (1) interest here? I’m not sure.

[jrose (Jordan Rose)]

I didn’t check either, but my concrete suggestion is “downgrade this to a warning, and silence the warning if the code comes from a macro”.

[Jumhyn (Frederick Kellison-Linn)]

Yeah, I’ve suggested something similar in past situations where there’s been an undesirable syntax which is easier to generate systematically, but this runs afoul of goal (2) above. I’m still not totally sure that purity on (2) is always the right tradeoff to make against compromises on (1) and (3).

[wadetregaskis]

There's no such issues if you just consider extension S: Copyable acceptable.

If the intent is to require that copyability or not be specifiable only on the core type specification, then maybe it shouldn't be using protocol conformance, an existing mechanism which doesn't behave that way?

e.g. @nonCopyable struct S instead, or whatever.

In parallel, consider that a warning can always be added after the fact (such an addition isn't considered source-breaking, I believe). So it's also an option to just allow extension S: Copyable and just see how it goes; see if it actually turns out to cause significant problems.

[carlynorama]

There's a trickiness to this situation:

• ~ makes more sense when it is suppress implicit only
• People went to the effort to make Copyable a thing so it could be prevented for safety reasons (so "whats the point of ~Copyable if we can't do that...") so limiting Copyable to extendable only in the Source file is the compromise (generic associated type exception)?
• ETA: Now in generics its triply hard because it still needs suppressing, but also it sounds like people want to be able to use ~Copyable kind of like AnyObject, "I only want to take the type of thing that's MoveOnly", which having a ~ mean suppress-implicit-conformance only doesn't get them. (If ~ meant "without SomeProtocol" not "suppress implicit SomeProtocol" it could, and I think that is what most people at a casual reading take it to mean.) ETA: So what we're getting is a compromise where ~ in the generic context means "Promises not to use any SomeProtocol features" (Maybe we need another glyph? Ha. one that means "un-using") But team - "I need to forbid this for safety reasons" will naturally find this insufficient. (And the casual reader will find confusing because they thought it meant "without-without")

I don't know what could/should be done to decorate Copyable or other root-file-extensions-only things as different than "the usual protocol" to indicate its not the ~ doing that it's the thing itself with that limitation.

Unless that is part of what ~ is supposed to mean? "I only suppress implicit protocols that also have an added at the root only limitation"

[taylorswift]

@wadetregaskis

So it's also an option to just allow extension S: Copyable and just see how it goes; see if it actually turns out to cause significant problems.

this isn’t really the right approach to take when adding features to a mature language such as Swift. if it becomes possible to re-Copyable a noncopyable struct, people will start doing it, and removing it would be source breaking.

[wadetregaskis]

@taylorswift

…and removing it would be source breaking.

Not if it's merely a compiler warning.

[michelf (Michel Fortin)]

@taylorswift

1. never Copyable
2. sometimes Copyable, depending on generics
3. always Copyable

There is a certain value in thinking about types that way: "never Copyable" means a struct can have a deinit while "sometimes Copyable" means it cannot have one. The restrictions and capabilities are different.

This is also pretty close to what I suggested earlier in the thread:

1. ~Copyable — type is never copyable
2. ?Copyable — type is sometime copyable depending on generics (also causes extensions to be Copyable by default unless you suppress it with where Self: ~Copyable)
3. Copyable (the default)

[Slava_Pestov]

@taylorswift

if it becomes possible to re-Copyable a noncopyable struct, people will start doing it, and removing it would be source breaking.

Type metadata directly encodes whether a type is copyable or not, so its an intrinsic property of a type; a conformance to Copyable (conditional or not) can only be declared in the same source file as the type.

[taylorswift]

@Slava_Pestov

Type metadata directly encodes whether a type is copyable or not, so its an intrinsic property of a type; a conformance to Copyable (conditional or not) can only be declared in the same source file as the type.

i’m not sure how this relates to the revocability of extension T:Copyable?

[kavon (Kavon Farvardin)]

@wadetregaskis

In parallel, consider that a warning can always be added after the fact (such an addition isn't considered source-breaking, I believe). So it's also an option to just allow extension S: Copyable and just see how it goes; see if it actually turns out to cause significant problems.

But much more importantly: making it an error after the fact becomes impossible without a source break.

@wadetregaskis

Is this any different to any other protocol conformance via an extension, though? Special-casing Copyable in this respect would make the overall system more complicated.

I disagree.

I'd much rather it be an error to write an extension adding an unconditional Copyable conformance to a type marked with ~Copyable, as it is implemented and proposed today. I argue it is always a code smell as there are many ways, other than the dead-obvious extension discussed so far, to accidentally make the type always Copyable. Suppose you see this:

struct S: ~Copyable {}

extension S: P {}

You'd think S is still noncopyable, right? Not if we loosen the error! If we had this:

protocol P {}

then S becomes unconditionally copyable through that extension, because P inherits from Copyable (implicitly) and Copyable has no explicit requirements. That extension was obviously a mistake!

Furthermore, if you want to permit this:

struct S: ~Copyable {}
extension S: Copyable {}

Then you must also permit this awful thing:

struct S: ~Copyable, Copyable {}

As they are equivalent rewritings.

@Jumhyn

I tend to agree that this is confusing and undesirable, though I find Jordan’s point about macros compelling as well.

I think macros are going to also stumble upon the same problem that I just described, where it accidentally introduced an unconditional Copyable conformance with an extension. It'll just be a bit more hidden since the extension isn't explicitly written.

Hypothetical macros that would benefit from allowing such extensions haven’t yet been demonstrated, so I think we should not change this proposal to accommodate them.

@wadetregaskis

If S cannot be made Copyable even by an extension, then that is completely novel behaviour in the language (and IMO a much more complicated protocol conformance model than what we have now).

Not really. It's already possible to write requirements that conflict with each other:

protocol P {}
func f<T>(_ t: T)
  where T == Int, T: P {}
// error: no type for 'T' can satisfy both 'T == Int' and 'T : P'

What's been proposed is that T: ~Copyable and T: Copyable conflict with each other if they completely overlap. Creating an exception for nominal types and their extensions actually is the special case that I think lacks justification for the additional confusion it will cause.