SE-0427: Noncopyable Generics

Evolution Proposal Reviews
, ,
1

Hello, Swift community!

The review of SE-0427: Noncopyable Generics begins now and runs through March 22nd, 2024.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager by email or DM. When contacting the review manager directly, please put "SE-0427" in the subject line.

Trying it out

If you'd like to try this proposal out, you can download a toolchain supporting it for Linux, Windows, or macOS using the latest development snapshot from main at Swift.org - Download Swift. You will need to add -enable-experimental-feature NoncopyableGenerics to your build flags.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at:

https://github.com/apple/swift-evolution/blob/main/process.md

Thank you,

Holly Borla
Review Manager

33 Likes
2 
Targeted mechanisms are being developed to preserve ABI compatibility when adopting `~Copyable` on previously-shipped generic code.
This will enable adoption of this feature by standard library types such as `Optional`.
Such mechanisms will require extreme care to use correctly.

Will these mechanisms be available outside the standard library, or will changing an existing generic type to also support ~Copyable types always be an ABI-breaking change for third-party libraries?

2 Likes
3

I'm excited for this to land!

I know this has been covered in prior discussions, but with this one in particular I'm lamenting the choice of ~ as a short hand for "suppress this conformance". Even though this proposal spells it out pretty clearly and I know better, I still find myself reading this as "not copyable" (especially since ~ is used as the bitwise negation operator). Something like @suppress(Copyable) seems like it would create less mental load.

5 Likes
4

This part feels syntactically strange to me:

struct Pair<T: ~Copyable>: ~Copyable {...}
extension Pair: Copyable /* where T: Copyable */ {...}

I feel like it would be much clearer if explicitly spelling out Copyable in an extension like this would require that you explicitly specify either T: Copyable or T: ~Copyable for all possible constraints, so it’s clearer in what cases Copyable conformance is available. I recognize the benefit of assuming Copyable in most cases, but here where you’re already being explicit I think it would make sense to require you to be fully explicit.

13 Likes
6

Yeah, I agree. It seems odd to me that this syntax is used.

1 Like
7

When we say that the mechanisms to retrofit ~Copyable on existing generics will need extreme care to deploy, what kind of mechanisms are we looking at, and what are the consequences for getting it wrong?

1 Like
8

When you say here you are being explicit: this extension is right next to the type declaration, but the main use of extensions is to write them elsewhere, in other people's code than where the type was defined.

For example, a follow-on proposal to this one will adopt this change within types in the standard library such as Optional, which will become:

enum Optional<Wrapped: ~Copyable>: ~Copyable {
  case none, some(Wrapped)
}

This is an important change that will unlock, for example, being able to return an optional non-copyable type from functions. So a stack would be able to have pop() -> Element? even when those elements were non-copyable. Further out, Array and other collections will acquire the ability to hold non-copyable types.

Optional and Array are popular types to extend with all sorts of convenience methods. It would be source breaking to require all extensions on them to now need to explicitly state they were extensions where Element: Copyable.

But more important than the source break (which we could gate under a language version or upcoming feature flag), it would be the wrong thing to do. Non-copyable types are an advanced feature. As part of Swift's progressive disclosure, we need to keep the default of implicit copies in place for most users, and that includes when adding simple extensions on basic types.

Swift is not a language that should throw the concept of ownership in your face with even the most early code you try to write. Rather, these tools will be there when you need them, but should stay out of your way when you don't.

To take a more theoretical angle on why this is the right default, rather than the user experience one:

Copyability is the default in Swift, and when you want to explicitly suppress it, you use ~Copyable. But that suppression is not sticky. You suppress copyability in struct Stack<Element: ~Copyable> { } and so within the braces you don't get to assume that Element is copyable. But that doesn't mean that in an extension, that suppression persists. So when you write an extension, you need to restate it.

Here's a (perhaps tenuous) claim for some precedent for this: I think it's similar to how implicit unwrapping of optionals works. An IUO is an optional, but with a special property that you don't need to explicitly unwrap it:

let iuoString: String! = "hello"
print(iuoString + " world") // prints hello world

But when you assign it to a new variable, it doesn't stay implicitly unwrappable. If you want that, you have to restate it explicitly:

let maybeString = iuoString
print(maybeString + " world") // error: Value of optional type String? must be unwrapped
16 Likes
9

A question about the new Copyable protocol: what happens if you try to manually write a redundant Copyable conformance? Would it make sense for this to result in an error or warning?

// Copyable conformance is redundant, should this be an error / warning?
struct MyCopyableType: Copyable { ... }

And what about in cases where Copyable is used as a redundant generic constraint?

// Copyable constraints are redundant, should this be an error / warning?
extension Array where Element: Copyable { ... }

func onlyTakesCopyables<T: Copyable>(_ t: Copyable) { ... }
1 Like
10

Mostly, the consequences are that methods may change their mangling without applying any necessary "don't do that" annotations, causing an ABI break. If you are producing an ABI-stable library, you already need to be careful with basically any refactor that it not break ABI in this way, and need ways to check for any unintentional breaks. So from my perspective, I don't think this really introduces much additional risk beyond what ABI-stable library authors already have to handle.

Note that these annotations probably belong in a separate proposal that will allow existing ABI-stable libraries to apply these techniques, we should probably stick to the text of this proposal rather than discussing them here.

4 Likes
11

As others have said earlier in this thread, I also dont love the implicit Copyable requirement for generic type parameters in extensions:

struct Pair<T: ~Copyable>: ~Copyable {...}
extension Pair /* where T: Copyable */ {...}

I understand why something is needed to avoid breaking source for existing types we want to become ~Copyable. Instead could we leverage @retroactive to signal the Copyable requirement was dropped after the type was created, so extensions should assume Copyable. New ~Copyable types would not use this attribute and follow more obvious defaults.

With an existing type, Optional:

enum Optional<Wrapped: @retroactive ~Copyable> {
  case none, some(Wrapped)
}

// implicit `where Wrapped: Copyable` because the Copyable requirement was retroactively suppressed. 
extension Optional { ... }

With a new type Basket:

struct Basket<Fruit: ~Copyable> { ... }

 // Fruit is still ~Copyable as is naively expected 
extension Basket { ... }
6 Likes
12

I agree! I was only talking about the specific case where you’re conforming Foo: Copyable where I think you should be required to explicitly specify the copyability or noncopyability of all type parameters/associated types. For all other extensions where T: Copyable should be inferred for all type parameters/associated types T.

struct Foo<T: ~Copyable>: ~Copyable {}
// Error: must specify `where T: Copyable` or `where T: ~Copyable`
extension Foo: Copyable {}
// ok, assumes `where T: Copyable`
extension Foo: Equatable {}
5 Likes
13

The issue with this is that you don't know what implicit requirements are present in the following extension:

extension Foo {}

Is it where T: Copyable or where T: ~Copyable? You'd need to go lookup the type declaration. For some types this is ok, but for types in libraries you don't own, it's not as simple.

Having a consistent rule makes it obvious for every extension.

3 Likes
14

I agree partially, but I don't love the idea of needing to write:

extension Foo where T: ~Copyable, T: ~Escapable {}

All over the place, in the future.

3 Likes
15

We may be able to introduce a new typealias in the standard library (spitballing here, nothing concrete)

public typealias Nothing = ~Copyable & ~Escapable & ...

so you can say

extension Foo where T: Nothing {}

:person_shrugging:

1 Like
16

Oh I see! That seems like a very reasonable narrow counter proposal/tweak.

Anyway, I wanted to get that "this is why having to restate copyability is the right default" post out of my system, so thanks for being the prompt even if it was as a result of my misunderstanding :)

5 Likes
17

Can't say I love that, but it's certainly an idea. I wonder if there's another approach where you could tell the compiler to not assume Copyable / Escapable / Reflectable / etc... within a module and/or package.

This would keep all the same public source behavior, but within a package you could flip the defaults.

1 Like
18

I still thinks this suffers from someone reviewing a brand new codebase not knowing what that project's default is and having to lookup potentially compiler flags to figure it out..? It might also cause vertigo for some folks swapping between modules who have 1 default and another have the other which would be very confusing.

19

I think the overall rationale you've stated here is absolutely correct—and exactly as you say, not just from a source stability standpoint but also from a progressive disclosure standpoint. That is, even if we had no existing source code to worry about, I think we'd want some inference along the lines proposed.

I do want us to consider if we can rationalize and narrow the rule from that proposed even further than @j-f1 stated.

One of my nits here is that, as proposed, we are further diverging the behavior of protocol primary associated types and concrete types' generic parameters. Both written in angle brackets, the latter get inferred as Copyable in extensions but the former do not:

The default conformance in a protocol extension applies only to Self , and not the associated types of Self .

In thinking this through, it occurs to me that—at least for the specific cases of Pair in the proposal and of Optional as you give above—the much more limited rule proposed for protocols suffices. That is, given a standard library with:

enum Optional<Wrapped: ~Copyable>: ~Copyable { ... }
extension Optional: Copyable where Wrapped: Copyable { }

...for the end user, the following two inference rules are equivalent:

// Inference rule proposed for concrete types:
extension Optional /* where Wrapped: Copyable */ {
  // `Wrapped` is `Copyable` here
}

// Inference rule proposed for protocols, if applied to concrete types:
extension Optional /* where Self: Copyable */ {
  // `Wrapped` is `Copyable` here also
}

Obviously, this makes sense only for concrete types that have a conditional conformance to Copyable—but perhaps that's enough? In other words, can we achieve much or all of the behavior we want with a unified inference rule that's something like:

Extensions of conditionally copyable types or of protocols that suppress inheritance from Copyable infer the requirement where Self: Copyable unless explicitly suppressed.

3 Likes
20

Suppose you have a type with noncopyable generic parameters. The type itself may or may not be copyable. The type might be unconditionally noncopyable, so everything is suppressed:

struct G<T: ~Copyable, U: ~Copyable>: ~Copyable {}

The type might be unconditionally copyable (an example is UnsafePointer; even if T is noncopyable, the pointer can be copied):

struct G<T: ~Copyable, U: ~Copyable> /* : Copyable */ {}

The tricky case is conditional copyability. Because of the restrictions on conditional requirements of Copyable specifically, the conditional Copyable conformance requires that some subset of generic parameters is Copyable. With G, there are three possibilities, but two are mirror images of each other:

extension G: Copyable /* where T: Copyable, U: Copyable */ {}
extension G: Copyable where T: ~Copyable /* , U: Copyable */ {}

Now, if I instantiate a conditionally or unconditionally Copyable type with generic arguments that are all Copyable, the result must be Copyable as well. This means if I start with the "universe" of copyable types, and apply a bunch of type constructors that might be conditionally copyable, the result is always copyable; I cannot escape my "universe".

On the other hand, the first case of an unconditionally noncopyable type is different, because it allows you to construct a noncopyable type even if you didn't utter ~Copyable in your own source code.

So another way to change the extension rule is to say that there are no default conformance requirements in an extension of an unconditionally noncopyable type.

In an extension of a concrete type, Self is the generic type with the identity substitutions applied, so like G<T, U> here. Then, the requirement G<T, U>: Copyable simplifies into exactly the conditional requirements of Copyable.

I think this almost gets you all of the way, however the case of an unconditional Copyable conformance might not give you the right behavior. For example, given struct UnsafePointer<T: ~Copyable> /* : Copyable */ {}, an existing user who writes extension UnsafePointer would get extension UnsafePointer where Self: Copyable, which is UnsafePointer<T>: Copyable which simplifies away entirely (there's no implied requirement that T: Copyable because it doesn't need to be for UnsafePointer<T> to be copyable). So now your extension gives you a noncopyable T, which might not be what you wanted.

On the subject of redundant requirements in general:

So back in the day (Swift 5.7 or 5.8, I forget now) we used to diagnose redundant generic requirements. This was disabled by default at some point and then I removed the code recently. So writing T: Copyable anywhere is just silently ignored. The compiler happily accepts this for example even though the entire where clause is pointless:

func f<T: Sequence>(_: T) where T: Sequence, T.Iterator: IteratorProtocol {}
6 Likes
21

Good point—such a pared back rule would mean that suppressing Copyable on a generic parameter would be source-breaking in the case of an unconditionally copyable type like UnsafePointer.

Setting aside the source compatibility angle for a second, though, it's not entirely clear to me what the "right behavior" default should be for such types: There's an entire universe of operations, and indeed conformances to Copyable protocols (e.g.:, for pointers, striding) which make sense for unconditionally copyable types without implicitly constraining their generic parameters. A user who's not ready to learn about ownership could still be miffed if they wrote a perfectly good extension on UnsafePointer and found that it wasn't available for UnsafePointer<FileDescriptor>.

With this proposal subsetting out standard library adoption, the source compatibility angle here is a bit trickier to envision fully. I hesitate that we adopt an across-the-board rule that's more complicated, though, with source compatibility being the major driver of that decision if simpler alternatives would otherwise do and we're thinking that there will need to be compiler-internal features needed anyway to migrate the standard library.

A solution that's laser-targeted to source compatibility around existing type adoption may even be a flavor of the @retroactive usage suggested above: we could say that types can write UnsafePointer<T: @retroactive ~Copyable> so that—for those generic parameters only—extensions implicitly infer T: Copyable.

2 Likes