Support ~Copyable, ~Escapable in simple standard library protocols

Evolution Pitches
1

Hi Swift Evolution,

I shall doom myself by suggesting that this is a highly non-controversial proposal to pitch of a Friday afternoon, but something that hopefully will be able to unblock more adoption of non-Copyable and non-Escapable types.

Please direct spelling/grammar/technical errors to the PR, and disabuse me of my claims of ~Controversy here :slight_smile:

Support ~Copyable, ~Escapable in simple standard library protocols

Introduction

The following protocols will be marked as refining ~Copyable and ~Escapable:

  • Equatable, Comparable, and Hashable
  • CustomStringConvertible and CustomDebugStringConvertible
  • TextOutputStream and TextOutputStreamable
    LosslessStringConvertible will be marked as refining ~Copyable.

Additionally, Optional and Result will have their Equatable and Hashable conformances updated to support ~Copyable and ~Escapable elements.

Motivation

Several standard library protocols have simple requirements not involving associated types or elaborate generic implementations.

  • Equatable and Comparable tend only to need to borrow the left- and right-hand side
    of their one essential operator in order to equate or compare values;
  • Hashable only needs operations to fold hash values produced by a borrowed value
    into a Hasher;
  • The various String producing or consuming operations only
    need to borrow their operand to turn it into a string (as CustomStringConvertible.description
    does), or can create a non-Copyable values (as LosslessStringConvertible.init? could).

Use of these protocols is ubiquitous in Swift code, and this can be a major impediment to introducing non-Copyable types into a codebase. For example, it might be desirable to drop in a UniqueArray to replace an Array in some code where the copy-on-write checks are proving prohibitively expensive. But this cannot be done if that code is relying on that array type being Hashable.

Proposed solution

The following signatures in the standard library will be changed. None of these
changes effect the existing semantics, just allow them to be applied to non-Copyable/Escapable types.

protocol Equatable: ~Copyable, ~Escapable {
  static func == (lhs: borrowing Self, rhs: borrowing Self) -> Bool
}

extension Equatable where Self: ~Copyable & ~Escapable {
  static func != (lhs: borrowing Self, rhs: borrowing Self) -> Bool
}

protocol Comparable: Equatable, ~Copyable, ~Escapable {
  static func < (lhs: borrowing Self, rhs: borrowing Self) -> Bool
  static func <= (lhs: borrowing Self, rhs: borrowing Self) -> Bool
  static func >= (lhs: borrowing Self, rhs: borrowing Self) -> Bool
  static func > (lhs: borrowing Self, rhs: borrowing Self) -> Bool
}

extension Comparable where Self: ~Copyable & ~Escapable {
  static func <= (lhs: borrowing Self, rhs: borrowing Self) -> Bool
  static func >= (lhs: borrowing Self, rhs: borrowing Self) -> Bool
  static func > (lhs: borrowing Self, rhs: borrowing Self) -> Bool
}

protocol Hashable: Equatable & ~Copyable & ~Escapable { }

struct Hasher {
  mutating func combine<H: Hashable & ~Copyable & ~Escapable>(_ value: borrowing H)
}

extension Optional: Equatable where Wrapped: Equatable & ~Copyable & ~Escapable {
  public static func ==(lhs: borrowing Wrapped?, rhs: borrowing Wrapped?) -> Bool {
}

extension Optional: Hashable where Wrapped: Hashable & ~Copyable & ~Escapable {
  func hash(into hasher: inout Hasher)
  var hashValue: Int
}

protocol LosslessStringConvertible: CustomStringConvertible, ~Copyable { }

protocol TextOutputStream: ~Copyable, ~Escapable { }
protocol TextOutputStreamable: ~Copyable & ~Escapable { }

protocol CustomStringConvertible: ~Copyable, ~Escapable { }
protocol CustomDebugStringConvertible: ~Copyable, ~Escapable { }
protocol LosslessStringConvertible: CustomStringConvertible, ~Copyable { }

extension Result: Equatable where Success: Equatable & ~Copyable, Failure: Equatable {
  public static func ==(lhs: borrowing Self, rhs: borrowing Self) -> Bool
}

extension DefaultStringInterpolation
  mutating func appendInterpolation<T>(
    _ value: borrowing T
  ) where T: TextOutputStreamable & ~Copyable & ~Escapable { }

  mutating func appendInterpolation<T>(
    _ value: borrowing T
  ) where T: CustomStringConvertible & ~Copyable & ~Escapable { }
}

LosslessStringConvertible explicitly does not conform to ~Escapable since this
would require a lifetime for the created value, something that requires
further language features to express.

Note that underscored protocol requirements and methods in extensions are omitted
but will be updated as necessary.

Source compatibility

The design of ~Copyable and ~Escapable explicitly allows for source compatibility with
retroactive adoption, as extensions that do not restate these restrictions assume compatability.

So no clients of the standard library should need to alter their existing source except
with the goal of extending it to work with more types.

ABI compatibility

As with previous retroactive adoption, the existing pre-inverse-generics features used in the
standard library will be applied to preserve the same symbols as existed before.

The ABI implications of back deployment of these protocols is being investigated. It is hoped
this can be made to work – if not, these new features may need to be gated under a minimum
deployment target on ABI-stable platforms.

Future directions

There are many other protocols that would benefit from this approach that are not included.

Most of these are due to the presence of associated types (for example, RangeExpression.Bound), which is not yet supported. Once that is a supported feature, these protocols can be similarly refined with a follow-on proposal.

Codable and Decodable do not have associated types – but their implementation is heavily
generic, may not generalize to noncopyable types, and is out of scope for this proposal.

Now that these protocols support them, types such as InlineArray and Span could be made
to conditionally conform to Hashable, as Array does. There is some debate to be had about
the semantics of Equatable conformance for Span (though probably not for InlineArray),
and this should be the subject of a future proposal.

Allowing more types to be Custom*StringConvertible where Self: ~Copyable & ~Escapable, such as Optional, requires further work on the print infrastructure to be able to hand such types, so is out of scope for this proposal.

Alternatives considered

It can be argued that non-Copyable types have identity, and therefore should not be Equatable

in the current sense of the protocol. In particular:

Equality implies substitutability—any two instances that compare equally
can be used interchangeably in any code that depends on their values.

One might say that a noncopyable string type (one that does not require reference counting
or copy-on-write-checking overhead) should not be considered "substitutable" for another.

However, the definition also states:

Equality is Separate From Identity. The identity of a class instance is not part of an
instance's value.

Authors of non-Copyable types will need to decide for themselves whether their type should
be Equatable and what it means. The standard library should allow it to be possible, though.

43 Likes
3

I need to do a more in-depth reading of this but from a high level this looks really good and will be a big help in adopting ~Copyable and ~Escapable.

The one protocol that stands out as kind of odd is LosslessStringConvertible. Conceptually, it seems like any type that's losslessly convertible to and from String is copyable in practice:

struct S: ~Copyable, LosslessStringConvertible { ... }

let s = S(...)
let s2 = S(s.description)!  // look ma, a copy!

(I'm imagining some jokester defining protocol VerySlowlyCopyable: LosslessStringConvertible & ~Copyable {}...)

From a purely technical standpoint I can't think of a reason not to provide that suppressed conformance, but have you run into any real-world scenarios where a non-copyable LosslessStringConvertible type has been useful in practice? I'd love to see a couple motivating examples for that one.

1 Like
4

I have yet to run into any real-world scenario where a LosslessStringConvertible type has been useful in practice, so uses of it with non-Copyable types is a subset of that. I mostly did it for completeness sake.

5

Yes! Finally! We've needed this for a while, and this is too obvious to really need much feedback. Now we just need non-copyable parameter packs, and we'll be golden!

6

These generalizations make sense. I suspect that, in the case of the TextOutput*/*StringConvertible suite of protocols, their usefulness in ~Copyable and/or ~Escapable contexts will still be limited unless we also generalize print, debugPrint, default string interpolation, etc. to accept some ~Copyable & ~Escapable type(s) as well.

7 Likes
7

Is Escapable a supported feature yet? It's using an attribute to hide from public documentation.

EDIT: see swiftlang/swift#85620 by @kavon.

2 Likes
8

+1

9

Ship it.

11

You could have some resource that’s identifiable by a string (say, a file identified by name), and then have a noncopyable type wrap around that resource and also enforce uniqueness. Then you can round-trip it through String, but since only one instance will ever have a given description at a time, it’s still not copyable.

Or would that violate the contract behind LosslessStringConvertible?

12

Yeah this is similar to what I’d imagined in the thread about ~Copyable x Codable—in both cases you have an out because you can throw/return nil, though in the case of LosslessStringConvertible it does feel slightly more against the spirit of the protocol. I don’t know that that rises to the level where it’s worth totally banning at the library level, though!

1 Like
13

The wording of our semantic guarantee in LosslessStringConvertible includes:

If printing or otherwise obtaining the description were to be a consuming operation for a noncopyable type, then an initializer could plausibly re-create it? But afaict we're making description a borrowing operation here, so...

1 Like
14

The nit with that is that the String is copyable, so you’d still admit copies by implicitly copying the string and reconstituting it twice.

I think you need a non-copyable string to actually enforce the guarantee.

1 Like
15

Fair point. If it's possible to re-create an instance of a type from its string representation, then it must admit copies because String admits copies. Ergo, LosslessStringConvertible implies Copyable.

2 Likes
16

Yeah, but that's tantamount to saying that any non-copyable type that has an initializer that takes only copyable types should just be copyable.

1 Like
17

I don't think that follows for all types based on what Xiaodi is saying about LosslessStringConvertible. Consider this:

public struct FileDescriptor: ~Copyable {
  public init(byOpeningForReading path: String) {
    self.path = path
    self.fileDescriptor = open(...)
  }
  public var path: String
  private var fileDescriptor: Int
}

This initializer only takes copyable types as its arguments, and it's possible to get the value you passed in back out of it, but it's certainly not LosslessStringConvertible (it wouldn't be value-preserving), and it wouldn't be true to say that that type should be Copyable just because its initializer takes only Copyable types.

The key semantic guarantee provided by LosslessStringConvertible is

The description property of a conforming type must be a value-preserving representation of the original value.

So this is a semantic question specific to LosslessStringConvertible: if I can round-trip a value through String and preserve it perfectly, then it's effectively copyable and declaring that type ~Copyable would be a something of a contradiction.

5 Likes
18

I think the Copyable protocol’s only actual meaning is “implicitly copyable”. Types can still be copyable without being implicitly copyable.

For instance, it may not be a bad design in some circumstances to create a non-copyable string type where memory is simply allocated in init and deallocated in deinit with no reference counting. This string type can’t be copied implicitly by the language so should be ~Copyable, but it could still offer a copy() method to allocate copies manually. That kind of type is obviously a good fit for LosslessStringConvertible, assuming the protocol accepts non-Copyable types.

4 Likes
19

The documentation of the protocol disagrees:

A type whose values can be implicitly or explicitly copied.

What you're describing is something like the @noImplicitCopy attribute pitched here: Selective control of implicit copying behavior: `take`, `borrow`, and `copy` operators, `@noImplicitCopy`

Of course, we can split hairs and talk about a type that provides its own operation to clone its values despite being non-copyable, but that's not something we should call a "copy" because that term has specific meaning in Swift.

2 Likes
20

The type I’m describing is not a CoW type and is not using ARC, so it can’t be Copyable unless we introduce some kind of C++ -like copy constructor in Swift capable of allocating new storage on copy:

// sort of pseudo code
struct AllocatedString: ~Copyable {
   var storage: UnsafePointer<UInt8>
   init(utf8Bytes: Collection<UInt8>) { storage = malloc(utf8Bytes.count); ... }
   deinit { free(storage) }
}

Also, the point is to deterministically deallocate in deinit, meaning it can’t be Copyable within the current constraints on deinit for structs.

The main point is that while LosslessStringConvertible semantics sort of imply it can be used to make a copy (“it should be possible to re-create an instance from its string representation”), I don’t think it makes sense for the protocol to enforce the type is also copyable via Copyable. This restriction is not useful in any way and can prevent types with custom memory management from conforming to LosslessStringConvertible.

2 Likes
21

I think explicitly here means as in let x = y followed by continued use of y, rather than some philosophical concept of non-copyability that type authors are supposed to try and thwart.

9 Likes
22

Agree that this would be very useful since it would broaden what types are permitted to be ~Copyable or ~Escapable. Thank you, Ben.