Pitch: User-defined tuple conformances

Evolution Pitches
1

Tuple conformances

Introduction

Tuples cannot conform to protocols today, and this surfaces in the form of obvious limitations, such as not being able to use a tuple of Hashable values as a Dictionary key.

Motivation

The desire for tuples to conform to certain standard library protocols was previously addressed by the Motivation of SE-0283, which proposed built-in language support for Equatable, Comparable and Hashable tuples. Independently, the Swift concurrency effort added a language extension where a tuple of Sendable values is itself Sendable. We propose unifying all of these special-case behaviors with user-defined tuple conformances, which can now be expressed using parameter packs (SE-0393). Both SE-0283 and SE-0393 list tuple conformances as a Future direction.

Proposed solution

We propose introducing the parameterized extension syntax, described in the Generics Manifesto. This syntax will be allowed in one specific case to declare a tuple conformance in its most general form:

extension <each T> (repeat each T): P where repeat each T: P { ... }

We will also allow a generic type alias describing a tuple to be extended with a conditional conformance; we propose that the below Tuple type alias is added to the standard library to facilitate this:

protocol Shape {
  func draw()
}

typealias Tuple<each Element> = (repeat each Element)

extension Tuple: Shape where repeat each Element: Shape {
  func draw() {
    repeat (each self).draw()
  }
}

Recall that a requirement in a protocol is implemented by a witness in the concrete conforming type. In the above, we declare a tuple extension, and so the witness for draw() implements the protocol requirement draw() on tuples. The actual implementation calls draw() on each element, which is known to itself conform to Shape. Notice the use of each self within the repeat pattern in the body of draw().

Detailed design

Any unlabeled tuple can be obtained via type substitution from the "most general" unlabeled tuple type. If each T is some type parameter pack, this most general type is (repeat each T); that is, the tuple type formed from a pack expansion over the elements of each T.

Today, the extended type of an extension must be a nominal type, be it a struct, enum, class, or protocol. We propose allowing the most general tuple type to be extended; this is called a tuple extension. As extensions can declare protocol conformances, a tuple extension can then implement the protocol requirements for the most general tuple type. This is called a tuple conformance.

This means the type of self within a tuple extension is (repeat each T), where each T is a generic parameter of the extension declaring the conformance. As a consequence of SE-0399, a reference to each self within a pack expansion expression will expand over the elements of the tuple.

As with extensions of structs, enums, and classes, Self within a tuple extension refers to the type of self, that is, (repeat each T).

Once a tuple conformance to some protocol P has been declared, an arbitrary tuple type will satisfy a conformance requirement to P as long as the elements of the tuple satisfy the conditional requirements of the tuple conformance. We will see below that the conditional requirements must consist of exactly one requirement repeat each T: P. When a protocol requirement is called on a value of tuple type, a pack is formed from the elements of the tuple type; this becomes the generic argument for each T in the call to the protocol witness.

Orphan rule

For the most part, tuple conformances behave as if they were user-defined retroactive conformances on a standard library type. In particular, it would not be valid for two modules to define two distinct tuple conformances to the same protocol. For this reason, we prohibit tuple conformances to protocols outside of the defining module.

One-element tuple unwrapping

Under the rules laid out in the parameter packs proposal, one-element tuple types are unwrapped after substitution. This means that tuple conformances must be coherent with respect to this unwrapping.

This imposes some restrictions on the form that tuple conformances can take. We can understand all of the below restrictions in the form of a commutative diagram. The top row shows the most general tuple type, the corresponding tuple conformance, and the witness for some associated type A. Now, we apply a substitution to each object, replacing the type parameter pack each T with a pack containing a single concrete type, say X. We require that all paths through the diagram that begin and end at the same object produce the same result:

(repeat each T) ---> [(repeat each T): P] ---> (repeat each T).A
      |                        |                        |
      |                        |                        |
      v                        v                        v
      X -------------------> [X: P] -----------------> X.A

Concretely, these restrictions are as follows:

  • A tuple extension must declare conformance to exactly one protocol.

  • The conditional requirements of this conformance must be exactly repeat each T: P where each T is the type parameter pack of the extension, and P is the conformed protocol.

    That is, a tuple extension extension Tuple: P where repeat each T: Q would not make sense, because in the one-element case this decays to X: P where X: Q; a statement that is false in the general case when P and Q may be unrelated protocols.

  • An associated type requirement A of P must be witnessed by a type alias whose underlying type is exactly (repeat (each T).A); that is, a tuple type projecting A from each element.

That is, if X.A is Int, and Y.A is String, then we have no choice but to require that (X, Y).A is equal to (Int, String).

Note that as a consequence of all of these rules, the empty tuple () will conform to every protocol that has a tuple conformance.

Dynamic behavior

The above rules allow us to guarantee that a tuple conformance witness is never invoked with a one-element pack, with the call forwarding directly to the element conformance in that case. Thus, the runtime type of Self in a tuple conformance must always be a bona-fide tuple type, and not an unwrapped element.

If some function itself uses parameter packs to form a tuple value from a pack, calling protocol requirements on this value will either invoke the tuple conformance witness or witness for a single element, depending on the size of the pack.

Labeled tuples and variance

Tuple labels are not something that parameter packs can abstract over. However, the expression type system defines a subtype relationship between a labeled tuple and the corresponding unlabeled tuple.

By analogy with classes, if a conformance is declared on a non-final class C, and there is a subclass relationship where D inherits from C, then the conformance is also inherited by D.

For the substitution of D for C to be valid in the case of class inheritance, we require that Self is only used in covariant or contravariant position, not invariant. We must therefore impose the same restrictions on tuples as we currently have on non-final classes.

This allows the following:

  • Conforming to a protocol like Equatable, where Self appears in parameter position.
  • Conforming to a hypothetical Clonable protocol, with a func clone() -> Self requirement that returns Self.

On the other hand, this is prohibited:

  • Conforming to a protocol with a requirement having Self in invariant position, such as func f() -> G<Self>.

    In this case, it would not be fully sound to take a labeled tuple, and apply G<> to the corresponding unlabeled tuple type.

Scope of usage

Due to the subtle static and dynamic behaviors outlined above, we expect tuple conformances to remain an advanced feature. For many purposes, it is better to declare a special-purpose variadic generic struct via SE-0398, and conform that to a protocol, because this offers complete flexibility without any complications around coherence:

struct EggFactory<each Bird> {}

extension EggFactory: OmletMaker where repeat each Bird: Chicken {}

This pattern also allows the variadic type to define custom constructors and accessors to enforce invariants, and so on.

Tuples should only conform to protocols that have obvious "algebraic" implementation that generalizes to all combinations of element types in an inductive manner, such as the three standard library protocols discussed above.

For example, it would probably not be a good idea to conform tuples to IteratorProtocol, because there are at least two obvious implementations; either a zip, or a concatenation (in which case we would also need a requirement that all sequences have the same element type, something a tuple conformance cannot even express).

Source compatibility

As part of this proposal, we will implement tuple conformances for Equatable, Comparable and Hashable in the standard library. These will also replace the static overloads of the comparison operators and == on tuples. These changes should remain source compatible with existing code.

ABI compatibility

This is tentative and subject to change. None of this is a promise or commitment.

We hope that it will be possible to use both user-defined tuple conformances, and the tuple conformances to Equatable, Comparable and Hashable on older runtimes.

A tuple conformance to a protocol with associated types will probably require at least a Swift 5.9 runtime.

Dynamic casts, for example ((1, 2) as Any) as? any Equatable will possibly requires a new Swift runtime with support for tuple conformances.

Acknowledgments

I would like to thank Alejandro Alonso for the previous design and implementation of SE-0283 Tuples Conform to Equatable, Comparable, and Hashable.

42 Likes
2

I'm not sure I understand the need for the orphan rule. For example, I can retroactively conform Int to Identifiable today – what is stopping tuples from behaving the same? On a similar note, I was excited about tuple conformances allowing the use of tuples in Codable contexts. If manual conformance is not possible, could this proposal consider including Codable conformances for tuples?

2 Likes
3

Nothing, except that if both you and some module you import retroactively conform Int to Identifiable, bad behavior may result.

EDIT: There is another good reason, and that is resilient evolution of protocols, or rather the inability of tuple conformances to cope with them. Specifically, adding a new associated type with a default would break existing tuple conformances because the default does not satisfy the coherence condition. Restricting the protocol to appear in the same module prevents this possibility. I’ll update the pitch with details.

3 Likes
4

With the proposed generic extension syntax, can we also express more restricted tuple conformances, such as something like

extension <T, U> (T, U): PairProtocol { ... }

? In cases where neither single-element nor empty tuples can be the result of a substitution, we could loosen some of the coherence restrictions you laid out for fully general tuple conformances.

12 Likes
5

I'd prefer not to address this as part of the current proposal, but I don't think anything here rules out this possibility, except that you will still not be able to declare a fixed-arity conformance and an "inductive" one to the same protocol. I'll add it under future directions.

1 Like
6

I’m glad this is coming up, but I’d like to suggest that the parameterized extension syntax not be added in this proposal at all, requiring users to use the Tuple typealias instead. I think parameterized extensions are a big enough addition to the language that they deserve a proper wholesale review without pieces of them getting in before review.

That said, using the typealias depends on something which is not possible today: using the generic parameter names from a typealias. Today, if you extend a generic typealias, the type parameters are ignored completely; not only are they not in scope, but any constraints they impose are not applied to the extension. (If they were, they could be used for parameterized extensions.)

So either we do need parameterized extensions, or we need another language change to support extensions of generic typealiases, or both. Or we’re carving out a special case for tuples, which seems unfortunate.

17 Likes
7

That's a fair point. How do we feel about the asethetics of a standard Tuple typealias in the standard library?

Today, requirements of generic type aliases are applied to extensions if they have the simple form where the underlying type forwards the arguments exactly; in this case the type alias's where clause is a superset of the underlying type's.

I do hope that some day we can support extensions of arbitrary generic type aliases (with a language mode source break due to the unfortunate issue with names you described) and in fact the requisite support for this is exactly what's needed for parameterized extensions (minus the syntax).

The PR I have up so far does implement some of the machinery needed to create an extension's generic parameter list from a generic type alias rather than a nominal type, but it's only used for tuple extensions right now.

EDIT: I'll update Future directions to talk about generic type aliases and parameterized extensions.

5 Likes
8

Would you be able to extend tuples without making them conform to a protocol? For example:

extension <each T> (repeat each T) {
  public var count: Int {
    var count = 0
    for repeat each _ in self {
      count += 1
    }
    return count
  }
}

Or would you have to define a dummy protocol to do this? And would this work?

(1, 2).count // 2
(1, 3, 5).count // 3
5 Likes
9

We do not plan on allowing this as part of this pitch. The intent is that members of tuple extensions should only be visible to conformance checking. In your hypothetical count example, there would need to be an implementation of count on every type for the one-element case to work.

2 Likes
10

I'm so happy to see all of the work the Swift team has done with variadic generics to be able to reach this conclusion!

Are you proposing the Equatable, Hashable, and Comparable conformances for tuples in the standard library here as well, or would that be a followup? Besides that, I don't have much to say here besides I'm all in favor of finally allowing users to use tuples as first class citizens in the language!

Thank you @Slava_Pestov and the rest of the Swift team that helped make this happen!

8 Likes
11

Hopefully in this proposal as well. Once pack iteration lands I will put up a draft PR.

5 Likes
12

I have two questions about a potential use case where I often looked forward to tuple conformances:

  1. Will it be possible to provide static members? / Is Self == (repeat each T).Type?
  2. Will it be possible to provide tuple conformances for a protocol that inherits from a protocol outside of its defining module that itself provides tuple conformances (Sendable in my example)?
protocol ChangeToken: Sendable {
    static var beginning: Self { get }
}

extension <each T: ChangeToken> (repeat each T): ChangeToken {
    static var beginning: (repeat each T) {
        (repeat (each Self).beginning)
    }
}
13

Yes (Self is still unchanged in this case but the type of self is Self.Type).

Yep.

1 Like
14

I don’t see these conformances specified in the Detailed Design section, which is kind of strange.

Should we also implement Encodable and Decodable in this proposal? Should we reimplement Sendable? Are there any other tuple conformances we should consider adding?

2 Likes
15

I’ll include this in the proposal that will go out for review, but basically there is only one possible implementation of each.

Already done. The ASTContext synthesizes the extension though, so that we can continue to use older versions of the stdlib that don’t declare this conformance.

CustomStringConvertible perhaps?

2 Likes
16

I feel like that while this is still useful that makes it a lot less useful. Is there a future direction planned to actually allow you to add members to tuples like any other type?

17

We could potentially add Sequence and some of the collection ones if we can describe:
extension <T, each U> (repeat each U): Sequence where repeat each U == T

2 Likes
18

Personally, I think this pitch should stick to the tuple conformances already reviewed and accepted in SE-0283, not because we shouldn't add others, but because there are a lot of subtle implications of the interaction between tuple conformances and single-element tuple unwrapping that need to be carefully considered in this proposal, and I don't want the review of these semantics to be crowded out by all of the other interesting tuple conformances that we could add. We can always consider other tuple conformances in follow-on proposals.

12 Likes
19

The problem is again, one-element tuple unwrapping. Consider something like this:

extension Tuple {
  func foo() { print("Hello") }
}

func f<each T>(_ pack: repeat each T) {
  (repeat each pack).foo()
}

Without the name lookup restriction, (repeat each pack).foo() would be a static reference to foo() inside the extension. However if f() is called with a one-element pack, like f(123), then (repeat each pack) is actually just the Int value 123 and not a tuple.

While we could simply allow foo() to be called with a one-element pack, it creates the odd situation where 123.foo() is not valid, even though the indirect call via f(123) is allowed.

We could then say that members of tuple extensions are visible on all types, with non-tuple types just forming a one-element pack substitution for the call. But now namespace pollution becomes an issue, because you're just adding members to this single global namespace. It's just a can of worms that is not worth opening, in my opinion.

3 Likes
20

I don't find that to be any odder than the existing automatic reduction of single-element tuples to non-tuples. In fact it's the logical consequence of that design choice.

Though a more interesting question is whether (123).foo() should work. I think it should in order to be logically consistent, but I suspect it currently doesn't.

1 Like