SE-0283: Tuples conform to Equatable, Comparable, and Hashable

Evolution Proposal Reviews
23

They do, but one wonders if--had we thought of it--we could have made them @_alwaysEmitIntoClient immediately prior to Swift 5 ABI stability so as not to bake them into the ABI.

2 Likes
24

IIRC @aEIC came after ABI stability even – it was created to let us make changes despite the ABI.

1 Like
25

I don't think this will be a long-term problem. If we get a mechanism for general tuple conformance, then I think the special case ABI entry points would forward to the general mechanism, for backward compatibility, and other runtime code should move to the general mechanism.

26

There's a nonzero chance of behavior changes if we hide or remove the type-specific overloads, so we shouldn't assume we can without breaking some amount of source compatibility.

27

I love it! These three are the most useful protocol conformances for tuples, so we get 99% of the value of arbitrary protocol conformances for them. I'm very happy that someone implemented it. The fact that == and < already exist means that tuples practically beg for these conformances.

2 Likes
28

+1 on the utility of the addition.

There have been multiple occasions where I started to use a Tuple for something and then chose against it due to it not being Equatable!

The stated technical debt is my only concern. Should this should be added now or should it be held until it can be implemented natively? The Information needed to develop an educated opinion on this isn't really in the proposal, other than the Author's own opinion on the matter. That information being...

  1. What language features are required for generalized Tuple protocol adherence?
  2. What is the expected timeline of those features?
  3. Would that functionality be confined to that version and newer?

My best, semi-educated guess for #1 is the primary lynchpin is Variadic Generics.
For #2, Since On the road to Swift 6 mentions Variadic generics as a likely feature, and I'm (purely) guessing that will be in the fall 2021 release. I would expect Protocol-adhering extensions for Tuples could then be added either alongside Variadic Generis or the following release if additional features are needed.

So based on that, this would bring the availability of this feature foward by a year or 18 months. Swift has already existed for 6 years without this so IMO introducing non-removable technical debt seems unnecessary for only year of usage.

However, #3 potentially holds a lot of weight. Will a native Swift implementation of this be back-deployable to Swift 5.0 the way this version is? If not, that increases the availability-for-most-users to more like 3 or 4 years, which IMO makes the technical debt more palatable

Is there any more concrete info I'm not taking into consideration?

Also, I'm curious how both of these statements are true

these conformances are being implemented within the runtime which allows us to backward deploy these conformance

and

there is a level of runtime support needed to enable these conformances to work properly. Going forward this means we'll need to keep the entry points needed for these to work even after tuples are able to properly conform to protocols.

If the feature is deployable to any Swift 5+ client's runtime, why does it also need runtime support? (forgive me if I'm missing something obvious. Not a compiler/language engineer :innocent:)

1 Like
29

I was also confused by this -- the implementation adds the new runtime entry points to the back deployment static libs, so they will always be available.

The current proposal text states:

The conformances to Equatable , Comparable , and Hashable are all additive to the ABI. While at the time of writing this, there is no way to spell a new conformance to an existing type. However, these conformances are being implemented within the runtime which allows us to backward deploy these conformance to Swift 5.0, 5.1, and 5.2 clients.

I think this last sentence ought to be clarified:

However, code that deploys to ABI-stable OS versions whose Swift runtime predates this feature will include a copy of the conformance implementations in its binary executable. This allows code relying on this feature to back deploy to arbitrary OS versions.

2 Likes
30

I'll discuss more in detail about what is being added, and what is considered ABI (what needs to stick around) to hopefully provide some more light about the "technical debt" being added.

First, is the addition of a new protocol conformance kind for the compiler. This new conformance kind is used to inform the compiler that these conformances get lowered into special conformances provided by the runtime. When we see a tuple being used as an Equatable parameter or calling the == operator on tuples, we assign this new conformance kind to be used. Once variadic generics, parameterized extensions, and extending non-nominal types are all implemented, a normal tuple conformance to EHC (Equatable, Hashable, and Comparable) can be implemented within the standard library. Assuming once that's possible it'll require the newest Swift version, this new conformance kind should be able to go away because the compiler can check the type and protocol being used in a conformance and check against the deployment target. If we happen to see tuples with EHC against a lower Swift version when the regular conformances came out, we should be able to lower to the special back deployed conformances.

Second, are the runtime additions. There are two parts to adding a new conformance to a type in the runtime (in this case tuples with EHC). The first part are the conformance descriptors. There is a conformance descriptor for every single type conformance in Swift. This tells the runtime that said type conforms to said protocol along with other information like getting the protocol witness table. We have to add 3 of these to the Swift runtime to support the 3 protocol conformances. These conformance descriptors are ABI. Joe mentioned here: Special Case Protocol Conformance: Long-term Tradeoffs for Near-term Conformance - #21 by Joe_Groff that these conformance descriptors could become aliases to the real deal once it's available, meaning we can remove the structures from the runtime and alias them instead. The other part of the runtime additions are the protocol function implementations. It's exactly how it sounds, these are functions being added to the runtime that are being called every time you call one of EHC's functions or variable getters. E.g. (1, 2) == (2, 1) invokes the equal witness method in the runtime to determine if tuple1 is equal to tuple2. These functions are not ABI because they are a property of the conformance descriptors for the runtime to emplace into the witness table. So because these conformance descriptors can become aliases in the future, these function implementations should also be removed in favor of the real implementation. (@Joe_Groff did I butcher any of this?)

So, the only things we need to keep around forever are the conformance descriptors for older clients to use. Perhaps my use of entry point is confusing, but these are the parts of the ABI that we need to support, or potentially just make into an alias. Assuming variadic generics + parameterized extensions + non-nominal extensions require a specific Swift version, we'll also want to keep around some logic to be able to back deploy usage of tuple EHC conformance to older clients.

  1. Variadic generics, parameterized extensions, and extending non-nominals along with actually adding these conformances to the standard library. I do wonder if we also need @available on new conformances?
  2. I'm not sure anyone has a concrete timeline for all of these, but I know for parameterized extensions I want to release that sometime this year.
  3. I'm not sure how much runtime hackery is needed to back deploy all of those features, or if it's even possible, but it's best to assume they all will.

So the back deployment feature is being added to Swift compatibility libraries. There are only compatibility libraries for older runtimes that get linked given an older runtime compatibility version. There is not a compatibility library for the current Swift. If a client is not using an older Swift runtime, then we don't link against those libraries (or at least I don't think so, I'm still learning!) and instead use the Swift runtime's implementation of functions. We have to add this support to the Swift runtime along with the compatibility libraries to enable all parties to use this feature. Hopefully I didn't butcher explaining this :sweat_smile:

I think I did a poor job of explaining this last sentence. It was intended to say that because these implementations are in the runtime, we have a unique opportunity to utilize the compatibility libraries to back deploy this to older clients.

I know parts of this post were pretty technical, but I wanted to make it clear what is being added and what needs to be supported going forward. If you have more questions, please ask!

12 Likes
31

Thank you for the detailed explanation!

From the outside that seems like a pretty limited level of long-term debt. It also seems possible something like that could happen later in order to back-deploy the functionality. In any case it seems minimal enough to leave that decision to those having to maintain it. :smiling_face:

I look forward to hopefully using it this fall!

1 Like
32

Hmm. Doesn’t this pitch suggest that maybe there should be? If we know that a standard library feature is likely to be obsolete in the foreseeable future, maybe linking it into apps is better for long-term maintainability than keeping stubs around forever.

(However, this would of course require you to predict what will happen when an app is linked with the support library and the system uses the future approach. In this case I suspect it would be fine, but what do I know)

2 Likes
33

Any way to make this a fixed up to 6 arity like the equatable proposal?

Would a fixed arity help move it away from runtime feature and just available in newer versions of Swift?

Is there an ETA for the long term solution?

34

Background: Operators can be implemented as either top-level functions or type-level members. Tuples don't support type-level members, so a plan to transition == and < tuple support from top-level to type-level functions doesn't break user extensions.

But Hashable's members (past Equatable support) are expressed as instance-level members.

Core Query: What the heck happens if the tuple already has a hashValue (and/or hash or _rawHashValue) member?!

My naĂŻve guess is that the label still gets ignored as far as computing the hash is concerned. But what happens when user code (and/or a function taking a Hashable) accesses hashValue? Does it make a difference if the existing hashValue is an Int or not?

If the answer is:

...

...

Oh... [EXPLETIVE]! We forgot.

then we need to rescind this review.

2 Likes
35

This would end up being an overloading situation, where the tuple has both the members from its labels and from its protocol conformances.

36

I've checked the commit history of the Equatable implementation and this appears to be the case, but I just wanted to double check that this means that Void (i.e an empty tuple) will now be equatable?

1 Like
37

But the overloads cannot co-exist, since

myTuple.hashValue

must resolve to exactly one result type and one property accessing code. It seems that it must be an error, and the only fix (besides dumping the Hashable work) is to forbid "hashValue" as a user-named field.

We need the author to acknowledge this and add it (and possible resolutions) to both the source- and ABI-compatibility sections of the proposal.

1 Like
38

If you’re in a T: Hashable context, then it resolves to the protocol witness. If you’re working with a concrete type, it uses the type’s implementation.

We already have a similar situation with protocol extension methods:

protocol P { }
protocol Q { }

extension P { var x: Int { 1 } }
extension Q { var x: Int { 2 } }

struct S: P, Q { var x: Int { 3 } }

func fooS(_ s: S) { print(s.x) }
func fooP(_ p: P) { print(p.x) }
func fooQ(_ q: Q) { print(q.x) }
func fooPG<T: P>(_ p: T) { print(p.x) }
func fooQG<T: Q>(_ q: T) { print(q.x) }
//func fooPQ<T: P&Q>(_ t: T) { print(t.x) }     // Error: ambiguous

fooS(S())   // 3
fooP(S())   // 1
fooQ(S())   // 2
fooPG(S())  // 1
fooQG(S())  // 2
// fooPQ(S())  // error
2 Likes
39

Doesn't var x: Int also need to be a requirement of P and Q—not just a non-requirement added via extension—for the analogy with Hashable to hold? (And thus, all of these would return 3?)

40

It's worth noting that this proposal isn't changing anything to how tuple member lookup works. The example I used in the proposal, the .hashValue, was used to indicate that two different tuples had the same hashValue. The only way to access these members are in the T: Hashable context that @Nevin talks about. I imagine we'll want to change those rules once tuples can conform to anything and we can actually extend them.

41

myTuple.hashValue will resolve to the most specific thing named hashValue that's available in the type context of myTuple. There may be more than one candidate overload to pick from. You can end up with this situation with structs, enums, and classes today already, when extensions in different modules or on different protocols contain a property with the same name. We need better mechanisms for disambiguation when type-based disambiguation is insufficient, but this is not a new problem.

2 Likes
42

Yes, void is now Equatable, Comparable, and Hashable!

3 Likes