SE-0494: Add `isIdentical(to:)` Methods for Quick Comparisons to Concrete Types

Evolution Proposal Reviews
41

And that's ok and I'm on board with this process. I might be gatekeeping what goes in the proposal draft before the review has concluded, but I do not expect to have any ability to gatekeep the complete set of opinions and data LSG will use to make their own decision after review has concluded. I did explicitly indicate that I encouraged engineers to post here with suggestions for different names and that I would act in good faith to consider alternatives. Which I do think is consistent with what we expect proposal authors to bring to the table.

42

Span has a specific notion of identity because of its semantics as a memory-safe abstraction for accessing contiguous storage. Collections are not merely their underlying storage, and this review is what will establish the precedent for how such APIs concerning the underlying storage should be named on such higher-level abstractions. Personally, the discussion thus far has me more convinced rather than less that we need to cast about for an explicitly different name for the current operation under review.

2 Likes
43

Maybe it's worth thinking about future instances where we might want to expose implementation details of the standard library as APIs with not-fully-specified behavior, in order to allow for greater control over performance. For example, we could expose properties on String such as isKnownNFC or isKnownASCII. Thinking about conventions we would adopt for naming those APIs (such as including the word "known") could inform how we name this one.

2 Likes
44

So there's an implication here that seems to be saying "if one doesn't like the name isIdentical(to:) for this more general operation, then the prior art must have been wrong." I don't think that's true—I think it's reasonable to say that isIdentical(to:) is suitable for some specific types while also finding it less ideal for other types.

Span, for an example, is inherently an identity-driven data structure, so isIdentical(to:) feels natural there and doesn't really have other reasonable interpretations.

I'm also inclined to not hold swift-collections up as "prior art" because those APIs haven't gone through an official evolution process as the language and standard library do. I'm not saying we should ignore them completely, but if we were to decide that a different name was suitable, it's straightforward for that package to deprecate the old API and add new ones.

Back to this proposal, it's the String family of types where I struggle, because I think isIdentical(to:) can be easily misinterpreted there. Even though String is a collection, it's a very specific kind of collection that is more than just the sum of its parts. Someone might reach for it thinking it's testing "are these strings identical w.r.t. their code points" (i.e., not checking canonical equivalence).

@xwu is absolutely right here, and given that this operation is meant to be a low-level performance enhancement rather than a "natural" operation on values, I'm not sure we need or want a pithy name here. I think we should strive for a more descriptive, technical name that steers the average user away from it while simultaneously drawing to it the performance-minded folks who need it and making it self-documenting at the call site.

19 Likes
45

From a potential (new) user’s perspective, a isIdentical(to:) popping up in code completion after typing a dot (after variables of common types) has a lot of potential to be misunderstood as the underlying method behind == operator sugar and used as so, while isKnownIdentical(to:) itself sounds nuanced enough to trigger a Google search of said method.

Importantly, if it is actually misused, depends on the program it might actually work in some cases, but randomly fail in other cases, which I can only imagine involve a huge effort being made to debug such bugs. I think it's definitely good idea to add an extra word in the method name as a reminder (or trigger of curiosity).

Regarding the prior arts argument, I think it's already well discussed both here and in original pitch thread. Acknowledging that we are not adding isIdentical in the form of a protocol requirement but merely on concrete types, I'll add that in none of the prior arts, isIdentical is introduced as a somewhat "universal" method. They are all added in their very specific context (of Span, of Swift Collections package, or of underscored semi-private usage) and the method itself is never the focus of corresponding proposals, while the fact that this proposal references "isIdentical(to:) methods" in title is suggesting that it is somewhat unified, and is the focus of it. As much as I hope that we have every little detail of proposals fully reviewed with no mistakes, it certainly is not the way that we human operates. To me it's very good to have a pitch and proposal on this (previously somewhat neglected) topic, and personally I would suggest not wasting this chance to examine what we did.

7 Likes
46

I’m not sure this method should even have the word identical in it, which a term that is very much synonymous to equal.

isKnownIdentical(to:) doesn’t really mean anything different than isIdentical(to:). “These two are identical” vs “I know these two are identical,” is virtually saying the same thing.

47

My general advice for engineers would be to discourage a "strong" mental coupling between a representation being identical and a representation having identity.

The SE-0390 guide to noncopyable types tells us that:

All currently existing types in Swift are copyable, meaning it is possible to create multiple identical, interchangeable representations of any value of the type. However, copyable structs and enums are not a great model for unique resources. Classes by contrast can represent a unique resource, since an object has a unique identity once initialized, and only references to that unique object get copied.

We do not qualify the ability to produce a copy on a type being a class. We want to copy value types! This includes "hybrid" value types like a copy-on-write structure that "wraps" an identity reference and "pure" value types like Int and Bool that have no identity to begin with.

The established definition of Copyable means that value types like Int can produce identical representations without having any ability to produce a legit identity. This is totally supported.

All of the concrete types proposed here are Copyable. Which means all of the concrete types proposed here can produce identical representations. That ability is independent of whether or not all of these concrete types have any one "true" identity behind them.

Another clue was previously discussed:

Our library maintainers explicitly indicated that a representation that is distinguishable — which can also be called "not identical" — can be different not only by identity. The some other means is explicitly indicated as an available option to make this representation distinguishable.

48

If this is the kind of connection you want to discourage, choosing a name that doesn't share the same root would perhaps achieve that better than offering the discouragement as a rule-of-thumb after the fact?

3 Likes
49

My general feeling about isKnownASCII is this returns true to indicate "we know this is ASCII". A return value of false indicates "we don't know this is ASCII. A return value of false also indicates this *could" be ASCII… but we're not going to pay the potential O(n) performance penalty to tell you for sure.

This is not exactly what is being proposed here. We are proposing isIdentical(to:) to return true when we know that other is identical to self and false when we know that other is not identical to self. This is a stronger statement than returning false to indicate "we don't know". It's a binary decision on the part of our library maintainers: it either is or it isn't.

Migrating to isKnown for this operation in the interest of making the API look more "scary" is not the right choice IMO because we are mis-communicating what this return value really means. The precedent then seems to be that isKnown is for making APIs look scary instead of communicating that a false value indicates to the caller we don't know.

If engineers really wanted an isKnown API my advice would be to start with the definition of Equatable:

Equality implies substitutability. We formally define an equatable representation to be substitutable. We do not formally and explicitly define any substitutable to also be equatable… but I believe this is easy to derive:

For any values a, b, and c:

  • Assume a == b is true.
  • Assume b.isKnownSubstitutable(for: c) is true.
  • Because substitutability means that b and c "can be used interchangeably in any code that depends on their values" we can substitute c for b in our == operator that tests for equality on those values.
  • a == c is true.

Therefore we can derive that all representations that are interchangeable-substitutable are also equatable.

We already define the semantic guarantees that returning true from isIdentical(to:) implies that other must be equal to self. It would be easy to imply that other must be substitutable to self. It's the same semantic guarantee and also covers us for situations when self does not adopt Equatable.

The alternative then to isIdentical(to:) is isKnownSubstitutable(for:). We return true to indicate other is substitutable for — and therefore equal to — self and return false to indicate "we don't know if other is substitutable for self. We could also choose isKnownInterchangeable(for:)… but the definition of Equatable draws a more clear link to Substitutable as the term of art we would be looking for.

One drawback here is we are now making a weaker guarantee. Returning true to indicate other is identical to self is a stronger statement that returning true to indicate we know other is substitutable for self. We're dropping potentially useful and impactful information on the floor with that. This is also inconsistent with all our prior art that indicates isIdentical(to:) should be the correct name for this operation.

I don't hate isKnownSubstitutable(for:). I do think it is superior to isKnownIdentical(to:). But I still believe isIdentical(to:) is the best choice we have so far. But I'll throw this out there and see what the community and LSG think about that.

50

So that's a fair point… are you implying then the proposal is modified so that some concrete types ship a different name? Or are you implying that if we choose a different name for one concrete type we then want all to change to that new name?

So I think it's totally fair to point out that swift-collections does not ship on the same evolution process as standard library. But I also think it's fair to then point out that the project lead charted swift-collections with one goal being "to help incubate new functionality for the Swift Standard Library."[1]

That does not have to mean that all functionality from Collections must get some kind of "fast path" approval process from LSG. LSG should be free to use their best judgement when migrating functionality from Collections up to Standard Library. But if we are weighting and indexing here for "signal" from what we expect to happen when we ship isIdentical(to:) in the future — and we conjecture that library maintainers and product engineers would misuse or be confused by this API — then it also only seems fair to weigh and index here for signal from what actually happened when we shipped isIdentical(to:) outside of an official supported API in standard library.

So focusing specifically on String is interesting because we do already have isIdentical(to:) shipping as underscored:

It would be fair to point out that this API was landed outside the evolution process… but I could then also point out that this API could have also been renamed outside the evolution process. If at any point over the last two years this API has been used throughout Apple any one of those product engineers had thought this API should ship with a new name it would have been easy for them to land that change directly on main and deprecate the old name without any kind of major friction.[2] But that didn't happen… which is legit signal IMO. It's not a "bad" name. It's a good name.

  1. Introducing Swift Collections | Swift.org ↩︎

  2. The diff does point to a rdar… which I can't open. Nobody from Apple can confirm where this API is being used or how widespread it is being used. For all we know this is now being used in every system app that ships on macOS as a performance optimization. Or maybe not. We just don't know unfortunately — unless someone badges me in and shows me. ↩︎

2 Likes
51

For better or for worse… we already sort of "soft-overload" the concept of identity:

I didn't really start working with Swift until after SE-0261 already shipped. In hindsight… I believe engineers would probably wish we would have named this protocol something like IdentifierProviding. But it is what it is and we are where we are.

I could make the argument that plenty of engineers understand that Identifiable is not appropriate for comparing the identity of value types and LSG could make the argument that plenty of engineers still to this day do not understand that Identifiable is not appropriate for comparing the identity of value types. Is it confusing? Or is it not confusing? I guess it depends who you ask…

I don't know if there's one right answer here. There is precedent IMO that we can ship an API named "identical" and this can be decoupled from whether or not a value type has an identity. Whether or not the learned experience from shipping Identifiable is a Good Thing or a Bad Thing might not have an easy answer.

It's possible the overwhelming consensus across LSG is that our "original sin" comes from Identifiable. This is where we began to soft-overload the concept of identity outside of ObjectIdentifier and this is the road we don't want to continue going down. That would then make sense that LSG would push back on isIdentical(to:). This proposal could claim SE-0261 as a "prior art"… but LSG could then claim it was a flawed and imperfect prior art to begin with.

52

There are 17 hits of the word copy in the proposal, with requirement that copies should compare identical. If isIdentical(to:) seems to cause confusion, why nor consider isCopy(of:)?

1 Like
53

Probably then I would reference back to this earlier example:

And this:

Whether we go with isCopy(of:) or hasSameRepresentation(as:) we seem to be communicating something stronger than what we really need. We do not need — or want — to require that library maintainers return true only if other is a copy of self. It's good enough to return true if other is only "identical" to self: with the library maintainer being responsible for choosing to document in what capacity an identical representation could differ from a copy.

1 Like
54

Overall, I find this a valuable addition to the Standard Library, and a concept in general - I’m just not thrilled by the arbitrary nature of the functions defined on concrete types.

If we’re not ready to define a protocol yet, because we don’t have enough understanding of broader use cases beyond what’s been outlined from the Standard Library - I’m more in favor of freestanding functions that accept 2 instances to compare.

Swift has adopted the idiom that protocols are the methods for laying out the requirements and expectations for types to conform to, when conforming to the protocol. This proposal itself is laying out expectations for when library authors want to define their own versions of these functions.

But there’s nothing to help users understand why one type has these methods, and another doesn’t, beyond just the SE proposal that they should memorize.

At least with a few free floating functions, it better implies that this is meant for the Standard Library foremost, and if we later define a protocol, they can be deprecated in favor of the protocol conformance implementations.

Discoverability, progressive disclosure, idiomatic…

isIdentitical(_ first: String,  second: String) → Bool

is_
   ^ autocomplete now gives me a list of all overloads for concrete types
1 Like
55

So, something like hasRepresentationKnownIdentical(toThatOf:)?

56

Doesn’t “representation” refer to the layout and contents of memory underlying a value? I don’t think that’s part of the definition being pitched here. See my SubSet example above.

1 Like
57

One problem here is what happens when our library maintainer needs private state to determine whether or not two representations are identical:

struct S {
  private var storage = Storage()
}

func isIdentical(_ lhs: S, _ rhs: S) -> Bool {
  lhs.storage === rhs.storage
  // error: 'storage' is inaccessible due to 'private' protection level
}

This isn't directly a problem today on the concrete types currently being proposed: they depend on internal state. But I don't see any compelling reason to restrict us from depending on private state at any point in the future.

58

I don't think your SubSet example would work as written, because equivalent Sets have different indices, but leaving that detail aside I think @John_McCall is right that it wouldn't be in the spirit of the proposed API here. In any case we are reviewing the APIs proposed for the specific concrete types here.

59

Sure… but we can also see why "representation" might not be the best term of art by considering this case on String:

If testing for "same representation" is really testing for "is other a copy of self" then this is a stronger requirement than what we need. A library maintainer could choose to compare two representations as identical without being a copy and this does not have to break the semantic guarantees of our API design. The current API design puts this flexibility in the hands of the library maintainer that has the most context on how their type is built and how their type could define what "identical" is. I do not believe we need to make this a stronger guarantee that reduces the flexibility of the API. This is the right answer to the wrong problem.

60

I don't understand the distinction here you are making between "same representation" and "representations as identical": these seem like synonymous terms and I'm not sure why one implies different semantics than the other.