[Draft] Rename Sequence.elementsEqual

[xwu (Xiaodi Wu)]

That ordering can be arbitrary, but it shouldn’t leak internal

representation such that the method used to create identical things affects
the outcome of generic methods because of differences in internal
representation.

It would be better to say that the iteration order is well-defined.
That will almost always mean documented, and usually predictable though
obviously e.g. RNGs and iterating in random order will not be predictable
by design.

That's actually more semantically constrained than what Swift calls a
`Collection` (which requires conforming types to be multi-pass and(?)
finite). By contrast, Swift's `SpongeBob` protocol explicitly permits
conforming single-pass, infinite, and/or unordered types.

I think you’re talking about Sequence here, I’ve lost track of your
nonsense by now. Yes, the current Swift protocol named Sequence allows
unordered types. You seem to keep asserting that but not actually
addressing my argument, which is *that allowing Sequences to be
unordered with the current API is undesired and actively harmful, and
should* *therefore** be changed*.

What is harmful about it?

After thinking about it, I think the harmful bit is that unordered
sequences are leaking internal representation (In your example, this is
causing people to be surprised when two sets with identical elements are
generating different sequences/orderings based on how they were created).
You are correct when you say that this problem is even true for for-in.

I would not say it is a problem. Rather, by definition, iteration
involves retrieving one element after another; if you're allowed to do that
with Set, then the elements of a Set are observably ordered in some way.
Since it's not an OrderedSet--i.e., order doesn't matter--then the only
sensible conclusion is that the order of elements obtained in a for...in
loop must be arbitrary. If you think this is harmful, then you must believe
that one should be prohibited from iterating over an instance of Set.
Otherwise, Set is inescapably a Sequence by the Swift definition of
Sequence. All extension methods on Sequence like drop(while:) are really
just conveniences for common things that you can do with iterated access;
to my mind, they're essentially just alternative ways of spelling various
for...in loops.

I think an argument could be made that you shouldn’t be able to iterate
over a set without first defining an ordering on it (even if that ordering
is somewhat arbitrary). Maybe we have something like a “Sequenc(e)able”
protocol which defines things which can be turned into a sequence when
combined with some sort of ordering. One possible ordering could be the
internal representation (At least in that case we are calling it out
specifically). If I had to say
“setA.arbitraryOrder.elementsEqual(setB.arbitraryOrder)” I would definitely
be less surprised when it returns false even though setA == setB.

Well, that's a totally different direction, then; you're arguing that
`Set` and `Dictionary` should not conform to `Sequence` altogether. That's
fine (it's also a direction that some of us explored off-list a while ago),
but at this point in Swift's evolution, realistically, it's not within the
realm of possible changes.

I am actually suggesting something slightly different. Basically, Set
and Dictionary’s conformance to Collection would have a different
implementation. They would conform to another protocol declaring that they
are unordered. That protocol would fill in part of the conformance to
sequence/collection using a default ordering, which is mostly arbitrary,
but guaranteed to produce the same ordering for the same list of elements
(even across collection types). This would be safer, but a tiny bit slower
than what we have now (We could also potentially develop a way for
collections like set to amortize the cost). For those who need to recover
speed, the new protocol would also define a property which quickly returns
a sequence/iterator using the internal ordering (I arbitrarily called it
.arbitraryOrder).

I believe it would not be source breaking.

That is indeed something slightly different.

In an ideal world--and my initial understanding of what you were
suggesting--Set and Dictionary would each have a member like `collection`,
which would expose the underlying data as a `SetCollection` or
`DictionaryCollection` that in turn would conform to `Collection`;
meanwhile, Set and Dictionary themselves would not offer methods such as
`prefix`, or indexing by subscript, which are not compatible with being
unordered. For those who want a particular ordering, there'd be something
like `collection(ordered areInIncreasingOrder: (T, T) -> Bool) ->
{Set|Dictionary}Collection`.

What you suggest here instead would be minimally source-breaking.
However, I'm unsure of where these guarantees provide benefit to justify
the performance cost. Certainly not for `first` or `dropFirst(_:)`, which
still yields an arbitrary result which doesn't make sense for something
_unordered_. We *could* have an underscored customization point named
something like `_customOrderingPass` that is only invoked from
`elementsEqual` or other such methods to pre-rearrange the internal
ordering of unordered collections in some deterministic way before
comparison. Is that what you have in mind?

Something like that. Whatever we do, there will be a tradeoff between
speed, correctness, and ergonomics.

My suggestion trades speed for correctness, and provides a way to recover
speed through additional typing (which is slightly less ergonomic).

You haven't convinced me that this is at all improved in "correctness." It
trades one arbitrary iteration order for another on a type that tries to
model an unordered collection.

We could do something like you suggest. I don’t think the method would
need to be underscored… the ordering pass could just be a method on the
protocol which defines it as unordered. Then we could provide a special
conformance for things where order really matters based on adherence to
that protocol. That might be an acceptable tradeoff. It would give us
speed at the cost of having the correct implementation being less ergonomic
and more error prone (you have to remember to check that it is unordered
and call the ordering method when it mattered).

I’d still be a bit worried that people would make incorrect generic
algorithms based on expecting an order from unordered things, but at least
it would be possible for them check and handle it correctly. I think I
could get behind that tradeoff/compromise, given where we are in the swift
process and Swift's obsession with speed (though I still slightly prefer
the safer default). At least the standard library would handle all the
things correctly, and that is what will affect the majority of programmers.

What is an example of such an "incorrect" generic algorithm that would be
made correct by such a scheme?

To start with, the one you gave as an example at the beginning of this
discussion: Two sets with identical elements which have different internal
storage and thus give different orderings as sequences. You yourself have
argued that the confusion around this is enough of a problem that we need
to make a source-breaking change (renaming it) to warn people that the
results of the ‘elementsEqual’ algorithm are undefined for sets and
dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a
problem with its name; the result of this operation is not at all undefined
for two sets but actually clearly defined: it returns true if two sets have
the same elements in the same iteration order, which is a publicly
observable behavior of sets (likewise dictionaries).

I don’t see why a non-source-breaking change is suddenly off-limits.

But more than that, any generic algorithm which is assuming that the
sequence is coming from an ordered source (i.e. many things using
first/last). Some uses of first are ok because the programmer actually
means ‘any’, but anywhere where they actually mean first/last may be
problematic.

Such as...?

Currently, there is no way to test for ordered-ness, so there is no way for

even a careful programmer to mitigate this problem. By adding a protocol
which states that something is unordered, we can either branch on it, or
create a separate version of an algorithm for things which conform.

It is clearly the case that Swift’s protocol hierarchy fits sets and
collections imperfectly; however, it is in the nature of modeling that
imperfections are present. The question is not whether it is possible to
incur performance, API surface area, and other trade-offs to make the model
more faithful, but rather whether this usefully solves any problem. What is
the problem being mitigated? As I write above, Swift’s Set and Dictionary
types meet the semantic requirements for Collection and moonlight as
ordered collections. What is a generic algorithm on an ordered collection
that is “not OK” for Set and Dictionary? (“elementsEqual”, as I’ve said,
is not such an example.)

···

On Mon, Oct 16, 2017 at 05:48 Jonathan Hull <jhull@gbis.com> wrote:

On Oct 15, 2017, at 9:58 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sun, Oct 15, 2017 at 8:51 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Oct 14, 2017, at 10:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

[xwu (Xiaodi Wu)]

What useful generic algorithms would this protocol support that are not
already possible?

It would allow expressing generic algorithms depending on an order.

-Thorsten

We can already express generic algorithms that depend on an order—any
generic algorithm that works on a Sequence works on something that is
ordered. A Swift Set has an undefined order right now, but a generic
algorithm working on any arbitrary Sequence likely doesn’t care about
*what* the order, just that an order exists. And a Swift Set does indeed
have an order. If you have a generic algorithm that only works on inputs
sorted in a particular manner, then you’ve likely either documented that or
added a “sortedBy” parameter. Otherwise, you probably just want to be able
to iterate through everything.

Let’s assume, though, that you wanted to write an algorithm that works
only on MeaningfullyOrdered inputs.

func extractInfo<T: MeaningfullyOrdered>(_ input: T) { }
extractInfo(someArray)

What stops the caller from simply wrapping the Set in an Array?

extractInfo(Array(someSet))

The Array constructed here is going to reflect the arbitrary ordering
provided by Set, but as far as the type system is concerned, the input is
an Array, which is certainly meaningfully-ordered. Have we gained anything
by requiring the caller to wrap the input in an array? We’ve made the call
site a bit more awkward, and we’ve lost a bit of performance. We certainly
need to be able to convert Sets in to Arrays; to eliminate that would be
massively source-breaking, and it’s not clear that allowing that conversion
is actively harmful, so it’s unlikely to change in Swift 5.

Should/could we just rename `Set` to `UniquedArray` or something like
that? This is starting to feel a bit like the access control debate.

So essentially convert Set to OrderedSet and not offer a theoretical
unordered Set? I think that would be acceptable (if we apply it to
dictionaries as well), BUT that doesn't address the more general case of
other, potentially custom unordered Sequences.

Think of it this way: all Swift stdlib types that model unordered sequences
are in fact ordered—no attempt is made to support the modeling of unordered
sequences as unordered Swift types, custom or built-in.

···

On Mon, Oct 16, 2017 at 14:55 Kevin Nattinger via swift-evolution < swift-evolution@swift.org> wrote:

On Oct 16, 2017, at 11:23 AM, David Sweeris via swift-evolution < > swift-evolution@swift.org> wrote:
On Oct 16, 2017, at 10:42, BJ Homer via swift-evolution < > swift-evolution@swift.org> wrote:
On Oct 16, 2017, at 8:20 AM, Thorsten Seitz via swift-evolution < > swift-evolution@swift.org> wrote:
Am 16.10.2017 um 07:19 schrieb Xiaodi Wu <xiaodi.wu@gmail.com>:

- Dave Sweeris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[trs (Thorsten Seitz)]

[…]
Sets, as a mathematical concept, have no intrinsic order. However, instances of `Set`, which can be iterated over, *do* have at least one order which can be said to be intrinsic in the following sense: as long as iteration is possible, no API design can prevent that order from being observed and associated with the instance. Put another way, if you can use an instance of a type in a for...in loop, intrinsic to that functionality is a publicly visible order.

You keep saying this, I keep saying it’s only a technical “order” that is an implementation detail

You keep saying it's an implementation detail, which it emphatically is *not*. It's a *public guarantee* by the protocol that you can use an instance of `Set` in a `for...in` loop, thereby exposing a publicly visible order. An implementation detail is something

Being able to use a Set in a for...in loop does *not* make it ordered! The purpose is is just being able to do something with each element. That a for...loop works sequentially is just a side effect. Just imagine we had parallelized for...in loops.

No, it is not at all a "side effect." A for...in loop is a way of controlling the flow of code which accesses elements in a sequence one after another, and the correct behavior of code inside the loop depends on these semantics. A "parallel for" loop would be a totally different thing; arbitrary for...in loops can't be automatically "upgraded" to a "parallel for" loop because they have different semantics, and types that support "parallel for" would likely have to conform to a protocol other than `Sequence`.

Exactly.

that could go away with an alternative implementation. By contrast, no implementation that permits an instance of `Set` being iterated over in a `for...in` loop can avoid exposing at least one publicly visible order, because it's not a matter of implementation. Put another way, by allowing iteration, a type necessarily exposes at least one publicly visible order and thereby models a sequence in at least one way.

Wrong. I could easily implement Set or another type to iterate in a random order over its elements, so each iteration would expose a different (meaningless) order.

No, you cannot implement `Set` in this way because it conforms to `Collection`, which guarantees a multi-pass sequence. Set *must expose the same order on every iteration*.

Set conforming to Collection is even worse than just conforming to Sequence as a quote from the documentation shows: "In addition to the operations that collections inherit from the Sequence protocol, you gain access to methods that depend on accessing an element at a specific position in a collection."
Clearly the elements of a Set do not have specific positions.

That’s not at all clear to me, could you elaborate? My understanding is that elements of Set definitely *do* have a position, and that’s why you can use an index on a set to retrieve the element. The same index on the same set retrieves the same element.

That might be true, by virtue of implementation. But the notion of a Set is unordered (OrderedSet would be different but with a meaningful and stable order) so the notion of positions within a Set do not make sense IMHO. The indices provided by the Collection interface do might even make sense as references for a set but the concept of distances between such indices is just weird for sets.

-Thorsten

···

Am 16.10.2017 um 18:19 schrieb Michael Ilseman <milseman@apple.com>:

On Oct 16, 2017, at 7:20 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
Am 16.10.2017 um 07:19 schrieb Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>>:

On Sun, Oct 15, 2017 at 11:57 PM, Thorsten Seitz <tseitz42@icloud.com <mailto:tseitz42@icloud.com>> wrote:
Am 16.10.2017 um 00:41 schrieb Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>:

On Sun, Oct 15, 2017 at 2:32 PM, Kevin Nattinger <swift@nattinger.net <mailto:swift@nattinger.net>> wrote:

Furthermore my argument still stands for types derived from Sequence, so sidestepping it by pointing out that the situation is even worse for the current implementation of Set won't work. Can you explain why a type derived from Sequence which will iterate over its elements in random order should have methods like dropFirst()?

This demonstrates that being able to be used in a for...in loop is about doing somthing with each element and not about element order.

Again, Sequence *already doesn't* require the order to be meaningful or even repeatable.

I know. That's why I am arguing that certain methods of Sequence make no sense. At least we have reached that common ground.
So why do you insist on Sequence having methods like dropFirst() if the notion of "first" does not make any sense?

Notice that, above, I said at least one order. A conforming type could
expose as many different orders as iterations, but that changes nothing. I can still map an instance of that type to get an array of elements, and that array will reveal some order which is the result of the inner workings of the type.

The latter is an additional property which should be expressed in an additional protocol like Kevin suggested.

What useful generic algorithms would this protocol support that are not already possible?

It would allow expressing generic algorithms depending on an order.

-Thorsten

-Thorsten

and can’t be relied upon for anything and so we shouldn’t provide methods that rely on it. I think this part of the discussion has reached the “agree to disagree” stage.

[…]
You’re a fan of the principal of least surprise. Tell me, which would be less surprising: Set.dropFirst() actually drops a random element, or Set doesn’t have a dropFirst()? And if you think dropFirst() removing an element at random is not surprising, please explain why.

I think Set.dropFirst removing the first element that I observe on iteration is the least surprising answer, because Swift tells me that the stdlib Set models a set but that it is also a sequence.

Your logic is backwards. You’re saying it’s “least surprising” because that’s how it’s currently implemented, not that it should be implemented that way because it’s least surprising.

No, I'm saying it's least surprising because any type that supports iterated access thereby exposes an order--not as an implementation detail but as a matter of public API--and in the absence of any other order, "first" must refer to that order so exposed.

[…]

And that’s PRECISELY why lexicographicallyEqual does not make sense to apply to unordered sets. There is no lexicographical comparison possible, so why do you keep insisting they should have a method that falsely claims to lexicographically compare them?

I agree! It doesn't make sense if no comparison is possible! But Swift tells me that a `Set` is a `Sequence`!

You keep making the circular argument that a Set should do things because it currently does them. If you want to argue against changing things, argue that things shouldn’t be changed, not that the current implementation is correct because it is the current implementation.

No, I'm arguing that `Set`, by supporting iterated access, is not wrong to conform to a protocol called `Sequence` because it does have an intrinsic and publicly observable order, which is not an accident of a particular implementation but is inherent to any type that supports iterated access. Now, whether it's the *best choice* to conform `Set` to `Sequence` and offer order-dependent functions is a matter of taste, but it isn't *wrong*.

[…]
You will always have to account for this possibility, because Swift's `Equatable` explicitly allows "special values" to be not equal to themselves. This is, at least in part, in order to accommodate the IEEE decree that NaN != NaN:

let x = [Double.nan]
x.elementsEqual(x) // false

NaN is special, one-shot and unordered sequences are not. Unless you think that all unordered and single-pass sequences should compare false for `elementsEqual`, this is irrelevant for any sequence that doesn’t contain NaN and well-defined (false) for any that does.

Certainly, not all single-pass sequences should compare false to themselves, but some should: for instance, an infinite single-pass stream of all 1's should compare true to itself, but an infinite single-pass stream of alternating 1's and 0's should compare false to itself. If you write generic code that calls `elementsEqual`, it is pervasively incorrect to test for identity by assuming that elementsEqual will return true on reflexive comparison. NaN is only one of many reasons why such code would blow up.

Changing this behavior is way beyond the scope of this thread (and has been the topic of hours (actually, weeks and weeks) of fun on this list previously).

Yes, I’ve seen the discussion on NaN and Comparable. It’s not the same discussion.

[…]

It would be better to say that the iteration order is well-defined. That will almost always mean documented, and usually predictable though obviously e.g. RNGs and iterating in random order will not be predictable by design.

Wouldn't it then suffice to document, say, that a set's iteration order is the insertion order?

Now this actually gave me pause. I guess it does match what I said, but I still take issue with the fact that two Sets could compare `==` but not `elementsEqual`. I think that defining iteration order as insertion order adds a piece of publicly documented state that goes beyond what a Set really is. What you describe is really an OrderedSet, just without the random-access manipulation.

a) There is no semantic requirement on the part of `==` to be equivalent to an elementwise comparison when it is defined on a collection; in fact, one could imagine that some exotic sequence might legitimately define equality in a way that has nothing to do with elementwise comparison. Put another way, `==` returning `true` does not imply `elementsEqual` returning `true`, and `elementsEqual` returning `true` does not imply `==` returning `true`. This applies equally to ordered collections and is independent of the question of how to model unordered collections.

b) You keep writing that some Foo is really some Bar, but those are really just names. What would be the harm if Swift's `Set` indeed simply models an ordered set without random-access manipulation?

I’ll have to mull this over to see if I can come up with a coherent and (more) specific requirement for what makes an Iterable a Sequence, since clearly “documented” isn’t enough. Perhaps something along the lines that any two Sequences that compare equal must iterate the same.

[…]
Apple documentation calls this one of the "order-dependent" methods. It is surely acceptable for a type that conforms to an order-dependent protocol to have methods that are order-dependent; they do, however, have to be clearly order-dependent to avoid confusion on unordered types.

I’m not clear on what you’re trying to get across here. It seems you’re saying unordered types shouldn’t have order-dependent methods, which is exactly what I’ve been arguing.

No, I'm saying, essentially, that there are no truly unordered types in Swift; `Set` and `Dictionary` lead double lives modeling unordered collections on the one hand and ordered collections on the other. The order-dependent methods can continue to exist; they just need to be clearly named so that users know when they're using an instance of `Set` in the manner of an unordered collection and when they're using an instance of `Set` in the manner of an ordered collection.

[...]
Then there are all the methods that imply a specific order of iteration. If the “sequence” is unordered, who knows what you’ll get? It is incredibly easy for an engineer to write a method that implicitly relies on a passed sequence being intrinsically ordered and another engineer to pass it an unordered “sequence.” The first engineer could neglect to document the dependency, or even not notice it; or the second engineer could just fail to read the documentation thoroughly enough. There is currently no way for the compiler to enforce passing only an object that is (or at least claims to be) intrinsically ordered.

It is also incredibly easy for such an engineer to use `for...in` instead to accomplish the same task, generic over ordered and unordered sequences whatever you name such distinguished protocols. I think your beef really still boils down to Set being compatible with `for...in` at all, as Jon acknowledges.

Not providing ordered functions for unordered collections makes the developers think about what they actually need. If any object will do, they can use for…in, .makeIterator().next(), or an `anyObject()` we provide as a convenience. If they actually need the first from some specific order, it’s a reminder they need to sort the objects first to get the right one.

The whole point of protocol hierarchies is to enable useful generic algorithms. Here, the purpose of having a protocol that unites both ordered and unordered collections is to permit the writing of generic algorithms that operate on both; a user would want the first item from an ordered collection or an arbitrary item (but the same one on multiple passes) from an unordered collection. The name for that is currently `first`. Brent demonstrated a trivial one-line example of such a use.

That’s particularly useful for functions that actually need an ordered sequence; using OrderedSequence instead of Iterable (just as placeholders) would be a firm reminder not to pass in an unordered collection.

[…]

As I said, you're welcome to tackle the protocol hierarchy, but I really doubt it's within the realm of realistic endpoints for Swift 5. I'm just trying to propose a narrowly targeted pragmatic solution to one specific limited harm that might be deliverable by the next point release. As a great man once said, Swift is a pragmatic language.

If you want a pragmatic solution, fix the bug in functionality, don’t try and rename the method to something obscure to cover it up.

What I'm arguing is that there *is no bug in functionality*, only a naming problem. It is true that the current protocol hierarchy would not be my preferred design, but that's a matter of taste in terms of, again, where to draw the line between too much modeling or not enough. But that's not tantamount to a *bug*.

If you want to limit the harm, override `equalObjects` on unordered sequences to use `==` (very strongly preferred), or always `false` (less desirable, but at least consistent)

[…]

The Swift stdlib deliberately eschews modeling everything in protocol hierarchies with the highest level of granularity. There's some fudging, deliberately, to find a happy medium between obtuse and approachable, between too many/too specialized and not enough. For example, I pushed for protocols such as `Field` and `Ring` at the top of the numeric hierarchy, which might allow complex number types to slot into the hierarchy more sensibly, for example. But we have a compromise protocol `Numeric` which doesn't quite have the same guarantees but is much more approachable. Notice that we also don't break down numeric protocols into `Addable`, `Subtractable`, etc.; we also have that fudge factor built into `Equatable`, as I mentioned.

Eh, one or two corner cases on a protocol is probably fine. What’s not fine is over half (Sequence) or almost all (Collection) the methods not being applicable. There is a very clear gap there. We don’t need to fix everything, but this is something that can and should be addressed.

This would be based on the premise that an instance of `Set` has no intrinsic order; I disagree for the reasons above.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[trs (Thorsten Seitz)]

IMHO `elementsEqual` provides a nice example for a method which only makes sense on something meaningfully ordered:
What is the use case for `elementsEqual` that works with a Set?

-Thorsten

···

Am 16.10.2017 um 19:42 schrieb BJ Homer <bjhomer@gmail.com>:

On Oct 16, 2017, at 8:20 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Am 16.10.2017 um 07:19 schrieb Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>>:

What useful generic algorithms would this protocol support that are not already possible?

It would allow expressing generic algorithms depending on an order.

-Thorsten

We can already express generic algorithms that depend on an order—any generic algorithm that works on a Sequence works on something that is ordered. A Swift Set has an undefined order right now, but a generic algorithm working on any arbitrary Sequence likely doesn’t care about what the order, just that an order exists. And a Swift Set does indeed have an order. If you have a generic algorithm that only works on inputs sorted in a particular manner, then you’ve likely either documented that or added a “sortedBy” parameter. Otherwise, you probably just want to be able to iterate through everything.

Let’s assume, though, that you wanted to write an algorithm that works only on MeaningfullyOrdered inputs.

func extractInfo<T: MeaningfullyOrdered>(_ input: T) { }
extractInfo(someArray)

What stops the caller from simply wrapping the Set in an Array?

extractInfo(Array(someSet))

The Array constructed here is going to reflect the arbitrary ordering provided by Set, but as far as the type system is concerned, the input is an Array, which is certainly meaningfully-ordered. Have we gained anything by requiring the caller to wrap the input in an array? We’ve made the call site a bit more awkward, and we’ve lost a bit of performance.
We certainly need to be able to convert Sets in to Arrays; to eliminate that would be massively source-breaking, and it’s not clear that allowing that conversion is actively harmful, so it’s unlikely to change in Swift 5.

So I agree with Xiaodi; I don’t see what we would gain by splitting the protocols, other than some conceptual purity. Some have expressed concern over the existence of someSet.first, but even if we removed it, it would still be available as Array(someSet).first. And we still haven't any examples of actual algorithms that would surprise the user by behaving incorrectly when given an arbitrarily-ordered sequence, so it’s hard to make the argument that this restriction is actively harmful.

I agree that isOrderedEqual(to:) is a better name for elementsEqual()

[David_Sweeris]

Oh, this is weird... I replied to a different thread an hour ago. Somehow that message hasn't posted yet, but this one from three days ago apparently got reposted? Weird.

···

On Oct 19, 2017, at 12:11 PM, David Sweeris via swift-evolution <swift-evolution@swift.org> wrote:

On Oct 16, 2017, at 10:42, BJ Homer via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Oct 16, 2017, at 8:20 AM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Am 16.10.2017 um 07:19 schrieb Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>>:

What useful generic algorithms would this protocol support that are not already possible?

It would allow expressing generic algorithms depending on an order.

-Thorsten

We can already express generic algorithms that depend on an order—any generic algorithm that works on a Sequence works on something that is ordered. A Swift Set has an undefined order right now, but a generic algorithm working on any arbitrary Sequence likely doesn’t care about what the order, just that an order exists. And a Swift Set does indeed have an order. If you have a generic algorithm that only works on inputs sorted in a particular manner, then you’ve likely either documented that or added a “sortedBy” parameter. Otherwise, you probably just want to be able to iterate through everything.

Let’s assume, though, that you wanted to write an algorithm that works only on MeaningfullyOrdered inputs.

func extractInfo<T: MeaningfullyOrdered>(_ input: T) { }
extractInfo(someArray)

What stops the caller from simply wrapping the Set in an Array?

extractInfo(Array(someSet))

The Array constructed here is going to reflect the arbitrary ordering provided by Set, but as far as the type system is concerned, the input is an Array, which is certainly meaningfully-ordered. Have we gained anything by requiring the caller to wrap the input in an array? We’ve made the call site a bit more awkward, and we’ve lost a bit of performance. We certainly need to be able to convert Sets in to Arrays; to eliminate that would be massively source-breaking, and it’s not clear that allowing that conversion is actively harmful, so it’s unlikely to change in Swift 5.

Should/could we just rename `Set` to `UniquedArray` or something like that? This is starting to feel a bit like the access control debate.

- Dave Sweeris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[David_Sweeris]

How is the iteration order of an unordered set or dictionary “publicly observable”? If either is implemented such that it can asynchronously optimize its storage (maybe by rebalancing a tree or merging two non-contiguous array segments or something), its iteration order could change without changing what values it contains. Seems like consecutive calls to “elementsEquals” (or whatever we’re calling it) should return the same answer, if we don’t add, remove, or mutate elements.

- Dave Sweeris

···

On Oct 16, 2017, at 07:20, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Mon, Oct 16, 2017 at 05:48 Jonathan Hull <jhull@gbis.com> wrote:

On Oct 15, 2017, at 9:58 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Sun, Oct 15, 2017 at 8:51 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Oct 14, 2017, at 10:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

That ordering can be arbitrary, but it shouldn’t leak internal representation such that the method used to create identical things affects the outcome of generic methods because of differences in internal representation.

It would be better to say that the iteration order is well-defined. That will almost always mean documented, and usually predictable though obviously e.g. RNGs and iterating in random order will not be predictable by design.

That's actually more semantically constrained than what Swift calls a `Collection` (which requires conforming types to be multi-pass and(?) finite). By contrast, Swift's `SpongeBob` protocol explicitly permits conforming single-pass, infinite, and/or unordered types.

I think you’re talking about Sequence here, I’ve lost track of your nonsense by now. Yes, the current Swift protocol named Sequence allows unordered types. You seem to keep asserting that but not actually addressing my argument, which is that allowing Sequences to be unordered with the current API is undesired and actively harmful, and should therefore be changed.

What is harmful about it?

After thinking about it, I think the harmful bit is that unordered sequences are leaking internal representation (In your example, this is causing people to be surprised when two sets with identical elements are generating different sequences/orderings based on how they were created). You are correct when you say that this problem is even true for for-in.

I would not say it is a problem. Rather, by definition, iteration involves retrieving one element after another; if you're allowed to do that with Set, then the elements of a Set are observably ordered in some way. Since it's not an OrderedSet--i.e., order doesn't matter--then the only sensible conclusion is that the order of elements obtained in a for...in loop must be arbitrary. If you think this is harmful, then you must believe that one should be prohibited from iterating over an instance of Set. Otherwise, Set is inescapably a Sequence by the Swift definition of Sequence. All extension methods on Sequence like drop(while:) are really just conveniences for common things that you can do with iterated access; to my mind, they're essentially just alternative ways of spelling various for...in loops.

I think an argument could be made that you shouldn’t be able to iterate over a set without first defining an ordering on it (even if that ordering is somewhat arbitrary). Maybe we have something like a “Sequenc(e)able” protocol which defines things which can be turned into a sequence when combined with some sort of ordering. One possible ordering could be the internal representation (At least in that case we are calling it out specifically). If I had to say “setA.arbitraryOrder.elementsEqual(setB.arbitraryOrder)” I would definitely be less surprised when it returns false even though setA == setB.

Well, that's a totally different direction, then; you're arguing that `Set` and `Dictionary` should not conform to `Sequence` altogether. That's fine (it's also a direction that some of us explored off-list a while ago), but at this point in Swift's evolution, realistically, it's not within the realm of possible changes.

I am actually suggesting something slightly different. Basically, Set and Dictionary’s conformance to Collection would have a different implementation. They would conform to another protocol declaring that they are unordered. That protocol would fill in part of the conformance to sequence/collection using a default ordering, which is mostly arbitrary, but guaranteed to produce the same ordering for the same list of elements (even across collection types). This would be safer, but a tiny bit slower than what we have now (We could also potentially develop a way for collections like set to amortize the cost). For those who need to recover speed, the new protocol would also define a property which quickly returns a sequence/iterator using the internal ordering (I arbitrarily called it .arbitraryOrder).

I believe it would not be source breaking.

That is indeed something slightly different.

In an ideal world--and my initial understanding of what you were suggesting--Set and Dictionary would each have a member like `collection`, which would expose the underlying data as a `SetCollection` or `DictionaryCollection` that in turn would conform to `Collection`; meanwhile, Set and Dictionary themselves would not offer methods such as `prefix`, or indexing by subscript, which are not compatible with being unordered. For those who want a particular ordering, there'd be something like `collection(ordered areInIncreasingOrder: (T, T) -> Bool) -> {Set|Dictionary}Collection`.

What you suggest here instead would be minimally source-breaking. However, I'm unsure of where these guarantees provide benefit to justify the performance cost. Certainly not for `first` or `dropFirst(_:)`, which still yields an arbitrary result which doesn't make sense for something _unordered_. We *could* have an underscored customization point named something like `_customOrderingPass` that is only invoked from `elementsEqual` or other such methods to pre-rearrange the internal ordering of unordered collections in some deterministic way before comparison. Is that what you have in mind?

Something like that. Whatever we do, there will be a tradeoff between speed, correctness, and ergonomics.

My suggestion trades speed for correctness, and provides a way to recover speed through additional typing (which is slightly less ergonomic).

You haven't convinced me that this is at all improved in "correctness." It trades one arbitrary iteration order for another on a type that tries to model an unordered collection.

We could do something like you suggest. I don’t think the method would need to be underscored… the ordering pass could just be a method on the protocol which defines it as unordered. Then we could provide a special conformance for things where order really matters based on adherence to that protocol. That might be an acceptable tradeoff. It would give us speed at the cost of having the correct implementation being less ergonomic and more error prone (you have to remember to check that it is unordered and call the ordering method when it mattered).

I’d still be a bit worried that people would make incorrect generic algorithms based on expecting an order from unordered things, but at least it would be possible for them check and handle it correctly. I think I could get behind that tradeoff/compromise, given where we are in the swift process and Swift's obsession with speed (though I still slightly prefer the safer default). At least the standard library would handle all the things correctly, and that is what will affect the majority of programmers.

What is an example of such an "incorrect" generic algorithm that would be made correct by such a scheme?

To start with, the one you gave as an example at the beginning of this discussion: Two sets with identical elements which have different internal storage and thus give different orderings as sequences. You yourself have argued that the confusion around this is enough of a problem that we need to make a source-breaking change (renaming it) to warn people that the results of the ‘elementsEqual’ algorithm are undefined for sets and dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a problem with its name; the result of this operation is not at all undefined for two sets but actually clearly defined: it returns true if two sets have the same elements in the same iteration order, which is a publicly observable behavior of sets (likewise dictionaries).

[Jon_Hull]

But that iteration order is undefined and could easily change due to changes in the private/internal structure of sets/dictionaries. Algorithms that rely on that “publicly observable behavior” (i.e. leaking of internals) will suddenly break. You keep claiming that this bug is a feature because it is the current behavior… but that is tautological reasoning.

Thanks,
Jon

···

On Oct 16, 2017, at 7:20 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

To start with, the one you gave as an example at the beginning of this discussion: Two sets with identical elements which have different internal storage and thus give different orderings as sequences. You yourself have argued that the confusion around this is enough of a problem that we need to make a source-breaking change (renaming it) to warn people that the results of the ‘elementsEqual’ algorithm are undefined for sets and dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a problem with its name; the result of this operation is not at all undefined for two sets but actually clearly defined: it returns true if two sets have the same elements in the same iteration order, which is a publicly observable behavior of sets (likewise dictionaries).

[Adam_Kemp1 (Adam Kemp)]

There may not be one, but here’s the problem:

1. It’s generally useful for Set to conform to a protocol that allows you to iterate over its elements.
2. It’s generally useful to be able to ask if two objects that you can iterate over are equal by comparing the elements in the order that they’re iterated over.

The argument being made is that these two protocols should be different, but I don’t think the proponents of that idea have fully thought through what that would mean in practice. Consider a function like map, which takes a Sequence and produces another Sequence. This function is useful for both ordered and unordered elements so it would have to be defined in terms of the looser (unordered) type, which means its output type would be unordered. Imagine starting with an ordered enumerable and calling map on it. What’s the result? An unordered enumerable (the return type of map would be unordered to match its input type). Now you can’t use any of the methods that require the stricter (ordered) protocol on the result, even though you haven’t actually lost the order. You would have to do something to fix up the type and make the compiler see it as ordered again. Maybe there’s an asOrdered method or something. Imagine having to sprinkle that throughout your code just to make things compile. Does that sound like a usable API?

let people:[Person] = // Ordered
let orderedNames:[String] = // Ordered
let names = people.map { $0.fullName } // Result is unordered
return names.elementsEqual(orderedNames) // compile error: names is unordered
// Maybe: return names.asOrdered().elementsEqual(orderedNames)?

Avoiding this mess would require overloading such that every function that supports either ordered or unordered would have to be written both ways, which would just be a different mess.

All of that would be to solve a problem that in practice doesn’t seem to really cause any problems. I’m not aware of any evidence to suggest that this type causes a significant number of bugs, and we have another language/runtime (C#/.Net) with a large number of active developers and code bases with the same design and the same lack of evidence of a problem.

It seems like we’re being asked to make the library significantly harder to work with in order to solve a set of bugs that, as far as I can tell, doesn’t really exist in practice. I think in order to even consider this we would need to see the evidence that there’s a real problem to solve, and see a solution that didn’t make the library significantly harder to use.

···

On Oct 16, 2017, at 12:35 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:

IMHO `elementsEqual` provides a nice example for a method which only makes sense on something meaningfully ordered:
What is the use case for `elementsEqual` that works with a Set?

[trs (Thorsten Seitz)]

That ordering can be arbitrary, but it shouldn’t leak internal representation such that the method used to create identical things affects the outcome of generic methods because of differences in internal representation.

It would be better to say that the iteration order is well-defined. That will almost always mean documented, and usually predictable though obviously e.g. RNGs and iterating in random order will not be predictable by design.

That's actually more semantically constrained than what Swift calls a `Collection` (which requires conforming types to be multi-pass and(?) finite). By contrast, Swift's `SpongeBob` protocol explicitly permits conforming single-pass, infinite, and/or unordered types.

I think you’re talking about Sequence here, I’ve lost track of your nonsense by now. Yes, the current Swift protocol named Sequence allows unordered types. You seem to keep asserting that but not actually addressing my argument, which is that allowing Sequences to be unordered with the current API is undesired and actively harmful, and should therefore be changed.

What is harmful about it?

After thinking about it, I think the harmful bit is that unordered sequences are leaking internal representation (In your example, this is causing people to be surprised when two sets with identical elements are generating different sequences/orderings based on how they were created). You are correct when you say that this problem is even true for for-in.

I would not say it is a problem. Rather, by definition, iteration involves retrieving one element after another; if you're allowed to do that with Set, then the elements of a Set are observably ordered in some way. Since it's not an OrderedSet--i.e., order doesn't matter--then the only sensible conclusion is that the order of elements obtained in a for...in loop must be arbitrary. If you think this is harmful, then you must believe that one should be prohibited from iterating over an instance of Set. Otherwise, Set is inescapably a Sequence by the Swift definition of Sequence. All extension methods on Sequence like drop(while:) are really just conveniences for common things that you can do with iterated access; to my mind, they're essentially just alternative ways of spelling various for...in loops.

I think an argument could be made that you shouldn’t be able to iterate over a set without first defining an ordering on it (even if that ordering is somewhat arbitrary). Maybe we have something like a “Sequenc(e)able” protocol which defines things which can be turned into a sequence when combined with some sort of ordering. One possible ordering could be the internal representation (At least in that case we are calling it out specifically). If I had to say “setA.arbitraryOrder.elementsEqual(setB.arbitraryOrder)” I would definitely be less surprised when it returns false even though setA == setB.

Well, that's a totally different direction, then; you're arguing that `Set` and `Dictionary` should not conform to `Sequence` altogether. That's fine (it's also a direction that some of us explored off-list a while ago), but at this point in Swift's evolution, realistically, it's not within the realm of possible changes.

I am actually suggesting something slightly different. Basically, Set and Dictionary’s conformance to Collection would have a different implementation. They would conform to another protocol declaring that they are unordered. That protocol would fill in part of the conformance to sequence/collection using a default ordering, which is mostly arbitrary, but guaranteed to produce the same ordering for the same list of elements (even across collection types). This would be safer, but a tiny bit slower than what we have now (We could also potentially develop a way for collections like set to amortize the cost). For those who need to recover speed, the new protocol would also define a property which quickly returns a sequence/iterator using the internal ordering (I arbitrarily called it .arbitraryOrder).

I believe it would not be source breaking.

That is indeed something slightly different.

In an ideal world--and my initial understanding of what you were suggesting--Set and Dictionary would each have a member like `collection`, which would expose the underlying data as a `SetCollection` or `DictionaryCollection` that in turn would conform to `Collection`; meanwhile, Set and Dictionary themselves would not offer methods such as `prefix`, or indexing by subscript, which are not compatible with being unordered. For those who want a particular ordering, there'd be something like `collection(ordered areInIncreasingOrder: (T, T) -> Bool) -> {Set|Dictionary}Collection`.

What you suggest here instead would be minimally source-breaking. However, I'm unsure of where these guarantees provide benefit to justify the performance cost. Certainly not for `first` or `dropFirst(_:)`, which still yields an arbitrary result which doesn't make sense for something _unordered_. We *could* have an underscored customization point named something like `_customOrderingPass` that is only invoked from `elementsEqual` or other such methods to pre-rearrange the internal ordering of unordered collections in some deterministic way before comparison. Is that what you have in mind?

Something like that. Whatever we do, there will be a tradeoff between speed, correctness, and ergonomics.

My suggestion trades speed for correctness, and provides a way to recover speed through additional typing (which is slightly less ergonomic).

You haven't convinced me that this is at all improved in "correctness." It trades one arbitrary iteration order for another on a type that tries to model an unordered collection.

We could do something like you suggest. I don’t think the method would need to be underscored… the ordering pass could just be a method on the protocol which defines it as unordered. Then we could provide a special conformance for things where order really matters based on adherence to that protocol. That might be an acceptable tradeoff. It would give us speed at the cost of having the correct implementation being less ergonomic and more error prone (you have to remember to check that it is unordered and call the ordering method when it mattered).

I’d still be a bit worried that people would make incorrect generic algorithms based on expecting an order from unordered things, but at least it would be possible for them check and handle it correctly. I think I could get behind that tradeoff/compromise, given where we are in the swift process and Swift's obsession with speed (though I still slightly prefer the safer default). At least the standard library would handle all the things correctly, and that is what will affect the majority of programmers.

What is an example of such an "incorrect" generic algorithm that would be made correct by such a scheme?

To start with, the one you gave as an example at the beginning of this discussion: Two sets with identical elements which have different internal storage and thus give different orderings as sequences. You yourself have argued that the confusion around this is enough of a problem that we need to make a source-breaking change (renaming it) to warn people that the results of the ‘elementsEqual’ algorithm are undefined for sets and dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a problem with its name; the result of this operation is not at all undefined for two sets but actually clearly defined: it returns true if two sets have the same elements in the same iteration order, which is a publicly observable behavior of sets (likewise dictionaries).

But it is a behavior which has absolutely no meaning at all because the order does not depend on the elements of the set but on the history of how the set has been reached its current state.
So why should I ever use this method on a set?
What is the use case?

I don’t see why a non-source-breaking change is suddenly off-limits.

But more than that, any generic algorithm which is assuming that the sequence is coming from an ordered source (i.e. many things using first/last). Some uses of first are ok because the programmer actually means ‘any’, but anywhere where they actually mean first/last may be problematic.

Such as...?

Currently, there is no way to test for ordered-ness, so there is no way for even a careful programmer to mitigate this problem. By adding a protocol which states that something is unordered, we can either branch on it, or create a separate version of an algorithm for things which conform.

It is clearly the case that Swift’s protocol hierarchy fits sets and collections imperfectly; however, it is in the nature of modeling that imperfections are present. The question is not whether it is possible to incur performance, API surface area, and other trade-offs to make the model more faithful, but rather whether this usefully solves any problem. What is the problem being mitigated? As I write above, Swift’s Set and Dictionary types meet the semantic requirements for Collection and moonlight as ordered collections. What is a generic algorithm on an ordered collection that is “not OK” for Set and Dictionary? (“elementsEqual”, as I’ve said, is not such an example.)

On the contrary, `elementsEqual` is exactly such an example, because it makes no sense to use it on a Set.

let s1 = Set([1,2,3,4,5,6])
let s2 = Set([6,5,4,3,2,1])

Both sets have different iteration orders. Comparing those sets with some other collection using `elementsEqual` will give no meaningful result because the order - and therefore the result of `elementsEqual` - is in effect random.

-Thorsten

···

Am 16.10.2017 um 16:20 schrieb Xiaodi Wu via swift-evolution <swift-evolution@swift.org>:
On Mon, Oct 16, 2017 at 05:48 Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Oct 15, 2017, at 9:58 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Sun, Oct 15, 2017 at 8:51 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Oct 14, 2017, at 10:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[trs (Thorsten Seitz)]

IMHO `elementsEqual` provides a nice example for a method which only makes sense on something meaningfully ordered:
What is the use case for `elementsEqual` that works with a Set?

There may not be one, but here’s the problem:

1. It’s generally useful for Set to conform to a protocol that allows you to iterate over its elements.

Absolutely. There is no dispute about that.

2. It’s generally useful to be able to ask if two objects that you can iterate over are equal by comparing the elements in the order that they’re iterated over.

Here I disagree. This operation only makes sense if the iteration order has underlying well defined semantics. Otherwise the result of that operation will be a random value, depending on the undefined order. I argue that this is never useful. That is why I am asking for a use case for that.

The argument being made is that these two protocols should be different, but I don’t think the proponents of that idea have fully thought through what that would mean in practice. Consider a function like map, which takes a Sequence and produces another Sequence. This function is useful for both ordered and unordered elements so it would have to be defined in terms of the looser (unordered) type, which means its output type would be unordered. Imagine starting with an ordered enumerable and calling map on it. What’s the result? An unordered enumerable (the return type of map would be unordered to match its input type).

Good point. There are several solutions for this, e.g. covariant redefinition of the result type to be ordered in an ordered subclass, or more sophisticated solutions which could even allow to chose the result type if required to be different from a default.
The latter would allow e.g. mapping over a Set to commonly result in a generic Iterable but in some cases it might also result in another Set. The same holds for mapping over an ordered collection like an Array. The result might commonly be an array but it might also be a Set or something else.
Swift currently lacks the necessary capabilities in the type system to achieve this, though, so we would have to stay with the current solution that `map` always returns an Array (at least for the moment) — which already is not very satisfactory in itself.

-Thorsten

···

Am 16.10.2017 um 22:29 schrieb Adam Kemp <adam_kemp@apple.com>:

On Oct 16, 2017, at 12:35 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Now you can’t use any of the methods that require the stricter (ordered) protocol on the result, even though you haven’t actually lost the order. You would have to do something to fix up the type and make the compiler see it as ordered again. Maybe there’s an asOrdered method or something. Imagine having to sprinkle that throughout your code just to make things compile. Does that sound like a usable API?

let people:[Person] = // Ordered
let orderedNames:[String] = // Ordered
let names = people.map { $0.fullName } // Result is unordered
return names.elementsEqual(orderedNames) // compile error: names is unordered
// Maybe: return names.asOrdered().elementsEqual(orderedNames)?

Avoiding this mess would require overloading such that every function that supports either ordered or unordered would have to be written both ways, which would just be a different mess.

All of that would be to solve a problem that in practice doesn’t seem to really cause any problems. I’m not aware of any evidence to suggest that this type causes a significant number of bugs, and we have another language/runtime (C#/.Net) with a large number of active developers and code bases with the same design and the same lack of evidence of a problem.

It seems like we’re being asked to make the library significantly harder to work with in order to solve a set of bugs that, as far as I can tell, doesn’t really exist in practice. I think in order to even consider this we would need to see the evidence that there’s a real problem to solve, and see a solution that didn’t make the library significantly harder to use.

[trs (Thorsten Seitz)]

The new language features (which are sorely needed anyway) are not necessary for the proposed solution of splitting the protocol. They are necessary to fix or improve the definition of map which is currently lacking as well because it always returns an Array.
Alternatively we can continue to live with that. This is a separate question.

-Thorsten

···

Am 17.10.2017 um 00:15 schrieb Adam Kemp <adam_kemp@apple.com>:

On Oct 16, 2017, at 3:08 PM, Thorsten Seitz <tseitz42@icloud.com> wrote:

2. It’s generally useful to be able to ask if two objects that you can iterate over are equal by comparing the elements in the order that they’re iterated over.

Here I disagree. This operation only makes sense if the iteration order has underlying well defined semantics. Otherwise the result of that operation will be a random value, depending on the undefined order. I argue that this is never useful. That is why I am asking for a use case for that.

I deliberately didn’t specify in either of these cases whether the thing was ordered or not. I’m not disputing that this only makes sense for ordered things. I’m merely pointing out that making that distinction is impractical for other reasons.

There are several solutions for this, e.g. covariant redefinition of the result type to be ordered in an ordered subclass, or more sophisticated solutions which could even allow to chose the result type if required to be different from a default.
The latter would allow e.g. mapping over a Set to commonly result in a generic Iterable but in some cases it might also result in another Set. The same holds for mapping over an ordered collection like an Array. The result might commonly be an array but it might also be a Set or something else.
Swift currently lacks the necessary capabilities in the type system to achieve this, though, so we would have to stay with the current solution that `map` always returns an Array (at least for the moment) — which already is not very satisfactory in itself.

Now you’re inventing new language features in order to support your proposed solution to a problem that I still have seen no evidence to support being a significant source of real bugs. And those new language features would still result in a library and language that is harder to use than before. This is not a convincing argument for making a change.

[knatt (Kevin Nattinger)]

IMHO `elementsEqual` provides a nice example for a method which only makes sense on something meaningfully ordered:
What is the use case for `elementsEqual` that works with a Set?

There may not be one, but here’s the problem:

1. It’s generally useful for Set to conform to a protocol that allows you to iterate over its elements.
2. It’s generally useful to be able to ask if two objects that you can iterate over are equal by comparing the elements in the order that they’re iterated over.

The argument being made is that these two protocols should be different, but I don’t think the proponents of that idea have fully thought through what that would mean in practice. Consider a function like map, which takes a Sequence and produces another Sequence. This function is useful for both ordered and unordered elements so it would have to be defined in terms of the looser (unordered) type, which means its output type would be unordered. Imagine starting with an ordered enumerable and calling map on it. What’s the result? An unordered enumerable (the return type of map would be unordered to match its input type). Now you can’t use any of the methods that require the stricter (ordered) protocol on the result, even though you haven’t actually lost the order. You would have to do something to fix up the type and make the compiler see it as ordered again. Maybe there’s an asOrdered method or something. Imagine having to sprinkle that throughout your code just to make things compile. Does that sound like a usable API?

let people:[Person] = // Ordered
let orderedNames:[String] = // Ordered
let names = people.map { $0.fullName } // Result is unordered
return names.elementsEqual(orderedNames) // compile error: names is unordered
// Maybe: return names.asOrdered().elementsEqual(orderedNames)?

Avoiding this mess would require overloading such that every function that supports either ordered or unordered would have to be written both ways, which would just be a different mess.

That is a great point. Once we allow covariant functions to satisfy protocol requirements and have generalized existentials and recursive protocol requirements, wouldn't we be able to update thusly:

protocol Unordered {
  func map<T>(…) -> Any<U: Unordered where U.Element == T>
}
protocol Ordered: Unordered {
  func map<T>(…) -> Any<O: Ordered where O.Element == T>
}

?

Or the following; it also requires generalized existentials and recursive protocol requirements but not covariance; though I can imagine people arguing against introducing more associated types:

protocol Unordered {
  associatedtype MapResultType: Unordered
  func map<T>(…) -> Any<M: MapResultType where M.Element == T>
}

This would even allow an ordered collection to map to an unordered one or vice-versa. Maybe a collection type that spits out an ordered list when mapping to Comparables.
(n.b. the more I think about it, the more I like this option. Where are our existentials and recursive requirements? :) )

In this case we wouldn't even need an Ordered override; using concrete classes directly it would just work, and a function that requires an ordered result could specify it as any other constraint just as you do today:

func foo<O: Ordered>(in seq: O) where O.Element == Int, O.MapResultType: Ordered {
    let orderedMapping = seq.map { $0 + 1 } // type = Any<O: Ordered where O.Element == Int>
    …
}

I'd be ok with `map` and `filter` keeping their array returns (and that's what's in my original email about this) until we get the language features needed to support these. And I think we'd want to revisit the definitions of much of what is currently Sequence once we get these language features anyway.

All of that would be to solve a problem that in practice doesn’t seem to really cause any problems. I’m not aware of any evidence to suggest that this type causes a significant number of bugs, and we have another language/runtime (C#/.Net) with a large number of active developers and code bases with the same design and the same lack of evidence of a problem.

As I've been saying all along, elementsEqual returning a functionally random result when an unordered type is involved is a problem. At the least, we should make an OrderedSequence (conforming to Sequence, but could even be otherwise empty), move at least elementsEqual and lexicographicallyPrecedes to functions or extensions on that, and conform the appropriate stdlib types. Which would obviously not include Set or Dictionary.

It seems like we’re being asked to make the library significantly harder to work with in order to solve a set of bugs that, as far as I can tell, doesn’t really exist in practice. I think in order to even consider this we would need to see the evidence that there’s a real problem to solve, and see a solution that didn’t make the library significantly harder to use.

My aim for any such changes is to make the library more correct to use with little to no breakage of correct code. Done right, I think `Sequence: Iterable` can be done with little to no breakage beyond the direct use of Set or Dictionary as ordered, which I and others view as incorrect to begin with.

···

On Oct 16, 2017, at 1:29 PM, Adam Kemp via swift-evolution <swift-evolution@swift.org> wrote:

On Oct 16, 2017, at 12:35 PM, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Michael_Ilseman]

That ordering can be arbitrary, but it shouldn’t leak internal representation such that the method used to create identical things affects the outcome of generic methods because of differences in internal representation.

It would be better to say that the iteration order is well-defined. That will almost always mean documented, and usually predictable though obviously e.g. RNGs and iterating in random order will not be predictable by design.

That's actually more semantically constrained than what Swift calls a `Collection` (which requires conforming types to be multi-pass and(?) finite). By contrast, Swift's `SpongeBob` protocol explicitly permits conforming single-pass, infinite, and/or unordered types.

I think you’re talking about Sequence here, I’ve lost track of your nonsense by now. Yes, the current Swift protocol named Sequence allows unordered types. You seem to keep asserting that but not actually addressing my argument, which is that allowing Sequences to be unordered with the current API is undesired and actively harmful, and should therefore be changed.

What is harmful about it?

After thinking about it, I think the harmful bit is that unordered sequences are leaking internal representation (In your example, this is causing people to be surprised when two sets with identical elements are generating different sequences/orderings based on how they were created). You are correct when you say that this problem is even true for for-in.

I would not say it is a problem. Rather, by definition, iteration involves retrieving one element after another; if you're allowed to do that with Set, then the elements of a Set are observably ordered in some way. Since it's not an OrderedSet--i.e., order doesn't matter--then the only sensible conclusion is that the order of elements obtained in a for...in loop must be arbitrary. If you think this is harmful, then you must believe that one should be prohibited from iterating over an instance of Set. Otherwise, Set is inescapably a Sequence by the Swift definition of Sequence. All extension methods on Sequence like drop(while:) are really just conveniences for common things that you can do with iterated access; to my mind, they're essentially just alternative ways of spelling various for...in loops.

I think an argument could be made that you shouldn’t be able to iterate over a set without first defining an ordering on it (even if that ordering is somewhat arbitrary). Maybe we have something like a “Sequenc(e)able” protocol which defines things which can be turned into a sequence when combined with some sort of ordering. One possible ordering could be the internal representation (At least in that case we are calling it out specifically). If I had to say “setA.arbitraryOrder.elementsEqual(setB.arbitraryOrder)” I would definitely be less surprised when it returns false even though setA == setB.

Well, that's a totally different direction, then; you're arguing that `Set` and `Dictionary` should not conform to `Sequence` altogether. That's fine (it's also a direction that some of us explored off-list a while ago), but at this point in Swift's evolution, realistically, it's not within the realm of possible changes.

I am actually suggesting something slightly different. Basically, Set and Dictionary’s conformance to Collection would have a different implementation. They would conform to another protocol declaring that they are unordered. That protocol would fill in part of the conformance to sequence/collection using a default ordering, which is mostly arbitrary, but guaranteed to produce the same ordering for the same list of elements (even across collection types). This would be safer, but a tiny bit slower than what we have now (We could also potentially develop a way for collections like set to amortize the cost). For those who need to recover speed, the new protocol would also define a property which quickly returns a sequence/iterator using the internal ordering (I arbitrarily called it .arbitraryOrder).

I believe it would not be source breaking.

That is indeed something slightly different.

In an ideal world--and my initial understanding of what you were suggesting--Set and Dictionary would each have a member like `collection`, which would expose the underlying data as a `SetCollection` or `DictionaryCollection` that in turn would conform to `Collection`; meanwhile, Set and Dictionary themselves would not offer methods such as `prefix`, or indexing by subscript, which are not compatible with being unordered. For those who want a particular ordering, there'd be something like `collection(ordered areInIncreasingOrder: (T, T) -> Bool) -> {Set|Dictionary}Collection`.

What you suggest here instead would be minimally source-breaking. However, I'm unsure of where these guarantees provide benefit to justify the performance cost. Certainly not for `first` or `dropFirst(_:)`, which still yields an arbitrary result which doesn't make sense for something _unordered_. We *could* have an underscored customization point named something like `_customOrderingPass` that is only invoked from `elementsEqual` or other such methods to pre-rearrange the internal ordering of unordered collections in some deterministic way before comparison. Is that what you have in mind?

Something like that. Whatever we do, there will be a tradeoff between speed, correctness, and ergonomics.

My suggestion trades speed for correctness, and provides a way to recover speed through additional typing (which is slightly less ergonomic).

You haven't convinced me that this is at all improved in "correctness." It trades one arbitrary iteration order for another on a type that tries to model an unordered collection.

We could do something like you suggest. I don’t think the method would need to be underscored… the ordering pass could just be a method on the protocol which defines it as unordered. Then we could provide a special conformance for things where order really matters based on adherence to that protocol. That might be an acceptable tradeoff. It would give us speed at the cost of having the correct implementation being less ergonomic and more error prone (you have to remember to check that it is unordered and call the ordering method when it mattered).

I’d still be a bit worried that people would make incorrect generic algorithms based on expecting an order from unordered things, but at least it would be possible for them check and handle it correctly. I think I could get behind that tradeoff/compromise, given where we are in the swift process and Swift's obsession with speed (though I still slightly prefer the safer default). At least the standard library would handle all the things correctly, and that is what will affect the majority of programmers.

What is an example of such an "incorrect" generic algorithm that would be made correct by such a scheme?

To start with, the one you gave as an example at the beginning of this discussion: Two sets with identical elements which have different internal storage and thus give different orderings as sequences. You yourself have argued that the confusion around this is enough of a problem that we need to make a source-breaking change (renaming it) to warn people that the results of the ‘elementsEqual’ algorithm are undefined for sets and dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a problem with its name; the result of this operation is not at all undefined for two sets but actually clearly defined: it returns true if two sets have the same elements in the same iteration order, which is a publicly observable behavior of sets (likewise dictionaries).

How is the iteration order of an unordered set or dictionary “publicly observable”? If either is implemented such that it can asynchronously optimize its storage (maybe by rebalancing a tree or merging two non-contiguous array segments or something), its iteration order could change without changing what values it contains. Seems like consecutive calls to “elementsEquals” (or whatever we’re calling it) should return the same answer, if we don’t add, remove, or mutate elements.

Sets are values. If you add, remove, or mutate any elements you have a different Set and thus a potentially different ordering of elements.

···

On Oct 16, 2017, at 8:46 AM, David Sweeris via swift-evolution <swift-evolution@swift.org> wrote:
On Oct 16, 2017, at 07:20, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Mon, Oct 16, 2017 at 05:48 Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Oct 15, 2017, at 9:58 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Sun, Oct 15, 2017 at 8:51 PM, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>> wrote:

On Oct 14, 2017, at 10:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

- Dave Sweeris
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Adam_Kemp1 (Adam Kemp)]

2. It’s generally useful to be able to ask if two objects that you can iterate over are equal by comparing the elements in the order that they’re iterated over.

Here I disagree. This operation only makes sense if the iteration order has underlying well defined semantics. Otherwise the result of that operation will be a random value, depending on the undefined order. I argue that this is never useful. That is why I am asking for a use case for that.

I deliberately didn’t specify in either of these cases whether the thing was ordered or not. I’m not disputing that this only makes sense for ordered things. I’m merely pointing out that making that distinction is impractical for other reasons.

There are several solutions for this, e.g. covariant redefinition of the result type to be ordered in an ordered subclass, or more sophisticated solutions which could even allow to chose the result type if required to be different from a default.
The latter would allow e.g. mapping over a Set to commonly result in a generic Iterable but in some cases it might also result in another Set. The same holds for mapping over an ordered collection like an Array. The result might commonly be an array but it might also be a Set or something else.
Swift currently lacks the necessary capabilities in the type system to achieve this, though, so we would have to stay with the current solution that `map` always returns an Array (at least for the moment) — which already is not very satisfactory in itself.

Now you’re inventing new language features in order to support your proposed solution to a problem that I still have seen no evidence to support being a significant source of real bugs. And those new language features would still result in a library and language that is harder to use than before. This is not a convincing argument for making a change.

···

On Oct 16, 2017, at 3:08 PM, Thorsten Seitz <tseitz42@icloud.com> wrote:

[xwu (Xiaodi Wu)]

To start with, the one you gave as an example at the beginning of this

discussion: Two sets with identical elements which have different internal
storage and thus give different orderings as sequences. You yourself have
argued that the confusion around this is enough of a problem that we need
to make a source-breaking change (renaming it) to warn people that the
results of the ‘elementsEqual’ algorithm are undefined for sets and
dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a
problem with its name; the result of this operation is not at all undefined
for two sets but actually clearly defined: it returns true if two sets have
the same elements in the same iteration order, which is a publicly
observable behavior of sets (likewise dictionaries).

But that iteration order is undefined and could easily change due to
changes in the private/internal structure of sets/dictionaries. Algorithms
that rely on that “publicly observable behavior” (i.e. leaking of
internals) will suddenly break.

And an algorithm in which such “sudden breakage” would occur is...?

You keep claiming that this bug is a feature because it is the current

···

On Mon, Oct 16, 2017 at 10:49 Jonathan Hull <jhull@gbis.com> wrote:

On Oct 16, 2017, at 7:20 AM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
behavior… but that is tautological reasoning.

Thanks,
Jon

[trs (Thorsten Seitz)]

Indeed.

-Thorsten

···

Am 16.10.2017 um 17:49 schrieb Jonathan Hull via swift-evolution <swift-evolution@swift.org>:

On Oct 16, 2017, at 7:20 AM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:

To start with, the one you gave as an example at the beginning of this discussion: Two sets with identical elements which have different internal storage and thus give different orderings as sequences. You yourself have argued that the confusion around this is enough of a problem that we need to make a source-breaking change (renaming it) to warn people that the results of the ‘elementsEqual’ algorithm are undefined for sets and dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a problem with its name; the result of this operation is not at all undefined for two sets but actually clearly defined: it returns true if two sets have the same elements in the same iteration order, which is a publicly observable behavior of sets (likewise dictionaries).

But that iteration order is undefined and could easily change due to changes in the private/internal structure of sets/dictionaries. Algorithms that rely on that “publicly observable behavior” (i.e. leaking of internals) will suddenly break. You keep claiming that this bug is a feature because it is the current behavior… but that is tautological reasoning.

[xwu (Xiaodi Wu)]

That ordering can be arbitrary, but it shouldn’t leak internal

representation such that the method used to create identical things affects
the outcome of generic methods because of differences in internal
representation.

It would be better to say that the iteration order is well-defined.
That will almost always mean documented, and usually predictable though
obviously e.g. RNGs and iterating in random order will not be predictable
by design.

That's actually more semantically constrained than what Swift calls
a `Collection` (which requires conforming types to be multi-pass and(?)
finite). By contrast, Swift's `SpongeBob` protocol explicitly permits
conforming single-pass, infinite, and/or unordered types.

I think you’re talking about Sequence here, I’ve lost track of your
nonsense by now. Yes, the current Swift protocol named Sequence allows
unordered types. You seem to keep asserting that but not actually
addressing my argument, which is *that allowing Sequences to be
unordered with the current API is undesired and actively harmful, and
should* *therefore** be changed*.

What is harmful about it?

After thinking about it, I think the harmful bit is that unordered
sequences are leaking internal representation (In your example, this is
causing people to be surprised when two sets with identical elements are
generating different sequences/orderings based on how they were created).
You are correct when you say that this problem is even true for for-in.

I would not say it is a problem. Rather, by definition, iteration
involves retrieving one element after another; if you're allowed to do that
with Set, then the elements of a Set are observably ordered in some way.
Since it's not an OrderedSet--i.e., order doesn't matter--then the only
sensible conclusion is that the order of elements obtained in a for...in
loop must be arbitrary. If you think this is harmful, then you must believe
that one should be prohibited from iterating over an instance of Set.
Otherwise, Set is inescapably a Sequence by the Swift definition of
Sequence. All extension methods on Sequence like drop(while:) are really
just conveniences for common things that you can do with iterated access;
to my mind, they're essentially just alternative ways of spelling various
for...in loops.

I think an argument could be made that you shouldn’t be able to
iterate over a set without first defining an ordering on it (even if that
ordering is somewhat arbitrary). Maybe we have something like a
“Sequenc(e)able” protocol which defines things which can be turned into a
sequence when combined with some sort of ordering. One possible ordering
could be the internal representation (At least in that case we are calling
it out specifically). If I had to say
“setA.arbitraryOrder.elementsEqual(setB.arbitraryOrder)” I would definitely
be less surprised when it returns false even though setA == setB.

Well, that's a totally different direction, then; you're arguing that
`Set` and `Dictionary` should not conform to `Sequence` altogether. That's
fine (it's also a direction that some of us explored off-list a while ago),
but at this point in Swift's evolution, realistically, it's not within the
realm of possible changes.

I am actually suggesting something slightly different. Basically, Set
and Dictionary’s conformance to Collection would have a different
implementation. They would conform to another protocol declaring that they
are unordered. That protocol would fill in part of the conformance to
sequence/collection using a default ordering, which is mostly arbitrary,
but guaranteed to produce the same ordering for the same list of elements
(even across collection types). This would be safer, but a tiny bit slower
than what we have now (We could also potentially develop a way for
collections like set to amortize the cost). For those who need to recover
speed, the new protocol would also define a property which quickly returns
a sequence/iterator using the internal ordering (I arbitrarily called it
.arbitraryOrder).

I believe it would not be source breaking.

That is indeed something slightly different.

In an ideal world--and my initial understanding of what you were
suggesting--Set and Dictionary would each have a member like `collection`,
which would expose the underlying data as a `SetCollection` or
`DictionaryCollection` that in turn would conform to `Collection`;
meanwhile, Set and Dictionary themselves would not offer methods such as
`prefix`, or indexing by subscript, which are not compatible with being
unordered. For those who want a particular ordering, there'd be something
like `collection(ordered areInIncreasingOrder: (T, T) -> Bool) ->
{Set|Dictionary}Collection`.

What you suggest here instead would be minimally source-breaking.
However, I'm unsure of where these guarantees provide benefit to justify
the performance cost. Certainly not for `first` or `dropFirst(_:)`, which
still yields an arbitrary result which doesn't make sense for something
_unordered_. We *could* have an underscored customization point named
something like `_customOrderingPass` that is only invoked from
`elementsEqual` or other such methods to pre-rearrange the internal
ordering of unordered collections in some deterministic way before
comparison. Is that what you have in mind?

Something like that. Whatever we do, there will be a tradeoff between
speed, correctness, and ergonomics.

My suggestion trades speed for correctness, and provides a way to
recover speed through additional typing (which is slightly less ergonomic).

You haven't convinced me that this is at all improved in "correctness."
It trades one arbitrary iteration order for another on a type that tries to
model an unordered collection.

We could do something like you suggest. I don’t think the method would
need to be underscored… the ordering pass could just be a method on the
protocol which defines it as unordered. Then we could provide a special
conformance for things where order really matters based on adherence to
that protocol. That might be an acceptable tradeoff. It would give us
speed at the cost of having the correct implementation being less ergonomic
and more error prone (you have to remember to check that it is unordered
and call the ordering method when it mattered).

I’d still be a bit worried that people would make incorrect generic
algorithms based on expecting an order from unordered things, but at least
it would be possible for them check and handle it correctly. I think I
could get behind that tradeoff/compromise, given where we are in the swift
process and Swift's obsession with speed (though I still slightly prefer
the safer default). At least the standard library would handle all the
things correctly, and that is what will affect the majority of programmers.

What is an example of such an "incorrect" generic algorithm that would be
made correct by such a scheme?

To start with, the one you gave as an example at the beginning of this
discussion: Two sets with identical elements which have different internal
storage and thus give different orderings as sequences. You yourself have
argued that the confusion around this is enough of a problem that we need
to make a source-breaking change (renaming it) to warn people that the
results of the ‘elementsEqual’ algorithm are undefined for sets and
dictionaries.

No, I am arguing that the confusion about ‘elementsEqual’ is foremost a
problem with its name; the result of this operation is not at all undefined
for two sets but actually clearly defined: it returns true if two sets have
the same elements in the same iteration order, which is a publicly
observable behavior of sets (likewise dictionaries).

But it is a behavior which has absolutely no meaning at all because the
order does not depend on the elements of the set but on the history of how
the set has been reached its current state.
So why should I ever use this method on a set?
What is the use case?

One example: you can use it to check an instance of Set<Float> to determine
if it has a NaN value. (The “obvious” way of doing it is not guaranteed to
work since NaN != NaN.)

I don’t see why a non-source-breaking change is suddenly off-limits.

But more than that, any generic algorithm which is assuming that the
sequence is coming from an ordered source (i.e. many things using
first/last). Some uses of first are ok because the programmer actually
means ‘any’, but anywhere where they actually mean first/last may be
problematic.

Such as...?

Currently, there is no way to test for ordered-ness, so there is no way

for even a careful programmer to mitigate this problem. By adding a
protocol which states that something is unordered, we can either branch on
it, or create a separate version of an algorithm for things which conform.

It is clearly the case that Swift’s protocol hierarchy fits sets and
collections imperfectly; however, it is in the nature of modeling that
imperfections are present. The question is not whether it is possible to
incur performance, API surface area, and other trade-offs to make the model
more faithful, but rather whether this usefully solves any problem. What is
the problem being mitigated? As I write above, Swift’s Set and Dictionary
types meet the semantic requirements for Collection and moonlight as
ordered collections. What is a generic algorithm on an ordered collection
that is “not OK” for Set and Dictionary? (“elementsEqual”, as I’ve said,
is not such an example.)

On the contrary, `elementsEqual` is exactly such an example, because it
makes no sense to use it on a Set.

let s1 = Set([1,2,3,4,5,6])
let s2 = Set([6,5,4,3,2,1])

Both sets have different iteration orders. Comparing those sets with some
other collection using `elementsEqual` will give no meaningful result
because the order - and therefore the result of `elementsEqual` - is in
effect random.

No, it is not such an example; it’s misleadingly named but works
correctly—that is, its behavior matches exactly the documented behavior,
which relies on only the semantic guarantees of Sequence, which Set
correctly fulfills.

···

On Mon, Oct 16, 2017 at 14:21 Thorsten Seitz <tseitz42@icloud.com> wrote:

Am 16.10.2017 um 16:20 schrieb Xiaodi Wu via swift-evolution < > swift-evolution@swift.org>:
On Mon, Oct 16, 2017 at 05:48 Jonathan Hull <jhull@gbis.com> wrote:

On Oct 15, 2017, at 9:58 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Sun, Oct 15, 2017 at 8:51 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Oct 14, 2017, at 10:48 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

-Thorsten

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Adam_Kemp1 (Adam Kemp)]

The language change would be necessary to avoid to consequences I described if you split the protocol. If there are other benefits to that feature then that’s not relevant to this discussion. The point is whether the proposed world where Sequence is split into two protocols would be practical to use, and your suggestion for how to make it work was to add a feature to the language.

I’m not sure how that hypothetical feature relates to the current return type of map being an array. C# doesn’t have your feature, and yet Select takes an IEnumerable<T> and returns IEnumerable<T>. I think the fact that Swift’s map returns an array instead of a Sequence has more to do with other library design decisions. As far as I can tell there’s no reason it couldn’t have been the same as .Net with the current type system.

Your proposal, on the other hand, would require a new type system feature not present in either Swift or C# just to make it tolerable, and I still haven’t seen anyone in this long thread lay out any evidence of a real problem being solved.

···

On Oct 16, 2017, at 11:06 PM, Thorsten Seitz <tseitz42@icloud.com> wrote:

The new language features (which are sorely needed anyway) are not necessary for the proposed solution of splitting the protocol. They are necessary to fix or improve the definition of map which is currently lacking as well because it always returns an Array.
Alternatively we can continue to live with that. This is a separate question.

[Adam_Kemp1 (Adam Kemp)]

Once we allow covariant functions to satisfy protocol requirements and have generalized existentials and recursive protocol requirements, wouldn't we be able to update thusly:

protocol Unordered {
  func map<T>(…) -> Any<U: Unordered where U.Element == T>
}
protocol Ordered: Unordered {
  func map<T>(…) -> Any<O: Ordered where O.Element == T>
}

Now apply that to every order-preserving function that takes a Sequence and returns another Sequence. You’ve moved the burden from users of API to implementers of API. It reminds me of the const/non-const split that C++ developers have to deal with, where a lot of functions end up being implemented twice so that you can have a const version and a non-const version (generally one just calls the other). It’s a pain. I don’t want that when working with Sequences. I don’t think it’s worth it. And FWIW, when I was programming in C# I wrote functions that took an IEnumerable<T> and return another IEnumerable<T> very often. It’s a powerful feature that would have been ruined by a change like this.

As I've been saying all along, elementsEqual returning a functionally random result when an unordered type is involved is a problem.

In theory. Where is the evidence that this leads to a significant number of real-world bugs? All you’ve done is describe a conceptual problem, but you haven’t connected the dots to real-world problems. Again, I can point to .Net, which has a much larger community of developers who have been working with the same “problem” since version 2.0 released in 2005. If this is a significant source of bugs then there should be evidence of that. Where is that evidence?

···

On Oct 17, 2017, at 10:00 AM, Kevin Nattinger <swift@nattinger.net> wrote:

[knatt (Kevin Nattinger)]

How is the iteration order of an unordered set or dictionary “publicly observable”? If either is implemented such that it can asynchronously optimize its storage (maybe by rebalancing a tree or merging two non-contiguous array segments or something), its iteration order could change without changing what values it contains. Seems like consecutive calls to “elementsEquals” (or whatever we’re calling it) should return the same answer, if we don’t add, remove, or mutate elements.

Sets are values. If you add, remove, or mutate any elements you have a different Set and thus a potentially different ordering of elements.

An implementation detail. We could make it a class* and AFAICT that wouldn't break any guarantees on Sequence; and the argument applies equally well to any other unordered Sequence, which has no value type or semantics constraint.

*: obviously we won't, I don't think anyone is advocating that.