[Review] SE-0065 A New Model for Collections and Indices

(On the other hand, it might be that I'm conceiving of the purpose of `limitedBy` differently from you—I think of it as a safety measure, but you may be thinking of it specifically as an automatic truncation mechanism.)

Hi Brent,

Could you explain what kind of safety do you have in mind? Swift will
guarantee memory safety even if you attempt to advance an index past
endIndex using the non-limiting overload.

One challenge that I've run into is that the `limitedBy` methods throw away one bit of information—namely, did I move as far as I requested or not? Example:

let j = c.index(10, stepsFrom: i, limitedBy: c.endIndex)

There's no way to interpolate the answer to that question efficiently in a non-random-access collection. If `j` is equal to `c.endIndex`, that could be because `c.endIndex` is ten steps after `i` *or* because the limit kicked in, and without checking `c.distance(from: i, to: j)` there's no way to know for sure.

If the `limitedBy` methods returned an optional index, we'd get all the information that the index-moving algorithm finds (let's hear it for the Law of Useful Return!). With that API, we could decide whether to use the returned index or not:

// Use the resulting index no matter what:
let i = c.index(10, stepsFrom: c.startIndex, limitedBy: c.endIndex) ?? c.endIndex
let prefix = c.prefix(upTo: i)

// Only use the result if it wasn't limited:
if let j = c.index(10, stepsFrom: i, limitedBy: c.endIndex) {
    let sub = c[i..<j] // sub.count == 10
} else {
    // not enough elements...
}

// "Safe" successor:
if let j = c.index(1, stepsFrom: i, limitedBy: c.endIndex) {
    // ...
}

Nate

···

On Apr 12, 2016, at 4:15 AM, Dmitri Gribenko via swift-evolution <swift-evolution@swift.org> wrote:
On Mon, Apr 11, 2016 at 9:56 PM, Brent Royal-Gordon via > swift-evolution <swift-evolution@swift.org> wrote:

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Huh, that clarifies something. How about the non-`ing` variants?

  collection.travel(5, from: i)
  collection.stride(5, from: i)
  collection.advance(i, by: 5)

Active verb phrases are reserved for methods with side-effects according
to the API guidelines.

Is there *any* way of using a verb to indicate an argument is being modified? If not, I think that's a serious omission in the API guidelines.

···

--
Brent Royal-Gordon
Architechies

...and having talked it over at lunch, now I remember why we rejected
it: there's no good way to make a nonmutating version.

  let x = c.incremented(i) // reads like an assertion about the past
  let y = c.incrementing(i) // reads like it has side-effects and returns c, or
                             // a new version of c

APIs where the receiver returns a modified version of an argument don't
lend themselves to verb forms.

···

on Wed Apr 13 2016, Dave Abrahams <swift-evolution@swift.org> wrote:

Reverse is the best opposite we have of advance, so it makes sense to
me.

Oh, I get it.

Or we could use retreat. =) There are other pairs of words that work
as well, like “increment/decrement”.

Yeah, unfortunately those carry an incorrect implication when the
indices are numbers, because, e.g. the collection might be offsetting
the number by 2 for each position. One could of course argue that using
numbers that way as indices was a bad design choice.

I'll have to think about that idea again. We considered and rejected it
for a reason, but it might not be a really strong one. Thanks for
bringing it up.

--
Dave

Updated proposal:

···

on Wed Apr 13 2016, Dave Abrahams <swift-evolution@swift.org> wrote:

on Mon Apr 11 2016, Nate Cook <swift-evolution@swift.org> wrote:

Proposal link:
https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md

    On Apr 11, 2016, at 2:59 PM, Dave Abrahams via swift-evolution >> <swift-evolution@swift.org> wrote:

    Thanks for your comments, Brent!

    on Sun Apr 10 2016, Brent Royal-Gordon >> <swift-evolution@swift.org> wrote:

The shift described in this proposal is extremely valuable and makes
implementing collections far more intuitive, as all the collection's logic lives
"inside" the collection itself. My only hesitation is with the naming of the
method that Brent also called out:

... snip ...

                func index(n: IndexDistance, stepsFrom i: Index) -> Index

        Oof, I am really not a fan of this name. `steps` is sort-of a label on
        the `n` parameter, but it's attached to `i`.

Oof indeed! This is a very unusual method in the standard library, since we're
calling on one instance to perform an action on another. My problems with the
naming are twofold:

(1) Collision with the index(of:) and index(where:) APIs
The existing methods are used for searching a collection, possibly finding a
matching index, possibly not. The new ones deterministically find an new index
at a prescribed distance, with important and slightly complicated preconditions.
These differences make the use and "flavor" of the two sets of methods distinct
enough that I think they should have different names.

(2) Arguments are reversed
I think the ideal API for this would be index.advanced(by: 5, in: c), but I
prefer keeping the index-moving implementation in the collection, not the index.
I would favor any naming for this method that puts the index before the
distance, keeping the overall shape of the advanced(by:) method. c.advance(i,
by: 4) would be my pick.

Dmitri, Max, and I just talked this over and we think you're right on
both counts. Having these “index” overloads appear in completion lists
alongside the ones that are there for use as high-level algorithms seems
wrong. Also, since the method either returns a modified version of the
index or modifies the index in place, the index argument is “more
primary” and should come first. Here's what we came up with:

     let j = location(i, offsetBy: 5)

     formLocation(&i, offsetBy: -2)

Feedback welcome.

--
Dave

[offtopic for "A New Model for Collections and Indices"]

Just wanted to add my 2 cents to this new naming guidelines proposal that @Dave pointed to:
"Update API Naming Guidelines and Rewrite Set APIs Accordingly"

I strongly feel this "form" prefix is a wrong decision. Is it really most of us feel this "formXXXX" method name not confusing and think this is a good solution? Just can't believe in this.

My mind reads "form" as "from" first. And then, when I re-checked, I see "fORm". I believe we see "from" much more often than "form" as in code and in our usual life, so we'll read it as "from" first.
I have some kind of prove : "I cdn'uolt blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg"

Additionally, I totally refuse to feel the meaning of "form" word as good replacement for meaning for "InPlace". InPlace is probably "visually heavyweight"(as noted in proposal) but IMO much more explicit on what we are doing and what we'll have in result.

I have no right now good alternative for "form", and probably the proposal was already accepted or probably really most of us agree with "form".

Probably I'll prefer to leave InPlace as in current Swift, or event make it a suffix(but all lowecased, thinking if we are using "in-place" as one word, not "in place" as two words):

y.inplaceUnion(z)

or probably

y.assignByUnion(z)
(as we think of this command as
y = y.union(z)
, we can just read it - "y is assigned by the value of y.union(z) "
"assign the variable "name" the value computed by "right_hand_expression""
=> "assign the y the value computer by union(z)" => assignByUnion(z)
for example, first found: http://www.cs.utah.edu/~germain/PPS/Topics/assignment_statement.html\)

or may be

y.mutateByUnion(z)
imo clear and explicit. we have 'mutating' when dealing with structs, here is similar behavior.

Probably we should rethink the mutating methods at all, and not trying to find a good word but introduce new syntax in Swift like.. I don't know.. some kind of y&.union(z) or y$.union(z) or y:union(z) etc.

Just some opinion. Thank you for reading this.

[/offtopic]

···

On 13.04.2016 21:24, Dave Abrahams via swift-evolution wrote:

In other cases, the mutating pair of methods refer to the receiver, not the
>argument.
>
>x = y.union(z) // new value x
>y.formUnion(z) // mutates y, not z
>
>x = y.successor(z) // new value x
>y.formSuccessor(z) // mutates z (or replaces), not y

This is true, but we need a way to deal with these cases as well.

       Aside: `indices` being irregular can be a benefit in the context of
       auto-complete.

       * What is your evaluation of the proposal?

       +1, very much.

       As a change from the current model, it’s an across-the-board improvement
       for me,
       at least.

       In a bigger-picture sense I think Swift would be better off by going
       *further*
       on certain aspects, but have said all that before.

       * Is the problem being addressed significant enough to warrant a change
       to
       Swift?

       It is, again very much so.

       * Does this proposal fit well with the feel and direction of Swift?

       Depends on the framing of the question.

       Compared to the previous model, it’s an unqualified YES.

       As a general proposition, I think this design is a local optimum for
       overall
       Swift-ness, but even so it’s creating a little un-Swifty pocket. It’s
       “un-Swifty” in at least two ways:

       # 1: Relatively Unsafe, Pointer-Like Semantics

       Indices—unsurprisingly!—behave quite a bit like pointers, and similarly
       expose
       *numerous* crashing combinations of `(value,operation)`:

       - self[endIndex]
       - self[startIndex] // <- when empty
       - successor(of: endIndex)
       - predecessor(of: startIndex)

       …etc., which is *very much* reminiscent of the hazards of pointers.
       (Technically
       “undefined” not “crashing”, but being realistic “crashing" is usually
       accurate).

   No, these are unspecified in the general case, not undefined. Unless
   you're working with, e.g. `UnsafeMutableBufferPointer` (or you have a
   data race), there's no undefined behavior. The big problem with
   pointers isn't what happens when they crash; it's what happens when they
   *don't*.

       Although Swift uses `Optional` to mitigate the hazards of `nil` pointers
       (etc.),
       you’re still left to your own devices for handling indices.

   `Optional` is not “mitigating hazards;” it's encoding the possibility of
   null in the type system. It's non-optional things that mitigate hazards.

       This isn’t news to anyone here, I’m sure, and may even be unavoidable;
       I’m just
       pointing it out as an uncharacteristically-unsafe area in Swift’s
       standard APIs,
       and closer to how `!` and IOUs behave than otherwise typical.

   Any time there's a required relationship between two things, e.g. a
   receiver and an argument, you have a precondition. The existence of a
   precondition does not make something unsafe at all in the sense that
   Swift uses the term. Safety in swift is about type and memory safety in
   the absence of data races, not about having APIs that respond sensibly
   to every possible combination of arguments. Int.max + 1 will trap, but
   that doesn't make addition unsafe.

   Saying that it's close to how `!` behaves is not at all far from the
   truth, because `!` has a precondition that its argument is non-nil.

I meant it as a much more exact analogy.

In a collections-move-indices world, you *could* handle indices as pointers have
been handled, bringing in support from the type-system:

enum SaferIndex<T:Comparable> {
case Position(T)
case End
}

…(yes, this is more-or-less `Optional` by another name).

The assumption above is `T` would be today’s “Index” types, w/o the value used
for `endIndex` (e.g. 0..<self.count for an array, the non-`endIndex` values of
`DictionaryIndex` and `SetIndex`, and so on).

No, you can't, at least not usefully. An Index that's at the end of one
collection is in the middle of another, or with a suitably-modified version
of the same collection.

Sure, in certain concrete scenarios it’s possible for one collection’s
indices to have such relationships to some other collection.

But, what of it?

In a generic context you can’t assume this;

That's incorrect. A slice's indices are *documented* as having a
particular relationship to those of the thing it was sliced from. This
applies everywhere. A dictionary's keys and values use the same indices
as the dictionary itself, and have a correspondence.

in a concrete context you naturally have more information.

Slices would become problematic, I’ll grant.

var x = [1, 2]
let i = x.index(1, stepsFrom: x.startIndex)
x.removeLast()
x[i] // fatal error: Index out of range

Indices can become invalid; this imposes preconditions. I don’t get
it.

My point is that whether i is at the end or not cannot be encoded in i.

The converse is also true: subscripting on a collection's endIndex is
sometimes just fine, even with no mutation in sight.

let a = (0..<10).reversed()
print(Array(a)) // “[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]”

let b = a.prefix(9)
print(Array(b)) // “[9, 8, 7, 6, 5, 4, 3, 2, 1]”

print(a[b.endIndex]) // “0” (correct, supported behavior)

I believe we are back to “subscripting one collection with *another*
collection's `endIndex`, no?

Totally legit, as mentioned above. a.prefix(9) returns a slice of a.

Are there any circumstances where a collection *can* be
usefully-subscripted with its *own* `endIndex`?

var a = [1]
let i = a.endIndex
a.append(2)
print(a[i]) // “2”

Of course,

b[b.endIndex] // As a matter of QOI: fatal error: out of bounds: index >= endIndex

It would’ve been awkward to do this under the previous status quo—e.g. even for
arrays your indices would have to have a back-reference to get the count, and
thus couldn’t be plain integers—but the collection will now always be present to
provide such info.

Cons:

- more overhead than “bare” indices
- doesn’t address invalidation (but what does, really?)

Pros:

- easier in some ways to handle things like e.g 0…Int.max
- the endIndex equivalent *never* invalidates
- compile-time help for end-index checking

Overall this *would* bring the treatment of indices closer to that for `?`—e.g.,
redefine the core type to omit the `nil`-like value,

Sorry, but that's the opposite of what `?` is doing: it *adds* a nil
value.

…I must have been unclear.

Step 1: Define T* = { "all memory addresses” (nil included) }
Step 2: Define T = T* \ { nil } (e.g. "non-null pointers")

…is what I was trying to summarize via “redefine the core type to omit
the `nil`-like value” (which is the important part here).

Sorry, that's still unclear to me. I just don't see what you're getting
at.

Anyways, having `endIndex` directly inhabit the same type as the
“good” indices has some pros and some cons; it’s not an IMHO one-sided
situation as with `nil`.

Maybe, but my point is that many things in the current model are
incompatible with the other arrangement. If you wanted to change the
arrangement, you'd need to re-think the current model from the ground
up, including index invalidation, how algorithms interact, the
relationship of slices to the thing they're sliced from, etc...

So what you're suggesting is an interesting hypothesis, but to me it's
not by any means obviously workable.

On the one hand, in my own experience so far, it’s definitely been the
case that most custom collections I’d done have had indices that’re
effectively the `SaferIndex` above; it’s been rather rare that there’s
been a natural “1 past the rest” value to use of the same type as is
used to describe the position of a “good” index.

Seriously, just because Swift has Optionals and they're useful for
safety in some scenarios (compared with allowing everything to be
nullable) does not mean that it's going to be “Swiftier” to apply a
similar pattern everywhere.

use an enum to reintroduce that value when necessary—than to `!`.

I don’t think the above is an *improvement* over the proposal, but it’s a route
that could have been taken.

I believe it would be hard to make such a design work at all, and if you
could make it work I think you'd end up with exactly the problem this
proposal aims to solve: references inside indices. So, I don't think
it's even a possibility, really.

I can’t say I see the impossibility. I definitely have experienced the
clunkiness.

This is getting too involved for a hypothetical I was explaining, but
not advocating.

I will happily agree to drop this topic :-)

This proposal and the new design is a good design!

Thanks!

···

on Wed Apr 13 2016, plx <swift-evolution@swift.org> wrote:

On Apr 13, 2016, at 5:36 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Wed Apr 13 2016, plx <swift-evolution@swift.org> wrote:

   On Apr 12, 2016, at 5:25 PM, Dave Abrahams via swift-evolution >>> <swift-evolution@swift.org> wrote:
   on Tue Apr 12 2016, plx >>> <swift-evolution@swift.org> wrote:

       To help illustrate the claim, here’s a strawman “safe” API—for
       illustration
       only, not advocacy!—that would be safer and thus perhaps more “Swift-y”:

   I think there's a prevalent misunderstanding (IOW, I don't mean to
   single out this post or this poster) about what “safe” means in Swift
   and what the features of a Swifty API are and should be. This
   is a big topic worthy of much more time than I can devote here, but
   here's a thought to start with:

   A Swifty API helps you reason effectively about the correctness of your
   code, and in part that means we provide enough preconditions on
   arguments to avoid complicating result types, and code to handle
   results, with optional-ness.

   --
   Dave

   _______________________________________________
   swift-evolution mailing list
   swift-evolution@swift.org
   https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave

    Seriously, just because Swift has Optionals and they're useful for
    safety in some scenarios (compared with allowing everything to be
    nullable) does not mean that it's going to be “Swiftier” to apply a
    similar pattern everywhere.

This reminds me of something I was going to ask about earlier and forgot.

Coming from other languages, I’ve definitely brought with me an assumption that
“make invalid states unrepresentable (except where unavoidable)” is the right
way to go, with deviations from that rule requiring scrutiny.

Is that outlook actually something the core team considers “Swift-y” or not,
though?

That seems very consistent with our outlook.

Note, however, that a nil optional does not represent an invalid state.
An invalid state is one in which locally-known invariants are broken.
When I say

     let a = Int("foobaz")

and a is nil, that's a valid representation of the fact that "foobaz"
doesn't parse as an Int.

Part of what this comes down to is whether you're willing to call some
uses of an API wrong (i.e. impose preconditions). Some people are very
reluctant to do that, but I am not. It benefits both efficiency and the
ability to reason about code.

···

on Wed Apr 13 2016, plx <swift-evolution@swift.org> wrote:

    On Apr 13, 2016, at 5:36 PM, Dave Abrahams via swift-evolution > <swift-evolution@swift.org> wrote:
    on Wed Apr 13 2016, plx > <swift-evolution@swift.org> wrote:

The connection to the point you’re making here and the question is that adopting
this design style *will* lead to lots of optionals in lots of places.

        use an enum to reintroduce that value when necessary—than to `!`.

        I don’t think the above is an *improvement* over the proposal, but it’s
        a route
        that could have been taken.

    I believe it would be hard to make such a design work at all, and if you
    could make it work I think you'd end up with exactly the problem this
    proposal aims to solve: references inside indices. So, I don't think
    it's even a possibility, really.

        To help illustrate the claim, here’s a strawman “safe” API—for
        illustration
        only, not advocacy!—that would be safer and thus perhaps more “Swift-y”:

        I think there's a prevalent misunderstanding (IOW, I don't mean to
        single out this post or this poster) about what “safe” means in Swift
        and what the features of a Swifty API are and should be. This
        is a big topic worthy of much more time than I can devote here, but
        here's a thought to start with:

        A Swifty API helps you reason effectively about the correctness of your
        code, and in part that means we provide enough preconditions on
        arguments to avoid complicating result types, and code to handle
        results, with optional-ness.

        --
        Dave

        _______________________________________________
        swift-evolution mailing list
        swift-evolution@swift.org
        https://lists.swift.org/mailman/listinfo/swift-evolution

        _______________________________________________
        swift-evolution mailing list
        swift-evolution@swift.org
        https://lists.swift.org/mailman/listinfo/swift-evolution

    --
    Dave

    _______________________________________________
    swift-evolution mailing list
    swift-evolution@swift.org
    https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave

The proposal requires Index values to be Comparable. Does that mean that indices from different collection instances should be comparable i.e. have a strict total order?

And somewhat related: Is it safe in Swift (in contrast to C) to compare NativePointers into unrelated memory blocks?

- Stephan

···

On 2016-04-12 Dmitri Gribenko via swift-evolution wrote:

Not even to mention that
indices are valid only in context of a particular collection instance,
so in this model you could validate an index against one collection
and use it with another one.

    > Quick thought:
    >
    > Why are you reaching for the "form..." rule for the mutating methods when
    there
    > are clear verb counterparts?
    > location: locate
    > successor: succeed

    We're not using successor(i) anymore, as noted below, and furthermore
    c.succeed(&i) strongly implies the wrong meaning.

I thought that's what I understood from the email, but in the linked proposal
they're still there (as are the many types of Range protocols). Wrong link, or
just not updated?

My mistake; I pushed to the wrong repo. Please try again.

    I didn't consider
    using

    c. locate(...:&i ... )

    primarily because I never thought of it and nobody suggested it IIRC,
    but I also don't see how it would work in a family with
    c.location(after: i) et al. Suggestions?

I didn't read this proposal carefully on its initial presentation for review.
Looking at it now, I wonder about the wisdom of "location"--I understand the
rationale of avoiding multiple methods named "index" that do different things,
but these particular functions return or mutate indices, and nowhere else are
these called "locations". If you're asking for possible alternative suggestions
to avoid using "index", I'll suggest the following here because I don't recall
seeing them offered previously. They read as phrases or sentences:

// taking inspiration from ForwardIndexType, which is no more...
c.advancing(_ i: Index, by offset: IndexDistance, limit: Index)

As I've said before, the “ing” suffix strongly implies we're returning
(a version of) c, not of i. c.f.

Please hand me your coat, emptying the left pocket.

You're not going to get a pocket; you're getting a whole coat.

c.advance(_ i: inout Index, by offset: IndexDistance, limit: Index)

// or alternatively, using the terminology in the comments that sit above
location
c.offsetting(_ i: Index, by n: IndexDistance, limit: Index)
c.offset(_ i: inout Index, by n: IndexDistance, limit: Index)

// and then, in place of successor, etc.
c.incrementing(_ i: Index, limit: Index)
c.increment(_ i: inout Index, limit: Index)
c.decrementing(_ i: Index, limit: Index)
c.decrement(_ i: inout Index, limit: Index)

\(&quot;&#39;Limit&#39; doesn&#39;t read like a phrase,&quot; you might say\. Well, think of a coupon:
&quot;$1 off one tub of margarine\. Limit one per purchase\. Void if transferred or
sold\.&quot;\)

the limit label is the least of my concerns here :-)

···

on Mon Apr 25 2016, Xiaodi Wu <xiaodi.wu-AT-gmail.com> wrote:

On Mon, Apr 25, 2016 at 6:15 PM, Dave Abrahams <dabrahams@apple.com> wrote:
    on Mon Apr 25 2016, Xiaodi Wu <xiaodi.wu-AT-gmail.com> wrote:

    > On Mon, Apr 25, 2016 at 1:24 PM, Dave Abrahams via swift-evolution > > <swift-evolution@swift.org> wrote:
    >
    > on Wed Apr 20 2016, Chris Lattner <swift-evolution@swift.org> wrote:
    >
    > > On Apr 10, 2016, at 2:41 PM, Chris Lattner > > > <clattner@apple.com> wrote:
    > >
    > > Hello Swift community,
    > >
    > > The review of "A New Model for Collections and Indices" begins now and
    > runs
    > > through April 18th. The proposal is available here:
    > >
    > >
    >
    https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md

    >
    > >
    > > Reviews are an important part of the Swift evolution process. All
    reviews
    > > should be sent to the swift-evolution mailing list at:
    > > https://lists.swift.org/mailman/listinfo/swift-evolution
    > > or, if you would like to keep your feedback private, directly to the
    > review
    > > manager.
    > >
    > > A quick update: the core team met to discuss this. They agreed to accept
    > it with
    > > some naming-related revisions to the proposal (in response to community
    > > feedback). Dave is organizing this feedback, and I’ll send out the
    formal
    > > announcement when that is ready.
    >
    > The final revisions are reflected in the latest version of the
    > proposal:
    >
    >
    https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md

    >
    > Summary:
    >
    > * We decided to take Shawn Erickson's excellent suggestion
    > <http://article.gmane.org/gmane.comp.lang.swift.evolution/14450&gt; to
    > use “location” uniformly for index movement, so instead of
    > successor(i) and predecessor(i) we have location(after: i) and
    > location(before: i).
    >
    > * Since Brent Royal-Gordon pointed out
    >
    <http://news.gmane.org/find-root.php?message_id=156D8FB1-1FD3-448E-8C70-6E7400629BC0%40architechies.com

    > >
    > that two of the three proposed Range protocols would likely disappear
    > in future updates, we took another look at all of them. Finding
    > `RangeProtocol` itself to be a very weak abstraction, we removed all
    > three from the proposal.
    >
    > For those interested in details, implementation work proceeds apace on
    > the swift-3-indexing-model branch at
    >
    <https://github.com/apple/swift/tree/swift-3-indexing-model/stdlib/public/core

    > >.
    >
    > P.S. If anyone is interested in contributing, there are still plenty of
    > FIXMEs left for us to handle ;-)
    >
    > --
    > Dave
    >
    > _______________________________________________
    > swift-evolution mailing list
    > swift-evolution@swift.org
    > https://lists.swift.org/mailman/listinfo/swift-evolution
    >

    --
    Dave

--
Dave

>
>
> > Quick thought:
> >
> > Why are you reaching for the "form..." rule for the mutating
methods when
> there
> > are clear verb counterparts?
> > location: locate
> > successor: succeed
>
> We're not using successor(i) anymore, as noted below, and furthermore
> c.succeed(&i) strongly implies the wrong meaning.
>
> I thought that's what I understood from the email, but in the linked
proposal
> they're still there (as are the many types of Range protocols). Wrong
link, or
> just not updated?

My mistake; I pushed to the wrong repo. Please try again.

I see a new version, but I still see .successor().

> I didn't consider
> using
>
> c. locate(...:&i ... )
>
> primarily because I never thought of it and nobody suggested it IIRC,
> but I also don't see how it would work in a family with
> c.location(after: i) et al. Suggestions?
>
> I didn't read this proposal carefully on its initial presentation for
review.
> Looking at it now, I wonder about the wisdom of "location"--I understand
the
> rationale of avoiding multiple methods named "index" that do different
things,
> but these particular functions return or mutate indices, and nowhere
else are
> these called "locations". If you're asking for possible alternative
suggestions
> to avoid using "index", I'll suggest the following here because I don't
recall
> seeing them offered previously. They read as phrases or sentences:
>
> ```
> // taking inspiration from ForwardIndexType, which is no more...
> c.advancing(_ i: Index, by offset: IndexDistance, limit: Index)

As I've said before, the “ing” suffix strongly implies we're returning
(a version of) `c`, not of `i`. c.f.

   Please hand me **your coat, emptying the left pocket**.

You're not going to get a pocket; you're getting a whole coat.

Quite right; didn't mean to retread that. I feel the same deficiency
applies to using the "form" convention, though, in that (at least as has
been discussed on this list) the convention usually refers to the receiver
being mutated. Thus, `c.formLocation(...)` sounds like `c` should be
mutated in some way.

One way out that I can think of is looking to good ol' Objective-C
conventions. By this I mean that, in my mind, shorter method names like
`str.appending(...)` are derived by omitting redundant words from longer
ancestral names such as `str.stringByAppendingString(...)`. In this
particular case, certain words are not redundant and perhaps we should just
bravely put back those words that are necessary to clarify.

That is, if this were Objective-C, we'd have something like
"indexByAdvancingIndex". You're quite right that we can't use just
"advancing" because it implies returning a version of the receiver. We've
tried "index", but then it conflicts with another method "index". Now
there's renaming "index" to "location", even though it returns a thing of
type Index... Aren't the most succinct but still accurate method names
instead: `c.indexByAdvancing(i, ...)` and `c.advanceIndex(&i, ...)`?
[Incidentally, `c.advance` might read like c is being advanced.]

> c.advance(_ i: inout Index, by offset: IndexDistance, limit: Index)
>
> // or alternatively, using the terminology in the comments that sit above
> `location`
> c.offsetting(_ i: Index, by n: IndexDistance, limit: Index)
> c.offset(_ i: inout Index, by n: IndexDistance, limit: Index)
>
> // and then, in place of successor, etc.
> c.incrementing(_ i: Index, limit: Index)
> c.increment(_ i: inout Index, limit: Index)
> c.decrementing(_ i: Index, limit: Index)
> c.decrement(_ i: inout Index, limit: Index)
> ```
> ("'Limit' doesn't read like a phrase," you might say. Well, think of a
coupon:
> "$1 off one tub of margarine. Limit one per purchase. Void if
transferred or
> sold.")

the limit label is the least of my concerns here :-)

That said, orthogonally, I feel like many `limitedBy` labels can be
simplified to `limit` :)

···

On Mon, Apr 25, 2016 at 8:25 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Apr 25 2016, Xiaodi Wu <xiaodi.wu-AT-gmail.com> wrote:
> On Mon, Apr 25, 2016 at 6:15 PM, Dave Abrahams <dabrahams@apple.com> > wrote:
> on Mon Apr 25 2016, Xiaodi Wu <xiaodi.wu-AT-gmail.com> wrote:

> > On Mon, Apr 25, 2016 at 1:24 PM, Dave Abrahams via swift-evolution > > > <swift-evolution@swift.org> wrote:
> >
> > on Wed Apr 20 2016, Chris Lattner <swift-evolution@swift.org> > wrote:
> >
> > > On Apr 10, 2016, at 2:41 PM, Chris Lattner > > > > <clattner@apple.com> wrote:
> > >
> > > Hello Swift community,
> > >
> > > The review of "A New Model for Collections and Indices" begins
now and
> > runs
> > > through April 18th. The proposal is available here:
> > >
> > >
> >
>
https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md
>
> >
> > >
> > > Reviews are an important part of the Swift evolution process. All
> reviews
> > > should be sent to the swift-evolution mailing list at:
> > > https://lists.swift.org/mailman/listinfo/swift-evolution
> > > or, if you would like to keep your feedback private, directly to
the
> > review
> > > manager.
> > >
> > > A quick update: the core team met to discuss this. They agreed
to accept
> > it with
> > > some naming-related revisions to the proposal (in response to
community
> > > feedback). Dave is organizing this feedback, and I’ll send out
the
> formal
> > > announcement when that is ready.
> >
> > The final revisions are reflected in the latest version of the
> > proposal:
> >
> >
>
https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md
>
> >
> > Summary:
> >
> > * We decided to take Shawn Erickson's excellent suggestion
> > <http://article.gmane.org/gmane.comp.lang.swift.evolution/14450&gt;
to
> > use “location” uniformly for index movement, so instead of
> > successor(i) and predecessor(i) we have location(after: i) and
> > location(before: i).
> >
> > * Since Brent Royal-Gordon pointed out
> >
> <
http://news.gmane.org/find-root.php?message_id=156D8FB1-1FD3-448E-8C70-6E7400629BC0%40architechies.com
>
> > >
> > that two of the three proposed Range protocols would likely
disappear
> > in future updates, we took another look at all of them. Finding
> > `RangeProtocol` itself to be a very weak abstraction, we removed
all
> > three from the proposal.
> >
> > For those interested in details, implementation work proceeds
apace on
> > the swift-3-indexing-model branch at
> >
> <
https://github.com/apple/swift/tree/swift-3-indexing-model/stdlib/public/core
>
> > >.
> >
> > P.S. If anyone is interested in contributing, there are still
plenty of
> > FIXMEs left for us to handle ;-)
> >
> > --
> > Dave
> >
> > _______________________________________________
> > swift-evolution mailing list
> > swift-evolution@swift.org
> > https://lists.swift.org/mailman/listinfo/swift-evolution
> >
>
> --
> Dave
>

--
Dave

    > Quick thought:
    >
    > Why are you reaching for the "form..." rule for the mutating methods when
    there
    > are clear verb counterparts?
    > location: locate
    > successor: succeed

    We're not using successor(i) anymore, as noted below, and furthermore
    c.succeed(&i) strongly implies the wrong meaning.

I thought that's what I understood from the email, but in the linked proposal
they're still there (as are the many types of Range protocols). Wrong link, or
just not updated?

Gah!

    I didn't consider
    using

    c. locate(...:&i ... )

    primarily because I never thought of it and nobody suggested it IIRC,
    but I also don't see how it would work in a family with
    c.location(after: i) et al. Suggestions?

I didn't read this proposal carefully on its initial presentation for review.
Looking at it now, I wonder about the wisdom of "location"--I understand the
rationale of avoiding multiple methods named "index" that do different things,
but these particular functions return or mutate indices, and nowhere else are
these called "locations".

And I found this quite compelling, so after some discussion here I have
gone from “location” back to “index” everywhere.

···

on Mon Apr 25 2016, Xiaodi Wu <swift-evolution@swift.org> wrote:

On Mon, Apr 25, 2016 at 6:15 PM, Dave Abrahams > <dabrahams@apple.com> wrote:
    on Mon Apr 25 2016, Xiaodi Wu <xiaodi.wu-AT-gmail.com> wrote:

If you're asking for possible alternative suggestions to avoid using
"index", I'll suggest the following here because I don't recall seeing
them offered previously. They read as phrases or sentences:

// taking inspiration from ForwardIndexType, which is no more...
c.advancing(_ i: Index, by offset: IndexDistance, limit: Index)
c.advance(_ i: inout Index, by offset: IndexDistance, limit: Index)

// or alternatively, using the terminology in the comments that sit above
`location`
c.offsetting(_ i: Index, by n: IndexDistance, limit: Index)
c.offset(_ i: inout Index, by n: IndexDistance, limit: Index)

// and then, in place of successor, etc.
c.incrementing(_ i: Index, limit: Index)
c.increment(_ i: inout Index, limit: Index)
c.decrementing(_ i: Index, limit: Index)
c.decrement(_ i: inout Index, limit: Index)

("'Limit' doesn't read like a phrase," you might say. Well, think of a coupon:
"$1 off one tub of margarine. Limit one per purchase. Void if transferred or
sold.")

    > On Mon, Apr 25, 2016 at 1:24 PM, Dave Abrahams via swift-evolution > > <swift-evolution@swift.org> wrote:
    >
    > on Wed Apr 20 2016, Chris Lattner > <swift-evolution@swift.org> wrote:
    >
    > > On Apr 10, 2016, at 2:41 PM, Chris Lattner > > > <clattner@apple.com> wrote:
    > >
    > > Hello Swift community,
    > >
    > > The review of "A New Model for Collections and Indices" begins now and
    > runs
    > > through April 18th. The proposal is available here:
    > >
    > >
    >
    https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md

    >
    > >
    > > Reviews are an important part of the Swift evolution process. All
    reviews
    > > should be sent to the swift-evolution mailing list at:
    > > https://lists.swift.org/mailman/listinfo/swift-evolution
    > > or, if you would like to keep your feedback private, directly to the
    > review
    > > manager.
    > >
    > > A quick update: the core team met to discuss this. They agreed to accept
    > it with
    > > some naming-related revisions to the proposal (in response to community
    > > feedback). Dave is organizing this feedback, and I’ll send out the
    formal
    > > announcement when that is ready.
    >
    > The final revisions are reflected in the latest version of the
    > proposal:
    >
    >
    https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md

    >
    > Summary:
    >
    > * We decided to take Shawn Erickson's excellent suggestion
    > <http://article.gmane.org/gmane.comp.lang.swift.evolution/14450&gt; to
    > use “location” uniformly for index movement, so instead of
    > successor(i) and predecessor(i) we have location(after: i) and
    > location(before: i).
    >
    > * Since Brent Royal-Gordon pointed out
    >
    <http://news.gmane.org/find-root.php?message_id=156D8FB1-1FD3-448E-8C70-6E7400629BC0%40architechies.com

    > >
    > that two of the three proposed Range protocols would likely disappear
    > in future updates, we took another look at all of them. Finding
    > `RangeProtocol` itself to be a very weak abstraction, we removed all
    > three from the proposal.
    >
    > For those interested in details, implementation work proceeds apace on
    > the swift-3-indexing-model branch at
    >
    <https://github.com/apple/swift/tree/swift-3-indexing-model/stdlib/public/core

    > >.
    >
    > P.S. If anyone is interested in contributing, there are still plenty of
    > FIXMEs left for us to handle ;-)
    >
    > --
    > Dave
    >
    > _______________________________________________
    > swift-evolution mailing list
    > swift-evolution@swift.org
    > https://lists.swift.org/mailman/listinfo/swift-evolution
    >

    --
    Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave

    (On the other hand, it might be that I'm conceiving of the purpose of
        `limitedBy` differently from you—I think of it as a safety measure, but
        you may be thinking of it specifically as an automatic truncation
        mechanism.)

    Hi Brent,

    Could you explain what kind of safety do you have in mind? Swift will
    guarantee memory safety even if you attempt to advance an index past
    endIndex using the non-limiting overload.

One challenge that I've run into is that the `limitedBy` methods throw away one
bit of information—namely, did I move as far as I requested or not?

You're right, Nate. I think we'll try to fix that in the next iteration.

···

on Tue Apr 12 2016, Nate Cook <swift-evolution@swift.org> wrote:

    On Apr 12, 2016, at 4:15 AM, Dmitri Gribenko via swift-evolution > <swift-evolution@swift.org> wrote:
    On Mon, Apr 11, 2016 at 9:56 PM, Brent Royal-Gordon via > swift-evolution <swift-evolution@swift.org> wrote:

Example:

let j = c.index(10, stepsFrom: i, limitedBy: c.endIndex)

There's no way to interpolate the answer to that question efficiently in a
non-random-access collection. If `j` is equal to `c.endIndex`, that could be
because `c.endIndex` is ten steps after `i` *or* because the limit kicked in,
and without checking `c.distance(from: i, to: j)` there's no way to know for
sure.

If the `limitedBy` methods returned an optional index, we'd get all the
information that the index-moving algorithm finds (let's hear it for the Law of
Useful Return!). With that API, we could decide whether to use the returned
index or not:

// Use the resulting index no matter what:
let i = c.index(10, stepsFrom: c.startIndex, limitedBy: c.endIndex) ??
c.endIndex
let prefix = c.prefix(upTo: i)

// Only use the result if it wasn't limited:
if let j = c.index(10, stepsFrom: i, limitedBy: c.endIndex) {
let sub = c[i..<j] // sub.count == 10
} else {
// not enough elements...
}

// "Safe" successor:
if let j = c.index(1, stepsFrom: i, limitedBy: c.endIndex) {
// ...
}

Nate

    Dmitri

    --
    main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
    (j){printf("%d\n",i);}}} /*Dmitri Gribenko
    <gribozavr@gmail.com>*/
    _______________________________________________
    swift-evolution mailing list
    swift-evolution@swift.org
    https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave

x.modify(&y)

seems to work just fine.

···

on Tue Apr 12 2016, Brent Royal-Gordon <brent-AT-architechies.com> wrote:

Huh, that clarifies something. How about the non-`ing` variants?

  collection.travel(5, from: i)
  collection.stride(5, from: i)
  collection.advance(i, by: 5)

Active verb phrases are reserved for methods with side-effects according
to the API guidelines.

Is there *any* way of using a verb to indicate an argument is being
modified? If not, I think that's a serious omission in the API
guidelines.

--
Dave

(On the other hand, it might be that I'm conceiving of the purpose of `limitedBy` differently from you—I think of it as a safety measure, but you may be thinking of it specifically as an automatic truncation mechanism.)

Hi Brent,

Could you explain what kind of safety do you have in mind? Swift will
guarantee memory safety even if you attempt to advance an index past
endIndex using the non-limiting overload.

By "safety" here, I mean what I will call "index safety": not accidentally using an index which would violate the preconditions of the methods or properties you are planning to use it with. I think it's too easy to accidentally overrun the permitted range of indices, and the API should help you avoid doing that.

Hi Brent,

Thank you for the explanation. Assuming that I am interpreting you
correctly, I can't agree with your conclusions though.

I want to make a point that avoiding precondition violations by
removing preconditions is not the solution. When you design an API,
it frequently has some constraints on the arguments or on the
execution environment, which, when violated, prevent the API from
performing the operation correctly. As an API designer, you document
these constraints, but for the implementation you have two choices:

1. You rigorously check the constraints, and return back some
indicator of the failure back to the caller (a nil return, a thrown
error etc.) This decision makes the checks guaranteed, permanent,
documented part of your API behavior. Users can now rely on these
checks for their normal logic. In this case, the constraints are not
preconditions that the user is required to satisfy.

2. You perform the operation anyway, relying on the caller to satisfy
the constraints. The constraints become preconditions. The
implementation can still check some of the constraints for either QoI
reasons or memory safety reasons. Apart from preventing memory safety
violations, the implementation is not required to check the
constraints.

Now the question becomes, given a certain API design problem, how do
you make a choice between (1) and (2)? In some cases, the answer is
clear:

- You can be forced to make your constraints into checks in API
behavior. Example: a network i/o operation can fail. This is a
completely normal thing that networks do sometimes, and the caller of
networking code should be expected to deal with networking failures,
or such code can't work on a real network.

- Some preconditions just can't be checked at all, or can't be checked
in time that is acceptable for the intended API purpose. A trivial
example is UnsafePointer.pointee (we can't know whether the pointer
actually points to an instance of a correct type). Another example is
sort predicates -- we can't check that the order defined by the
predicate is indeed a strict weak ordering for all values of the type.
We can check that the predicate defines a strict weak order for all
values in a particular dataset, but that requires O(n^2) time while
sort() complexity is O(n log n).

What about those cases when both (1) and (2) are implementable? There
exists a trade-off between making the API "resilient" to programming
mistakes and hiding bugs by not stopping the application when an issue
happens. We are very conservative in Swift about hiding errors.

If your APIs document that they check preconditions and trap if they
are not satisfied, then developers can rely on it, and you are not
allowed to weaken remove the checks in future. If the app can recover
from a precondition failure, the recovery path just becomes a part of
the API behavior. You can end up with a system where every API can
signal failure back to the calling code. Now users have to handle
these failures. There are two ways to handle them:

- force-unwrap, or 'try!'.

- follow the advice of people who advocate never using force-unwrap
and 'try!'. The code that does so will likely have a lot of untested,
untestable, and dead branches that try to deal defensively with
situations that can't happen, but the API forces them to do so anyway.

I don't think either option leads to good code. Thus, it is only
reasonable that we design the API based on a purpose that we intend
for this API.

Let's talk about how this applies to Collection.

For example, Collection APIs in question that work with indices are
primitive APIs that the rest of Collection APIs build upon. One of
these APIs, basically the reason why indices exist, is
Collection.subscript(Index). Today the behavior is unspecified if the
index is not valid. If we follow the principle that you outlined:

not accidentally using an index which would violate the preconditions of the methods or properties you are planning to use it with.

Collection.subscript(Index) should return an optional. Does this
match your expectations? If so, how do you imagine even trivial
algorithms written?

for i in data.indices {
  data[i]! = data[i]! * 2 // assume that 'data[i]!' can be made settable.
}

This code has two force-unwraps in it, despite being completely safe.
If you follow the "never force-unwrap" school of thought, then you end
up with two branches, and an error return from this function, that are
dead, untestable code.

If every API can fail, then you can't write useful code. You need to
have some fundamental basis that you can rely on, and trust to operate
correctly.

The primitive Collection APIs are the basis of the Collection API. If
they would be burdened with required range checks, then, just as you
are saying, the code would be full of redundant range checks. For
example, UnsafeBufferPointer would also be burdened with them.

I think it's too easy to accidentally overrun the permitted range of indices, and the API should help you avoid doing that.

I wholeheartedly agree that APIs should help you to avoid making
mistakes. There are multiple ways in which they can do so:

1. (best) Make wrong code not compile. Swift has a strong type
system that allows us to express complex constraints between types,
preventing some mistakes from being compiled.

2. Check preconditions where it is possible (implementable), and when
the performance hit is reasonable, and deterministically trap as soon
as the violation was detected.

Unfortunately, validity of indices is a complex dynamic property of
the index-collection instance pair. We can't encode it in the static
type system. Consider the following example:

···

On Tue, Apr 12, 2016 at 4:27 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

---
var a = [1, 2, 3]
let i: Int = a.check(a.startIndex)
a.removeAll()
a[i] // trap
---

Ok, mutation can invalidate indices. Can immutability help?

---
let a = [1, 2, 3]
let b: [Int] =
let i = a.check(a.startIndex)
b[i] // trap
---

Since it is not possible to encode the validity relationship in the
type system, we want to encourage these operations to trap if they
detect violations.

Now let's go back to the demangling algorithm. We know that we have
weaknesses in the String parsing, and it is important for us to know
how well we are covering these use cases, and how we need to improve.
I think this is a great problem.

I implemented it using three different approaches:

A high-level point that I'd like to make is that when using
collections, you need to make use of algorithms to the biggest extent
possible. This allows to express the semantics of your operation in
your domain using a high-level vocabulary.

I would like to draw attention to the following part:

// Approach #3: change Collection.index(_:stepsFrom:limitedBy:) to return an
// optional index.
//
// This method has to perform the range check to stop advancing the index when
// it reaches the limit. Currently it just discards the information about
// whether it reached the limit or not. Instead, it can cheaply return it to
// the caller.
//
// Note that the same logic does not apply to other
// Collection.index(_:stepsFrom:) overloads.

We will change the index(_:stepsFrom:limitedBy:) overload to return an
optional, and we will see what other implications it has, and how it
fits into the rest of the system.

Thanks again, Brent.

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

Reverse is the best opposite we have of advance, so it makes sense to
me.

Oh, I get it.

Or we could use retreat. =) There are other pairs of words that work
as well, like “increment/decrement”.

Yeah, unfortunately those carry an incorrect implication when the
indices are numbers, because, e.g. the collection might be offsetting
the number by 2 for each position. One could of course argue that using
numbers that way as indices was a bad design choice.

I'll have to think about that idea again. We considered and rejected it
for a reason, but it might not be a really strong one. Thanks for
bringing it up.

...and having talked it over at lunch, now I remember why we rejected
it: there's no good way to make a nonmutating version.

let x = c.incremented(i) // reads like an assertion about the past
let y = c.incrementing(i) // reads like it has side-effects and returns c, or
                            // a new version of c

In fact, it does return a new version* of c; just like this:

let s2 = myString.appending(“foo”)

*new version: the result is related to the argument

This works out great:

let next = c.incrementing(first)
c.increment(&next)

- Tony

···

On Apr 13, 2016, at 12:57 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:
on Wed Apr 13 2016, Dave Abrahams <swift-evolution@swift.org> wrote:

APIs where the receiver returns a modified version of an argument don't
lend themselves to verb forms.

--
Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Updated proposal:

https://github.com/apple/swift-evolution/blob/master/proposals/0065-collections-move-indices.md

I like the use of `location(...)` instead of `index(...)`. I strongly
suggest that successor and predecessor become `location(after:)` and
`location(before:)` (or something like that) so that the index manipulation
API are all similarly named.

-Shawn

···

On Wed, Apr 13, 2016 at 4:45 PM Dave Abrahams via swift-evolution < swift-evolution@swift.org> wrote:

[offtopic for "A New Model for Collections and Indices"]

Just wanted to add my 2 cents to this new naming guidelines proposal that @Dave pointed to:
"Update API Naming Guidelines and Rewrite Set APIs Accordingly"
https://github.com/apple/swift-evolution/blob/master/proposals/0059-updated-set-apis.md

I strongly feel this "form" prefix is a wrong decision. Is it really most of us feel this "formXXXX" method name not confusing and think this is a good solution? Just can't believe in this.

My mind reads "form" as "from" first. And then, when I re-checked, I see "fORm". I believe we see "from" much more often than "form" as in code and in our usual life, so we'll read it as "from" first.

Non-native speaker here, I had/have the exact same problem. Always read “from”.

···

On Apr 14, 2016, at 9:00 AM, Vladimir.S via swift-evolution <swift-evolution@swift.org> wrote:

I have some kind of prove : "I cdn'uolt blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg"
Transposed letter effect - Wikipedia

Additionally, I totally refuse to feel the meaning of "form" word as good replacement for meaning for "InPlace". InPlace is probably "visually heavyweight"(as noted in proposal) but IMO much more explicit on what we are doing and what we'll have in result.

I have no right now good alternative for "form", and probably the proposal was already accepted or probably really most of us agree with "form".

Probably I'll prefer to leave InPlace as in current Swift, or event make it a suffix(but all lowecased, thinking if we are using "in-place" as one word, not "in place" as two words):

y.inplaceUnion(z)

or probably

y.assignByUnion(z)
(as we think of this command as
y = y.union(z)
, we can just read it - "y is assigned by the value of y.union(z) "
"assign the variable "name" the value computed by "right_hand_expression""
=> "assign the y the value computer by union(z)" => assignByUnion(z)
for example, first found: http://www.cs.utah.edu/~germain/PPS/Topics/assignment_statement.html\)

or may be

y.mutateByUnion(z)
imo clear and explicit. we have 'mutating' when dealing with structs, here is similar behavior.

Probably we should rethink the mutating methods at all, and not trying to find a good word but introduce new syntax in Swift like.. I don't know.. some kind of y&.union(z) or y$.union(z) or y:union(z) etc.

Just some opinion. Thank you for reading this.

[/offtopic]

On 13.04.2016 21:24, Dave Abrahams via swift-evolution wrote:

In other cases, the mutating pair of methods refer to the receiver, not the
>argument.
>
>x = y.union(z) // new value x
>y.formUnion(z) // mutates y, not z
>
>x = y.successor(z) // new value x
>y.formSuccessor(z) // mutates z (or replaces), not y

This is true, but we need a way to deal with these cases as well.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

No, you can't, at least not usefully. An Index that's at the end of one
collection is in the middle of another, or with a suitably-modified version
of the same collection.

Sure, in certain concrete scenarios it’s possible for one collection’s
indices to have such relationships to some other collection.

But, what of it?

In a generic context you can’t assume this;

That's incorrect. A slice's indices are *documented* as having a
particular relationship to those of the thing it was sliced from. This
applies everywhere. A dictionary's keys and values use the same indices
as the dictionary itself, and have a correspondence.

You’re right, of course; I rarely use slices and completely overlooked them.

I also phrased it badly, b/c what I was trying to express is that code like the below is (I think?) unlikely to work generically:

  extension Collection where Element:Equatable {

     // plz don’t do this
     func hasValueMismatch(with other: Self, at index: Index) -> Bool {
       return self[index] != other[index]
     }

     // plz don’t do this either
    func hasValueMismatch<K:Collection where K.Index == Index, K.Element == Self.Element>(with other: K, at index: Index) -> Bool {
      return self[index] != other[index]
    }

  }

…(you would’t write the above anyway, but it illustrates the kind of "generic context" I had in mind when I wrote it).

in a concrete context you naturally have more information.

Slices would become problematic, I’ll grant.

var x = [1, 2]
let i = x.index(1, stepsFrom: x.startIndex)
x.removeLast()
x[i] // fatal error: Index out of range

Indices can become invalid; this imposes preconditions. I don’t get
it.

My point is that whether i is at the end or not cannot be encoded in i.

I see the miscommunication, now. Of course you can’t encode that.

I’ve put a couple examples down below as a last effort at communicating what I’m getting at it.

The converse is also true: subscripting on a collection's endIndex is
sometimes just fine, even with no mutation in sight.

let a = (0..<10).reversed()
print(Array(a)) // “[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]”

let b = a.prefix(9)
print(Array(b)) // “[9, 8, 7, 6, 5, 4, 3, 2, 1]”

print(a[b.endIndex]) // “0” (correct, supported behavior)

I believe we are back to “subscripting one collection with *another*
collection's `endIndex`, no?

Totally legit, as mentioned above. a.prefix(9) returns a slice of a.

Are there any circumstances where a collection *can* be
usefully-subscripted with its *own* `endIndex`?

var a = [1]
let i = a.endIndex
a.append(2)
print(a[i]) // “2”

Of course,

b[b.endIndex] // As a matter of QOI: fatal error: out of bounds: index >= endIndex

It would’ve been awkward to do this under the previous status quo—e.g. even for
arrays your indices would have to have a back-reference to get the count, and
thus couldn’t be plain integers—but the collection will now always be present to
provide such info.

Cons:

- more overhead than “bare” indices
- doesn’t address invalidation (but what does, really?)

Pros:

- easier in some ways to handle things like e.g 0…Int.max
- the endIndex equivalent *never* invalidates
- compile-time help for end-index checking

Overall this *would* bring the treatment of indices closer to that for `?`—e.g.,
redefine the core type to omit the `nil`-like value,

Sorry, but that's the opposite of what `?` is doing: it *adds* a nil
value.

…I must have been unclear.

Step 1: Define T* = { "all memory addresses” (nil included) }
Step 2: Define T = T* \ { nil } (e.g. "non-null pointers")

…is what I was trying to summarize via “redefine the core type to omit
the `nil`-like value” (which is the important part here).

Sorry, that's still unclear to me. I just don't see what you're getting
at.

Anyways, having `endIndex` directly inhabit the same type as the
“good” indices has some pros and some cons; it’s not an IMHO one-sided
situation as with `nil`.

Maybe, but my point is that many things in the current model are
incompatible with the other arrangement. If you wanted to change the
arrangement, you'd need to re-think the current model from the ground
up, including index invalidation, how algorithms interact, the
relationship of slices to the thing they're sliced from, etc...

So what you're suggesting is an interesting hypothesis, but to me it's
not by any means obviously workable.

You’re completely right about slices. I’ll provide a couple concrete examples before addressing the rest.

Here are three collection-combinators (or adapters I think you’d call them):

  // Collection with elements of A, then elements of B.
  struct ChainCollection<A:Collection,B:Collection> : Collection {
    let a: A; let b: B;
  }

  // Collection with elements `(a,b)` for each pair in the cartesian product of `A` and `B`.
  struct ProductCollection<A:Collection,B:Collection> : Collection {
    let a: A; let b: B;
  }

  // Collection with adjacent elements from A
  struct AdjacentElementCollection<A:Collection> : Collection {
    let a: A
  }

…each of which I’ve declared `: Collection` but each which will still need some suitable `Index` implementation.

Here’s one way to write these indices (henceforth, the V1 indices):

  // `endIndex` will be .InB(b.endIndex)
  enum ChainCollectionIndex<A:Collection,B:Collection> {
    // Precondition: the index isn’t `a.endIndex`.
    case InA(A.Index)
    case InB(B.Index)
  }

  // `endIndex` will have both `.aIndex` and `.bIndex` equal to their “source”'s `endIndex`
  struct ProductCollectionIndex<A:Collection,B:Collection> {
    let aIndex: A.Index
    let bIndex: B.Index
  }

  // `endIndex` will be both of these set to `a.endIndex`
  // - all other situations expect `upper` is `lower`’s successor, and both != `endIndex`
  struct AdjacentElementCollectionIndex<A:Collection> {
    let lower: A.Index
    let upper: A.Index
  }

…(I trust the index-manipulation boilerplate is easy to fill-in).

There’s absolutely nothing wrong with the above! Each of these types has the capability to represent the “good” indices, and also has a reasonable way to represent `endIndex` values.

But, they could also be written like so (henceforth, the V2 indices):

  enum ChainCollectionIndex<A:Collection,B:Collection> {
    // Precondition: the index isn’t `a.endIndex`.
    case InA(A.Index)
    // Precondition: the index isn’t `b.endIndex`.
    case InB(B.Index)
    // `endIndex` sentinel
    case End
  }

  enum ProductCollectionIndex<A:Collection,B:Collection> {
    // Precondition: neither index is the source collection’s `endIndex`
    case Item(A.Index, B.Index)
    // `endIndex` sentinel
    case End
  }

  enum AdjacentElementCollectionIndex<A:Collection> {
    // Precondition: `upper` is `lower`’s successor, *both* are != `a.endIndex`
    case Adjacency(lower: A.Index, upper: A.Index)
    // `endIndex` sentinel
    case End
  }

…each of which is essentially the V1 version, except now with a dedicated `endIndex` value tacked-on.

Tacking on the dedicated `endIndex` isn’t necessary, but at least for me taking V2-style approaches has been very advantageous.

Most of the advantage has been from being able to enforce stricter invariants on the non-endIndex indices, which generally makes the code simpler and also easier to reason about; I’m also fortunate in that I’m not working on the standard library, and thus can choose how heavily to weight “time to correct implementation” vis-a-vis “proximity to maximum possible efficiency”.

The above indices are drawn from what are admittedly simple combinators/adaptors, but they feel like representative examples for me; even for fancier, actually-custom things it’s almost always been much more natural to go with a V2-style index (and sometimes no natural V1-style approach even exists).

I’ve done a fair amount of experimenting with the model implied above—moving `endIndex` into a dedicated sentinel, etc.—and although it’s pretty easy overall to translate back and forth between the two models, the “alternative” approach *at best* comes out a wash...at best.

For the most part, invalidation is about the same between the two models for all non-end-index—for such indices, the same mutations invalidate the same indices under either approach.

What *is* different between the two is that a cached `endIndex` will never become valid due to mutation, e.g. this won’t happen:

  // status quo:
  var items = [“a”, “b”]
  let cachedEndIndex = items.endIndex // this is just `2`
  items.append[“c”]
  items[cachedEndIndex] // returns “c"

  // dedicated `endIndex`:
  var items = [“a”, “b”]
  let cachedEndIndex = items.endIndex
  items.append[“c”]
  items[cachedEndIndex] // goes boom, b/c `cachedEndIndex` is still the `endIndex`

…and things like this wouldn't work the same way:

  // status quo:
  var items = [“a”, “b”]
  let cachedEndIndex = items.endIndex // this is just `2`
  items.insert(“c”, at: cachedEndIndex) // [“a”, “b”, “c”]
  items.insert(“d”, at: cachedEndIndex) // [“a”, “b”, “d”, “c”]

  // dedicated `endIndex`
  var items = [“a”, “b”]
  let cachedEndIndex = items.endIndex // this is `.End`
  items.insert(“c”, at: cachedEndIndex) // [“a”, “b”, “c”]
  items.insert(“d”, at: cachedEndIndex) // [“a”, “b”, “c", “d”]

…because the end-index sentinel would *always* refer to the logical end of the collection.

Slices would truly be problematic here. For the rest, it’s hard to see insurmountable difficulties—especially given the ease of converting between the two approaches, given access to the collection—but I’d expect it to be far clunkier and a tad slower.

On the one hand, in my own experience so far, it’s definitely been the
case that most custom collections I’d done have had indices that’re
effectively the `SaferIndex` above; it’s been rather rare that there’s
been a natural “1 past the rest” value to use of the same type as is
used to describe the position of a “good” index.

Seriously, just because Swift has Optionals and they're useful for
safety in some scenarios (compared with allowing everything to be
nullable) does not mean that it's going to be “Swiftier” to apply a
similar pattern everywhere.

use an enum to reintroduce that value when necessary—than to `!`.

I don’t think the above is an *improvement* over the proposal, but it’s a route
that could have been taken.

I believe it would be hard to make such a design work at all, and if you
could make it work I think you'd end up with exactly the problem this
proposal aims to solve: references inside indices. So, I don't think
it's even a possibility, really.

I can’t say I see the impossibility. I definitely have experienced the
clunkiness.

This is getting too involved for a hypothetical I was explaining, but
not advocating.

I will happily agree to drop this topic :-)

It’s dropped, now; I only felt the need to reply one more time b/c I could tell I’d previously failed to communicate clearly-enough.

This proposal and the new design is a good design!

Thanks!

I really do mean it! Just look at those examples and think of how many redundant back-references they used to need...

···

On Apr 14, 2016, at 12:12 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

      To help illustrate the claim, here’s a strawman “safe” API—for
      illustration
      only, not advocacy!—that would be safer and thus perhaps more “Swift-y”:

  I think there's a prevalent misunderstanding (IOW, I don't mean to
  single out this post or this poster) about what “safe” means in Swift
  and what the features of a Swifty API are and should be. This
  is a big topic worthy of much more time than I can devote here, but
  here's a thought to start with:

  A Swifty API helps you reason effectively about the correctness of your
  code, and in part that means we provide enough preconditions on
  arguments to avoid complicating result types, and code to handle
  results, with optional-ness.

  --
  Dave

  _______________________________________________
  swift-evolution mailing list
  swift-evolution@swift.org
  https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Dave

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Not even to mention that
indices are valid only in context of a particular collection instance,
so in this model you could validate an index against one collection
and use it with another one.

The proposal requires Index values to be Comparable. Does that mean that
indices from different collection instances should be comparable i.e. have a
strict total order?

No, comparing indices from unrelated instances produces unspecified
results (incl. traps).

And somewhat related: Is it safe in Swift (in contrast to C) to compare
NativePointers into unrelated memory blocks?

Unfortunately, I don't know, but I'd like to know the answer, too :)
I don't think this is documented anywhere though, so it is technically
unspecified.

Dmitri

···

On Fri, Apr 15, 2016 at 1:30 PM, Stephan Tolksdorf <st@quanttec.com> wrote:

On 2016-04-12 Dmitri Gribenko via swift-evolution wrote:

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

Thanks for your reply!

These kind of type-specific special preconditions for the Comparable protocol probably should be documented more prominently. A similar case is the behaviour of the Comparable implementation for floats when NaNs are involved.

- Stephan

···

On 2016-04-15 Dmitri Gribenko wrote:

On Fri, Apr 15, 2016 at 1:30 PM, Stephan Tolksdorf <st@quanttec.com> wrote:
The proposal requires Index values to be Comparable. Does that mean that
indices from different collection instances should be comparable i.e. have a
strict total order?

No, comparing indices from unrelated instances produces unspecified
results (incl. traps).

And somewhat related: Is it safe in Swift (in contrast to C) to compare
NativePointers into unrelated memory blocks?

Unfortunately, I don't know, but I'd like to know the answer, too :)
I don't think this is documented anywhere though, so it is technically
unspecified.