Critic notes about Swift Range types jungle

It would really kill you to link some prior art for people just entering the topic? I bet you're real fun at happy hour.

1 Like

This is not appropriate for these forums. You are welcome to ask for help if youā€™re unable to find something, but not in this way.

8 Likes

I mean I did ask for help and you said that's not your job. Dunno what else to say.

1 Like

@dcowuno didn't this already get linked? Unify and expand Range and its likes - #4 by beccadax

@Lantua thank you for the link. Some interesting examples there.

Shouldn't this work?

let a = (UInt8(1)...).min() // āŒ Runtime crash Thread 1: EXC_BREAKPOINT (code=1, subcode=0x1aa7d82d0)

NaĆÆve thinking suggests that this should be the case:

let x: PartialRangeFrom<Int> = 1...
let z: PartialRangeFrom<Int> = x.reverse().reverse() // šŸ›‘
1 Like

I think on Sequence, min() iterates over all elements before finding the value.

for i in UInt8(250)... {
    print(i)
}

This prints numbers between 250 and 254 (no 255) and crashes.
I guess this works "as designed"... looks weird still.
I.e. the only way iterating throw the unbound range could work is by crashing when I go past the maximum value.

PartialRangeFrom.Iterator is frozen, and it only stores a non-optional value, so I think using Bound._step(after:from:by:) isn't possible?

Yep, that discussion was quite fruitful. It ended abruptly leaving some unanswered questions, e.g. this.

Not as-is, but it is at minimum a bug that the iterator never allows 255 to be printed, and there are measures that can be taken to supersede PartialRangeFrom with a new mangled name while keeping the old one around with the original mangled name.

cc @scanon

4 Likes

This position isn't wrong, but it's also not helpful. It comes across as gatekeeping: "don't bother participating unless you're willing to put in hours or days of work digging through ancient threads that may or may not actually be relevant anymore just so you can know about some particular piece of nuance that was discussed five years ago."

It seems like there's an opportunity here (perhaps with the help of the @swift-website-workgroup?) to compile summaries of pitch discussions that can be easily referred to.

Otherwise we're putting up a nigh-insurmountable barrier and effectively prohibiting anyone from participating in the forums and evolution of the language unless they have copious amounts of free time or an enormous amount of pre-existing knowledge, and that will only get worse the older Swift gets.

26 Likes

On the contrary, the purpose is to lower the barrier for participation, particularly for those who are new. If discussions are launched with no context, then itā€™s everyone else who wishes to knowledgeably participate who has to do this study for themselves, and to repeat themselves ad nauseam whenever the same topic comes up. No, we have always said here that the expectation is that the one person who wants to stimulate a discussion do the legwork of catching everyone up on the key points of whatā€™s already been said.

Participate, no. Kick off a discussion, yes. If hours of work are unreasonable, the implication is that minutes of study in a topic with five years of discussion history is sufficient. In my view at least, that is supremely disrespectful and misguidedā€”how can anyone suppose that a thought which popped into their head in minutes would be so insanely great that itā€™s a meaningful contribution to a topic that others have reckoned with for years?

A well done pitch often reflects not hours or days, but weeks or months of effort. It is not required, however, to pitch anything to participate meaningfully in these forumsā€”I myself havenā€™t done so in multiple years.

2 Likes

Here the "critique" was mainly from the "user" point of view (if you feel this topic belongs more to "use Swift" - fine, we can move it there). As a user I don't need to do a deep research scanning through old, half-obsolete discussions and proposals. Instead, after a relatively shallow search (done on this forum or on Google) I go with these three devices: documentation, intuition and questioning those in the know. Documentation like this is obviously lacking (sometimes looking through the headers / swift interfaces reveals more, and as for the intuition the concept of range is totally lost on me. Not one thing but half a dozen? "Countable" - is it the same as "countable" from math (where natural numbers / rational numbers are countable), and then is Double (being of a finite bit size) actually modelling rational or all "real" numbers, and (being of a finite bit size) is it "countable" (which it should be from the math pov)? Partial ranges - what does it even mean for a type like UInt8 which is bound to 255 on one side and 0 on another? Or even a partial range made of Integer / Double which are also bound, just to a higher min / max values. What my API should use, is it RangeExpression to be liberal to what users might want? If I want to support bound ranges only (Range / ClosedRange) what my API should look like? You telling me 1...9 and 1..<10 are incompatible types and I'd need two functions to support both? and that reverse sometimes is fast, sometimes is slow and sometimes is crashing? This type system is frozen, will never change, perhaps to the point it is worthless even discussing it, just note that it totally NOT user friendly type system to use.

4 Likes

In that case, the opening as is may be fine, but IMO this thread could then use some re-framing.

Aside from the fact that this is posted on Evolution (people use wrong tags all the time), it is proposing a change, a pitch/proposal, and I have been treating it as such. In which case, digging into past discussion has been the (de facto?) bare minimum asked of the proposer so that everyone new and old to the topic is brought up to speed.

5 Likes

Anyway, on that note about prior discussions.

There's this very detailed and iterated thread of an idea similar to this one (personally, I'd suggest the entire thread from v1 due to the myriad of good thoughts therein):

This relatively light thread:

These very old threads about combining Range & ClosedRange:


Tangentially related thread about sugar for index offsetting:

8 Likes

Very weirdly, this is behaving as documented:

The behavior of incrementing indefinitely is determined by the type of Bound. For example, iterating over an instance of PartialRangeFrom<Int> traps when the sequence's next value would be above Int.max.

I don't think that this behavior is great, but on a quick glance the options for "fixing" it are pretty tightly constrained by ABI stability. I'll have a think about what we might be able to do here.

5 Likes

Okay, as a moderator here, I'd like to course-correct this thread a little and lay a few things out for future threads.

This topic is definitely something that's been talked about a lot before. It's okay to start a new thread that builds on those conversations; in fact, we'd rather people do that than resuming a long-idle thread. We want to make sure we're building on those past conversations, though, instead of repeating a lot of stuff that's been said before. The best way to do that is for the original poster (the "OP") to explicitly link and refer back to previous discussions.

If that hasn't happened, it's okay for other posters to refer the OP to some things they might want to read. This should always been done in a welcoming and constructive way, and that almost certainly means linking some of the proposals and/or threads you have in mind, as well as avoiding "moderator voice" or lecturing the OP about what they need to do. A simple post like "We talked a lot about this in the pitch for SE-0172" will usually do. Being welcoming and constructive is particularly important if the OP is a new user (which of course @tera is not), but it's expected regardless.

With that said, @tera, I think it would help to consider what your goal for this thread is. Your original post definitely reads like an early pitch for an alternative ranges design centered around a unified, currency range type, and you've been here long enough to know about our expectations around such threads. If this is meant to be a request to improve the docs, I think it would help to make it a clear and constructive suggestion. If it's just an unconstructive gripe about the API, I mean, I'm not sure we need that here.

15 Likes

Oof, that documentation is subtle but technically accurateā€¦

Back to the original topic of Range types being insufficient, I recently had to make my own type to cover all these cases because I wanted a succinct way of expressing ranges in a variety of cases. The only benefit that I see however for these types being part of the standard library are two fold:

  • For library authors to use as a currency type for describing a range in an abstract way divorced from its current use in Collections.
  • Light compiler support for verifying the user doesnā€™t construct invalid ranges through static analysis.

I think RangeExpression could likely have more expressivity to reach most of these goals today however, and a future version of swift could consider a new syntax for abstract ranges that is potentially closer to their mathematical representation? (ie. for a in [0;10[, [;10] for an unbounded start, [;] for an all-inclusive range, etcā€¦)

To go back to the original question postedā€¦

Sure. Itā€™s always preferable to have a single concrete type instead of multiple types unified by a protocol. Whenever you donā€™t, the reason is usually combination of performance and expressivity

For example, why have both Array and Dequeue? Because there are things dequeue can do performantly that a swift array canā€™t, and vice versa. This mostly is a question of performance, but there are expressivity cases too, for example, itā€™s useful to have both a sorted and unordered set type, because not everything type can be put into a sorted order.

So with this in mind, how does your suggested range shape up? The most important question is, would it be possible to replace the existing range type with your far more complex type that tracked the openness of either end. Bear in mind the performance of ranges is paramount for carrying out various operations efficiently, like iteration and slicing. The current range type can optimize very efficiently in many cases to a few instructions. It wouldnā€™t be too hard to try out an (ABI-breaking)[1] change and see just how big the impact on the benchmark suite is, for some early signal. Optimizer heroics might enable similar performance to that with your more complex type, but optimizer heroics are very brittle. You can sometimes mitigate the performance impact through other clever tricks in the implementation, but this tends to get hard quickly too (or leak out into the public API).

Several of your points go to the expressivity issues from unifying the type:

Is 1 ..< 9 equal to 1 ... 10 [sic]? Practically speaking yes, but you can't compare them and their hashValues are different.

Having a single unifying type will not solve this problem on its own, because the bounds of ranges arenā€™t always strideable. So 1..<10 might be equivalent to 1ā€¦9 but in the general case you cannot confirm whether 1..<i is equal to 1ā€¦j because you cannot determine succ(j) == i.

Sometimes the answer here is ā€œgive them conditional conformance where Bound: Strideableā€ but this then has weird effects where, unless you constrain to that everywhere, you lose the ability to equate these range types even when they have equatable bounds.

Is "1..." equal to "1 ... .max" - again, for practical purposes yes

Again, youā€™re assuming a bounds type that has a maximum value. For Int it does but for some bignum type, this isnā€™t the case.

Also, even with Int, my take would be that the claim that the two are equal is incorrect, especially if you start considering some maybe future where there are implicit one-way conversions between e.g. Int8 and Int16.

So to preserve the current status quo, this Ć¼ber type really does need to capture this difference. You imply this with your .inf sigil but what is the actual implementation mechanism for this given this isnā€™t actually a value on every possible Bound type (nor is .max btw). Youā€™d need to add more data to the range type, loading on more potential performance challenges.

As to being able to express open/closed on the left-hand-sideā€¦ Iā€™m pretty skeptical of the pressing need for this. Maybe it would be useful? But Iā€™d want to see a lot of real world example use cases proving that need. Rounding out the types for neatnesses sake is not enough.

Re unbounded range.. yeah, itā€™s a weird one. Solving this would be nice, but the current generics system, and the (somewhat hacky) way it was implemented, probably make fixing it to be a true range expression prohibitively costly. And again, this might run up against the question of performance trade-offs.

Finally, Iā€™d also suggest doing a comparative study of other languages like Java (which I donā€™t think has a Range of its own? But Guava has one), Rust, Ruby, Python, Kotlin. Rust for example has a similar design to Swiftā€™s, but with more problems (some of which are to do with copyability, something we can look forward to having similar hard choices over). Iā€™ll note that in my spot checks it didnā€™t seem like many solutions catered to open/closed on the lhs which reinforces my priors about that being unimportant to address.


  1. and of course, the ABI challenges of changing any of this would be considerable, so anyone embarking on that kind of experiment would need to be ok with it being wasted effort if those end up being showstoppers ā†©ļøŽ

12 Likes