[Second review] SE-0516: `Iterable`

I would also like to add that, at least in my experience, most people do not write their own sequence types with custom iterators.

It's also kind of a given in Swift that the absence of an ownership term usually means copying. So the list posted by @michelf makes a lot of sense to me, and I believe developers who have advanced far enough in their Swift journey to write custom sequence types will value the explicitness over a concise term, and will have enough understanding of the ownership model to see the benefits of a borrowing iterator quickly.

5 Likes

I think those names make sense for the iterator, but think we could still keep Iterable:

Copying Borrowing
container Sequence Iterable
iterator IteratorProtocol BorrowingIteratorProtocol
getter makeIterator() makeBorrowingIterator()
advancing next() nextSpan() & skip()

We could keep the more API-consumer-visible name as Iterable, which fits with, as @scanon says, it being

Then have the underlying less-consumer-visible mechanism use the more semantics-focussed Borrowing-prefixed names.

2 Likes

This is the clearest and most straightforward protocol hierarchy I've seen while reading through the review threads. Definitely +1

9 Likes

I think the names IterableIterator and IterableIteratorProtocol are fine. This is an advanced feature, and any time a dev would come over it, they would probably reach out to the documentation, to either understand what it does, or to refresh their memory, regardless of what it will be called.

The actual gains of finding the “perfect” names would be minor.

1 Like

What if we just design APIs that have a chance of informing the developer about its purpose, rather than forcing everyone to documentation? BorrowingSequence at least has the possibility of a developer getting it the first time they read the name. And if, as you say, this is an advanced feature (I'm not so sure), that means we can make names more verbose and descriptive, not less.

I support @michelf's naming suggestions.

11 Likes

Ditto :slight_smile:

The names IterableIterator and IterableIteratorProtocol are, simply put, both outlandish and terrible.

Naming should be optimized for people using a Iterable type (most people) rather than for people writing a Iterable type (comparatively few people, and more knowledgeable in Swift).

In that regard, if this protocol is meant to become how most people perform iteration in Swift, BorrowingSequence wouldn't be a great choice: it breaks progressive disclosure (borrowing and lifetimes are a much more advanced topic than iteration) and suggests that Sequence —and not BorrowingSequence— is the "main" iteration protocol.

On the other hand, Iterable doesn't have any of those issues: it rightly suggests iteration, without introducing any advanced concepts like lifetimes, and instead of sounding more restrictive than Sequence (like BorrowingSequence) it derives its name from something even more generic (conceptually, all sequences can be iterated, but not all things that can be iterated are sequences).

I think this beautifully conveys the goal of making Iterable the protocol of choice to deal with iteration in as many types as possible, in a way that BorrowingSequence absolutely doesn't.

I agree that there's a compromise in naming the iterator IterableIterator and the method makeIterableIterator. Perhaps there's room to change the naming of those two without compromising on the name of the protocol. Personally, this post convinced me that there isn't.

I think the names are fine as proposed, given the constraints.

1 Like

I still have concerns about the proposed throwing behavior here. I strongly believe that the end-of-iteration behavior should be uniform across all types that use the for-in syntax.

As for naming, I like BorrowingSequence.

The "primary" name Iterable clearly indicates an intent to replace Sequence. I just doubt things will go this way. We'll just fall in a "we have two standards" situation.

I think it's preferable to have a common name for all things for-in can work with. That common name currently is "sequence", found both in Sequence and AsyncSequence.

5 Likes

Progressive disclosure does not mean intentionally obfuscating the names of things. If it did, we should be renaming anything that mentions borrow at all. Progressive disclosure is the fact that users can use for in without having to care about how those loops are performed at all, relying on the compiler to choose the optimal method for the usage seen. The names of the sequence protocols are designed for consumption by developers implementing collections or writing generic code, both more advanced usages which would benefit greatly from clear and obvious naming. Nobody writing that code would benefit from the name Iterable.

5 Likes

To that end, SpanSequence wouldn't sound all bad to me.

2 Likes

I hope you can see how that's a strawman.

Regardless, it's not difficult to refute: The Swift Programming Language documentation already mentions the Sequence protocol multiple times in one of its very first sections, Control Flow.

An updated version of this explanation, for instance:

The examples above use a for-in loop to iterate ranges, arrays, dictionaries, and strings. However, you can use this syntax to iterate any collection, including your own classes and collection types, as long as those types conform to the Sequence protocol.

Surely would benefit from this new protocol being called Iterable instead of BorrowingSequence.

It shouldn't be controversial to say that mentioning the name of the most general iteration-providing protocol in Swift is valuable when explaining how iteration works, and having to mention borrows in order to do it, isn't.

That's not to say that BorrowingSequence wouldn't have other merits. But aiding progressive disclosure isn't one of them.

SpanSequence is a good name too. It's a sequence (meaning it works with for-in) and it literally works by iterating over spans of elements. It's even more descriptive than "borrowing" (which is implied when you know about Span). :+1:

1 Like

It isn’t possible to abstract over totally different ownership behaviors, so unification isn’t a consideration.

Is this not what Ref<T> is for? consuming a Ref<T> is equivalent to borrowing a T? Rust literally does this (Iterable supports both borrowing and consuming iteration & as discussed in the pitch thread, it seems like Swift's attachment of lifetimes to methods instead of types means that we could get "lending" iteration with the same protocol too).

for x in y where x is Ref<T> would be slightly unergonomic for now, but we're well aware there's a missing SE for turning Ref<T> into borrow bindings. I don't think that's a consideration this SE should be concerned with.

3 Likes

Rust does this by modeling ownership as a part of the type system and thus something that generic types can abstract over. Swift doesn’t model ownership that way, so it can’t.

1 Like

It's not. At worst it's a hyperbolic reductio ad absurdum. But really it's just an example of the designs we should see in Swift if "progressive disclosure" meant what you assert. Your examples are more about succinctness than managing complexity. In fact, given what we know today, names like Sequence and Collection would likely be slightly more verbose to better differentiate them from the various types we want now.

Not really, no. You'd simply be trading Sequence for BorrowingSequence. To anyone not already familiar with Swift's protocol hierarchy, they're both rather opaque. In fact, I'd say Sequence is slightly more descriptive than Iterable, given Sequence at least has a common English counterpart. Iterable is practically jargon.

If we're designing a language with multiple types of iteration under for in or other loops, it behooves us to be able to distinguish them quickly and easily. What's the difference between Sequence and Iterable? You'll have to read some docs to find out! Protocols like RangeReplaceableCollection make it much more obvious how they're useful, and without having to read documentation.

Have to disagree. Given the name isn't disclosed until the developer has to work with it, a more precise name, if it isn't too verbose, is always preferable. Put another way, since we'd interact with these collections using common looping constructs and not direct iterator access, there is no disclosure to progress in the first place. Only once you get into generic collection operators or writing your own collections does that begin to matter, and in those contexts, making the semantics clear with a slight increase is verbosity seems well worth it.

3 Likes

Rust does this by modeling ownership as a part of the type system and thus something that generic types can abstract over. Swift doesn’t model ownership that way, so it can’t.

Sorry, but I still don't understand this statement. Here's my understanding:

This is a tradeoff that seems to swing in Swift's favor here — we can say what the lifetimes are for returned borrows, without any necessity that they apply in general to Escapable types.

Rust has:

pub trait Iterator {
    type Item;
    fn next(&mut self) -> Option<Self::Item>;
}

Roughly in Swift:

public protocol Iterator: ~Copyable, ~Escapable {
    associatedtype Item: ~Copyable, ~Escapable
    @_lifetime(copy self) // justified below
    mutating func next() -> Item?
}

Consuming Iteration

A consuming iterator given these definitions is simple:

struct MyIterator { ... }
impl Iterator for MyIterator {
    type Item = Whatever;
    fn next(&mut self) -> Option<Whatever> { ... }
}
struct MyIterator: ~Copyable, ~Escapable, Iterator {
    mutating func next() -> Whatever? { ... }
}

Lifetimes don't apply in either language: in Rust you simply don't have any, in Swift they're irrelevant and ignored so long as Whatever is Escapable. This compiles in Swift 6.4.

Borrowing Iteration

In Rust, if you want to write a borrowing iterator, you have to write type Item = &'a T, where a is some generic lifetime parameter. The only place that lifetime parameter can live is on your iterator type, so a borrowing iterator kinda inherently looks like:

struct MyIterator<'a> { ... }
impl<'a> Iterator for MyIterator<'a> {
    type Item = &'a Whatever;
    fn next(&mut self) -> Option<&'a Whatever> { ... }
}

In Swift Ref takes on the lifetime declared on the method that returns it, so given the above translation of Iterator, you could write

struct MyIterator: ~Copyable, ~Escapable, Iterator {
    typealias Item = Ref<Whatever>
    @_lifetime(copy self) // most equivalent to the Rust code, but see below
    mutating func next() -> Ref<Whatever> { ... }
}

The lifetime is now relevant, and has the same restriction as Rust — it must match whatever the Iterator was borrowed from. The same protocol supports borrowing iteration as well as consuming iteration. I can't test this in Swift 6.4 yet because Ref isn't in Xcode 27b1, but I believe it will compile.

Lending Iteration

That means you can't create a "lending" iterator, where the iterator itself has some internal storage that next returns a reference to. But in Swift (and unlike Rust, without getting into Generic Associated Types) we can further relax the protocol:

public protocol Iterator: ~Copyable, ~Escapable {
    associatedtype Item: ~Copyable, ~Escapable
    @_lifetime(borrow self)
    mutating func next() -> Item?
}

Now the returned item can have a shorter lifetime than the iterator. The slight tradeoff is that now the reference to the item can't outlive the iterator, even when that would've been valid (it's borrowed from the container). So it's not a complete home-run, but a deliberate choice we can make in either direction.

1 Like

Saying "I think it should be possible to explain iteration in Swift without mentioning lifetimes" doesn't mean "I think we should never ever use the word borrow in a name", nor is the latter a consequence of the former.

I believe (not assert) that progressive disclosure means layering the information as to not overwhelm the programmer with information about more advanced features of the language right away. I also believe that 'Iteration' (and even 'Generics') are a few layers away from 'Ownership' .

Naming the protocols Sequence and BorrowingSequence would certainly highlight a difference between the two protocols: it uses a borrowing iterator. I don't think that's the main takeaway of what the Iterable type is fundamentally meant to be, though. I believe its most important property is that it extends iteration to other types (~Copyable, ~Escapable...) that couldn't adopt it before, becoming the most general iteration-providing protocol.

BorrowingSequence doesn't convey that. If anything, it sounds more restrictive than Sequence by adding a qualifier (rightly so, there's significant precedent in Swift of using the single-word name to signal what the "default" type is meant to be[1], e.g. Array vs ContiguousArray).


I'll refrain from making further comments on Iterable vs BorrowingSequence, I think I've given my point of view already and I'd hate to take the attention away from other topics (speaking of which, I hope this doesn't get lost, the maximumCount → maxCount and skip(by:) → advance(by:) —or similar— are no-brainers!).


  1. And it seems to me like Iterable is meant to be the new default. It goes as far as to suppress Copyable and Escapable by default from the associated type so you can write simple generic code on a Iterable parameter. ↩︎

1 Like

”you can use this syntax to iterate any collection, including your own classes and collection types, as long as those types conform to the Sequence protocol or the BorrowingSequence protocol.”

”you can use this syntax to iterate any collection, including your own classes and collection types, as long as those types conform to the Sequence protocol or the Iterable protocol.”

One of these spells out the difference between the two protocols and suggests why you might want to use one over the other. The other one does not. In fact it only creates more questions.

What is the relationship between the two protocols? If one is iterable, does that mean the other one isn’t? If one is a sequence, does that mean the other one isn’t? If I want a type to both be iterable and be used as a sequence, do I need to conform to both? Etc.

5 Likes

maximumCount → maxCount: sorry, this actually should have been in the revised proposal, as it should match the precedent elsewhere. We'll update.

skip(by:) → advance(by:) I don't think is truly a no brainer. Advancing an index is conceptually different to skipping over elements during iteration IMO. Especially in cases where skipping those elements might do real work (say if you were unfolding a sequence or running an expensive predicate) – so I have a preference to keep it as is. I don't feel like consistency with the indexing APIs is a goal here.