[Second review] SE-0516: `Iterable`

Evolution Proposal Reviews
1

Hello, Swift community!

The second review of SE-0516: Iterable (previously named BorrowingSequence) begins now and runs through June 18, 2026.

The proposal has been revised to include the following changes:

  • Renamed to Iterable: The protocol and all associated types and methods have been renamed to reflect the protocol's more universal role.
  • Throwing iteration: Both Iterable and IterableIteratorProtocol now include a Failure: Error associated type, enabling typed throws during iteration.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager by email or DM. When contacting the review manager directly, please put "SE-0516" in the subject line.

Trying it out

If you'd like to try this proposal out, you can download a toolchain supporting it for macOS at swift-PR-89630-2329-osx.tar.gz and enable the BorrowingSequence experimental feature flag (the experimental feature flag has not been renamed). Toolchains for the other platforms are currently building on the implementation PR. The toolchain links are also included in the proposal header.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at:

https://github.com/swiftlang/swift-evolution/blob/main/process.md

Thank you,

Holly Borla
Review Manager

13 Likes
2

I'm generally in favour of the proposal. However:

This name is not great. During the pitch phase, I and others suggested a few alternative names. Is there a reason none were considered? Or if they were, can they be included in the alternatives considered section so we can understand the reasoning behind choosing this name?

15 Likes
3

I would be great if this could be answered:

To be more concrete, what performance overhead would we expect for the case where the Iterable returns Span's of just a single element where Failure == Never but Element is not known at compile time? How does it compare to Sequence?

5 Likes
4

Neat idea overall, but there are two aspects I find problematic:

First, the names IterableIteratorProtocol and IterableIterator are simply terrible. For a language that otherwise places significant emphasis on elegance in its syntax and naming, these names are unacceptable.

Secondly, while AsyncSequence clearly defines its end-of-iteration behavior, this proposal appears inconsistent in this regard. The proposal does state, "once an iterator returns an empty Span, every subsequent call to nextSpan() must return an empty Span" which aligns well with the existing {Async}Sequence protocols. However, it then states "An iterator's behavior after throwing an error is undefined" which deviates from the behavior of the only other protocol that allows throwing during iteration. I would expect Iterable to provide the same simple end-of-iteration guarantees as the {Async}Sequence protocols do.

9 Likes
5

What about SpansIterator and SpansIteratorProtocol? My rationale is that, if IteratorProtocol has a next() method, it would make sense for an iterator that has nextSpan() to be prefixed with Spans (or just Span).

6

This should probably say unspecified rather than undefined, since I believe the intention is to give iterators the option of continuing after throwing a recoverable error, rather than requiring iteration to terminate after throwing (as AsyncIterator does), and not to give them the option of invoking nasal demons, as "undefined" would imply.

(I'm not sure how useful this option is in practice, but I believe that's the intention here).

4 Likes
9

As is far too common, there was a lot of great discussion in the pitch thread, almost none of which was either rebutted in the pitch thread, nor is it in evidence in the version of this proposal actually submitted for review. Bypassing community engagement should result in automatic rejection of a proposal.

I still find the rationale for such a complex protocol unconvincing. I'd like to see at least a rebuttal of the "just copy Rust" option, which seems at first glance to be just as optimizable (given SpanIterator is always inlined), whilst also covering all of the "Other forms of iteration" from "Future Directions".

I think the split into Iterable/IterableIteratorProtocol is unfortunate; my impression of the existing Swift protocols is that whilst IteratorProtocol is fine, Sequence/Container/... are regrettable. I (weakly) think this protocol should define only the iterator protocol, and leave how that iterator is obtained from the container unspecified.

Given that this doesn't cover any of the "Other forms of iteration", I think the name Iterable is too generic. How does this naming extend to those other use-cases? { Iterable, ConsumingIterable, MutatingIterable, GeneratingIterable } is pretty nasty, let alone the ambiguity of what exactly a for…in loop does given conformances to more than one of those things (another argument for leaving the container out of the protocols).

I find the maximumCount parameter doesn't meet its design goals. From the perspective of a consumer of this protocol, for…in loops always pass .max and combinators inevitably end up passing 1. Other values must be handled correctly, but will never be specified in practice. But passing .max only does anything for direct for…in over a container; iteration over any kind of combinator chain inevitably decays to 1. I don't think that direct container iteration is so common compared to combinator chains, nor that .max is so much more optimizable than 1 (if SpanIterator is always inlined) to justify the complexity.

Speaking of combinators, I attempted in the pitch thread to implement map for Iterable, and found it difficult-to-impossible. I'd like the basic combinators (map, filter, flatMap, compactMap, reduce, at least, say) to be included in this proposal — if for no other reason than to prove that they are actually possible to write in current Swift. Having another iterator formulation doesn't help much if we can't do anything with it.

  • -1. Please engage with the actual points raised in the pitch thread before submitting the proposal for review.
  • -1. I think this is overly specialized for direct for…in iteration of containers, with insufficient regard for combinators or iteration not rooted in containers.
  • -1. I think this doesn't adequately anticipate non-borrowing requirements for iteration; there seems to be no reason why a single proposal can't cover mutating and consuming iteration as well as borrowing iteration. But even if you reject unifying all modes of iteration, this proposal steps on the possibility space for defining those other modes of iteration.

(edit): I'm also deeply concerned given the proximity to WWDC, that Apple is going to ram this through half-baked for 6.4. I think we need a rule that all proposals for Swift.x must be approved at least y months before the release.

17 Likes
10

For MachO parsers we want recoverable errors in a small number of (very important) cases. In practice 99% of failures should trigger us to stop parsing the file, so non-recoverable errors would be fine. But we also have tools like otool where we need to keep trying to parse as much of file as we can, even after we encounter an error.

Today we have completely distinct parsers for those two use cases, but as we evaluate using Swift to implement a MachO parser in the future we really want one high quality battle hardened implementation with a natural interface that we can use for everything from non-allocating embedded contexts (hence the need for Iterable) up through tools like otool that use the user space Swift stack, so making Iterable support recoverable errors is a very desirable for us.

3 Likes
11

Is there a NonIterableIterator?

At the risk of sounding blunt, these names cannot go to production.

The problem is that it sounds like it's saying the iterator itself is iterable, because in English, if we say "irritable irrator" it means the person doing the irritating is, themselves, irritable. It does not mean that they irritate the irritable. So it's just very confusing.

It goes without saying that an iterator can only ever iterate over iterable things—it is implied in the name. Plus, there aren't any NoniterableIterators.

I might suggest simply Iterator associated type and Iterating protocol—short and sweet.

7 Likes
12

No, but there is an existing Iterator associated type on Sequence which naming this one Iterator would conflict with. It is a goal for the existing collection types to conform to this, one way or another.

There may also need to be refinement iterators that make stronger lifetime guarantees about the Span than IterableIterator can.

1 Like
13

Given the shortcomings of this as a general iteration mechanism, and the apparent desire for something to optimize for…in over containers which have 1 or more buffers of contiguous storage without concern for general iteration patterns, why not:

protocol ContiguousStorageIterator<Element>: ~Copyable, ~Escapable {
    associatedtype Element: ~Copyable
    @_lifetime(copy self)
    mutating func nextSpan() -> Span<Element>?
}

extension Array /* just an extension, no protocol */ {
    @_lifetime(borrow self)
    var contigousStorage: some ContiguousStorageIterator<Element> { get }
}

extension ContiguousArray { /* the same */ }
extension UniqueArray { ... }
extension Span { ... }
extension MutableSpan { ... }

Then anyone who wants to adopt it can do

for await element in container.contiguousStorage { ... }

(element being implicitly borrowed from the span from the iterator from the container)

  • No conflict with existing for…in loops
  • No impact on the future design space
  • Still works for Collections.{ Deque, TreeDictionary, ... } (being the types we have that need to return multiple spans; the stdlib types are all 1-span-ers)
  • No expectation of combinators existing
3 Likes
14

Or even IteratorProtocol, if we want Iterator to be the associated type name in Iterating. I somehow forgot that IteratorProtocol is already the name of the existing protocol. BorrowingIterator might work. I do agree that IterableIterator should not be the final name.

PS. I think the name BorrowingIterator is safe even if we did decide to support an iterator protocol that produces single element references, since the natural design for that is actually:

protocol OwningIterator<Element>: ~Copyable, ~Escapable {
  associatedtype Element: ~Copyable
  mutating func next() -> Element?
}
protocol BorrowingSequence<Element>: ~Copyable {
  associatedtype Element: ~Copyable
  associatedtype Iterator: OwningIterator<Ref<Element>>
  // whatever the lifetime annotation is
  func borrowIterator() -> Iterator {}
}
protocol MutablyBorrowingSequence<Element>: BorrowingSequence, ~Copyable {
  associatedtype MutableIterator: OwningIterator<Mut<Element>>
  // whatever the lifetime annotation is
  func mutablyBorrowIterator() -> MutableIterator {}
}

Actually, wait. We don't even need a new iterator protocol for that, that's just the natural extension of IteratorProtocol to non-copyable types. And a better name for IteratorProtocol to boot.

15

That would prevent using such containers directly with for…in; you’d have to add the method call that produces its iterator. It’s also a roadblock for generic functions that need to iterate containers.

16

Thanks, I see how we got here now.

Seems like we have two issues then:

Issue (1): "IteratingIterableProtocol" name is awkward

It's not the authors' fault at all—they did the best they could in the current situation. It makes sense when you consider how we got here:

  1. Sequence.Iterator's protocol is IteratorProtocol.
  2. So Iterable.Iterator's protocol can't be IteratorProtocol.
  3. But to be consistent, it should have IteratorProtocol in the name.
  4. SpanIterator is already in use elsewhere.
  5. Hence IterableIteratingProtocol.

I would however +1 to @Nobody1707's BorrowingIterator over IterableIterator. Drops the Protocol suffix (which is really just a leftover from the old Generator→Iterator rename, not something a new protocol needs to inherit), and it names the iterator by its ownership relationship.

That said, I think this part is actually far less consequential than issue 2.

Issue (2): Putting associatedtype Iterator on both Sequence and Iterable would cause a conflict

The authors had to call Iterable's associated type "IteratingIterator" because of the underlying limitation that Swift currently lacks a way to disambiguate two associated types that share a type name on a type that conforms to those two protocols. There'd be no way to disambiguate Sequence's Iterator from Iterable's Iterator on a type that conforms to both.

Suggested Solution to Issue 2

Adding support for a new syntax feature might requires a new pitch/proposal cycle on its own, and I don't know what the Apple's timeline looks like (and am not suggesting this necessarily delay their feature). However there is precedent for including syntax features with proposals like this, though it's not common.

Caveats aside, ideally, Swift could be updated to support disambiguating associated types of the same name:

/// Declaration site:
struct FileHandleBag: Sequence, Iterable {
    typealias Sequence.Iterator = FileHandler  
    typealias Iterable.Iterator = SpanReader  
}

/// Call site:
func processFiles<T: Sequence & Iterable>(handler: T) {
    var a = handler.makeIterator()          // fine — method name disambiguates
    let fileListIterator: (handler as Sequence).Iterator = ... 
    let byteSpanIterator: (handler as Iterable).Iterator = ... 
}

If this proposal already considered this option, please forgive me, as I must have misread.

Discussion of this Solution

Adding disambiguation for associated types would allow Iterable to simply use the associated type Iterator, making it more consistent with Sequence. It would also alleviate the need need for the IteratingIteratorAdapter by allowing callers to disambiguate at the call site even if the associated type names are identical.

More importantly, this would also solve the issue permanently for all future Swift authors, unless it's technically impossible (@xwu chime in?).

Seems better overall for the evolution of the language as a whole, and what kind of precedents changes like this set. Do we add confusing names like IteratingIterableProtocol into the standard library or do we make some tweaks to keep Swift elegant?

At the conformance, as mentioned above, that'd look like qualifying which protocol each witness satisfies:

struct FileHandleBag: Sequence, Iterable {
    typealias Sequence.Iterator = FileHandler
    typealias Iterable.Iterator = SpanReader
}

Protocol.AssociatedType on the left of a typealias reads naturally and has a clear meaning — you're already declaring witnesses, this just says which protocol's requirement you're satisfying when the names collide.

Hopefully it's not as hard of a change to implement as it might seem:

The witnesses are already separate at the ABI level. Associated types are stored per-conformance, so Sequence's witness table has its own Iterator slot and Iterable's has its own. Witnessing both with different types needs no new runtime machinery; this is a source-level name-resolution problem, not an ABI one. (Happy to be corrected by someone who knows the mangling better than me.)

The source impact is scoped to genuinely-ambiguous sites only. Existing functions written where T: Sequence that name T.Iterator are unaffected and continue compiling untouched. One protocol supplies Iterator, no ambiguity. An error only fires when both Iterator-supplying protocols are named in the same bound, i.e. code that's opted into both after Iterable releases. Even then, it's a compile error, not a silent gotcah, and since the compiler knows the candidates, it can offer a fix-it that inserts the qualifier.

**EG: Someone who later widens a constraint to T: Sequence & Iterable will need to update existing bare T.Iterator uses wherever the compiler can't disambiguate based on member names. But that's local to a function they're already editing, mechanical, and fix-it-able — a one-time migration for adopters, versus the IterableIterator name everyone reads and writes forever. SE-0503 and SE-0346 both establish that carefully-scoped source breaks are acceptable when there's a migration story, and this one's about as mechanical as it gets.

The capability is the point, not the syntax I suggested. SE-0346 considered explicit associated-type signifiers at use sites (the Collection<.Element == String> form), found them parseable, and explicitly left the door open to adding them later, so there's precedent that this shape of syntax is acceptable. Whether the disambiguator ends up (T as Iterable).Iterator or something else is a side question.

Caveats

I kept the ABI claim hedged ("happy to be corrected") because I am just a humble Swift developer. In all likelihood everything I just suggested was already thought about and ruled out as requiring 5 months of compiler refactoring, or something.

Thanks if you actually read all that.

1 Like
17

The naming here reminds me of the C++ standard library, where newer APIs that intend to supersede older APIs are given names that don't accurately describe the differences between them; for example, std::scoped_lock supersedes std::lock_guard, and std::copyable_function supersedes std::function. To avoid the confusion that can cause, I think the names of these new protocols should describe the differences from the older protocols Sequence and IteratorProtocol, such as by having "borrowing" in the name like in the original revision. Alternatively, the older protocols could be deprecated, which would also help avoid confusion about which protocols to use.

Iterable would have a more familiar name than Sequence to people coming from other languages like Python and JavaScript, even though arguably Iterable is the more advanced of the two.

1 Like
18

? Not sure that I specifically have anything to say about this.

Fundamentally, we need a distinct name for the associated type and its corresponding protocol. It would not make it better if there were some last-ditch disambiguation syntax. We would never want to assign the same name to two distinct things that we can readily anticipate will clash like this.

The sum total of options that I can see are:

  • Naming the new thing after Iterable (i.e., IterableIterator)—as proposed.
  • Naming the new thing after "how it achieves what it is" instead of "what it is" (e.g. SpanIterator)—our general approach is to avoid naming things by the "how" instead of the "what."
  • Naming the new thing with some description of the ways in which "what it is" differs from the old thing—this starts to blur with the previous approach and often ends up being too reductive (thus possibly misleading) and/or too wordy: we want a name and not a description.
  • Naming the new thing Iterator2.
19

Thanks, well that answers the question then.

BTW I wasn't trying to hold up this proposal for a last-ditch change, it's more of a question of if it was a simple change or not, and, after looking deeper, well it seems to require a new kind of grammar in AST RequirementMachine. So yeah, not anytime soon.

That being said, someday it would be nice to bring the ability to disambiguate types with matching names to associated types to be consistent e.g. with:

import AvatarFetcher
import ProfilePicFetcher

class ProfileBuilder {
    var avatarFetcher: AvatarFetcher.Fetcher(...)
    var profilePicFetcher: ProfilePicFetcher.Fetcher(...)
...

Since why should associatedtype be a special exception? But anyway... fodder for another pitch. Thanks

21

Amusingly, one thread over: [Pitch] Associated Type Disambiguation

22

Exactly.

Only if they need to both iterate the container and do other non-iteration-y things with it (otherwise they can simply accept the iterator instead of the container). I don't think I, personally, have ever written such a function.

Generally I think the Container hierarchy of protocols are ill-conceived; it's simply not the case that most functions which can use one type of container are equally valid with any other, and it's not really the case that collections have interchangeable API in any meaningful sense.

23

I'm a little confused. The WWDC video What's new in Swift mentions this proposal as if it has already been accepted and is part of Swift 6.4.

I believe I'm not alone in thinking that, in its current state, this proposal is unacceptable. I'd appreciate some clarification on this.

P.S. I would also appreciate it if the proposal author would take part in the discussion here.

5 Likes