[Concurrency] AsyncSequence

[Karl]

@jrose

One thing that I find interesting is the presence of both AsyncSequence and AsyncIterator. It’s been discussed that one of the flaws of Sequence is that it’s not really clear (and definitely not enforced) whether conforming types must support multiple passes.

This. Users are supposed to assume that Sequence is single-pass, but it actively tries to confuse them with methods like makeIterator() that can't mutate the Sequence, on top of the fact that IteratorProtocol also exists and is an excellent model of a single-pass Sequence. I wrote about this a while ago, but it bears repeating here:

@Karl

I would draw your attention to this part, however:

Repeated Access

The Sequence protocol makes no requirement on conforming types regarding whether they will be destructively consumed by iteration. As a consequence, don’t assume that multiple for - in loops on a sequence will either resume iteration or restart from the beginning:

for element in sequence {
  if ... some condition { break }
}
for element in sequence {
  // No defined behavior
}

In this case, you cannot assume either that a sequence will be consumable and will resume iteration, or that a sequence is a collection and will restart iteration from the first element. A conforming sequence that is not a collection is allowed to produce an arbitrary sequence of elements in the second for - in loop.

The problem isn't over-specification; it's the opposite. You should always be very cautious when you see the phrase "No defined behaviour".

...

The thing about this documentation is that it is too high-level: to really grasp what this means, you need to go back to the first sentence and remember that this applies to all iteration , and that iteration is the only way to get the Sequence 's elements (every method, like .prefix or .first(where:) , uses iteration, so they are interchangeable with the for loops in the example). That's important because it's quite rare to write multiple consecutive for loops over the same Sequence . If you just read the above example, it might not be clear how this affects you and your code (not you personally, of course - Swift developers in general).

Essentially what this reduces to is that you can perform one and only one read operation on a Sequence , and every subsequent operation returns . Either you call makeIterator() and read via that iterator, or you call something like .first(where:) , but not both . Otherwise the result is not predictable. If you pass the sequence in to a function, you have to assume that it might iterate it unless it is guaranteed not to (even logging/debugging code might iterate the sequence). Similarly, once you've done your read and possibly consumed the sequence (or not - who knows?), you must not pass that Sequence as an argument to another function - otherwise their results also become unpredictable ("A conforming sequence ... is allowed to produce an arbitrary sequence of elements in the second for - in loop.").

All of which leads me to say that Sequence is useless for writing generic algorithms. It might make a convenient entry-point, but all you should do in that function is get an iterator and pass that to your real implementation. Treat them as yucky, slimy things and touch them as little as you possibly can.

You get one shot - one chance to inspect the elements in the sequence, and after that, anything you do is semantically undefined. We don't provide any guidance on what a Sequence should do if somebody tries to iterate it again after consuming it (return an empty iterator? trap? produce random values?).

func myAlgorithm<S>(_ input: S) where S: Sequence, S.Element == Int {
  if input.contains(42) {
    // We have consumed 'input'. Who knows what accessing it again will do?
  }
  // We have consumed 'input'. Who knows what accessing it again will do?
}

None of the Sequence API makes this behaviour clear. Actually, it actively tries to confuse you by having the same operations with the same names as you'd find on a Collection, but with totally different composition semantics. I would bet that at least 3/4 of generic algorithms which accept a Sequence are incorrect - and again, we don't provide any guidance about what might happen in those cases. They likely won't be "unsafe" in the sense of memory safety, but they will produce unpredictable results and possibly lose or corrupt user data.

All of these flaws are inherited by this proposed AsyncSequence design. In fact it actually adds one:

@Tony_Parker

count() async rethrows -> Int

Sequence explicitly does not include a count property, because like all Sequence APIs which inspect the contents, it would consume it. It would tell you how many elements it used to contain, but that's kind of irrelevant because you'll never see that sequence of elements again.

That's not to say that Sequence is entirely useless, though! It's there because if you have an algorithm which really is single-pass, it would be nice for Collections to have access to it without the ugliness of creating an iterator. The downside is that the same method handles both consuming and non-consuming cases, with the same spelling.

Since we don't have an async Collection, it's hard to justify an async Sequence.

EDIT: Updated reason why we have Sequence (I had to look it up!)