A lazy version of Collection.split

[CTMacUser (Daryle Walker)]

@Ben_Cohen

CTMacUser:

A bigger challenge to work on next: LazySplitSequence . I have to somehow pass on a copy of a partially-expended wrapped-level iterator so the sequence-level iterator and use it for the next subsequence.

My suggestion would be for the lazy sequence split to pull off [Element] chunks.

So all you need to do is stash the base iterator in the lazy splitting iterator, then consume from it into an array that takes the place of the old "subsequence" that it then returns. The iterator will also have to stash one element if coalescing empty regions (because it needs to hunt forward for the first non-empty one, thus consuming the first element of it from the iterator).

This is admittedly no longer quite so lazy (you're producing arrays for the fields) but I don't think you can do it the other way without getting entangled with reference semantics for the iterator.

I was going to do the lazy-sequence-of-lazy-sequences route, but then I considered trying your lazy-sequence-of-eager-sequences idea, and what common code both can use. A starting point would be something like a filter; given an Iterator, a separator closure, and other split arguments, make another iterator that vends just the non-separator elements. Further, include an index for which split sub-sequence that element would be part of. For instance, given

"ABCDEF"

with "D" as a separator. You would get as a return from this splitting iterator:

("A", 1), ("B", 1), ("C", 1), ("E", 2), ("F", 2)

The problem with this kind of return are sequences like:

"DABCDDEFD"

and elect to include empty sub-sequences, giving split-element-index pairs of:

("A", 2), ("B", 2), ("C", 2), ("E", 4), ("F", 4)

You can infer where the purely empty sub-sequences by where the index suddenly jumps. But, unless the iterator includes a property for the count of total sub-sequences after it starts returning nil from next(), you'd miss that there's another empty sub-sequence at index 5. So I had to add place markers for empty sub-sequences:

(Empty, 1), ("A", 2), ("B", 2), ("C", 2), (Empty, 3), ("E", 4), ("F", 4), (Empty, 5)

Both the slightly- and really-lazy split sequences' iterators wrap around this iterator. The version with eager sub-sequences was pretty quick. The reference-link dance needed for the deeply lazy version took some work, but there's some more issues, coming up in a later post.