[Proposal] Change guarantee for GeneratorType.next() to always return nil past end

I know this is veering off-topic, but I don't know if that can
fix this issue in Swift. I very much doubt we're going to want to
create a world that has non-copyable types, and that's really what you
need to make this issue go away. Or, you can decree that every
generator is a reference type.

···

on Sat Mar 05 2016, Brent Royal-Gordon <swift-evolution@swift.org> wrote:

In fact, I don't think the post-nil behavior of generators is even
the biggest potential gotcha. A much bigger issue is how generators
behave when you make copies of them. If you ever copy a generator
you're supposed to never touch the original again, but it's easy to
violate that accidentally, and most generators actually behave
sanely when copied anyway.

That's true, but it's just not addressable until we have some kind of
borrowing system in place.

--
-Dave

Anyone who uses Generators directly has to understand their behavior and has to understand their requirements. The prohibition on not copying a Generator and invoking next() on both copies I think is actually more of a potential problem than handling post-nil behavior. In my personal experience, I've accidentally violated the copy prohibition before, but I've never even had to think about post-nil behavior except when implementing my own Generator. And allowing post-nil behavior to be implementation-defined is potentially useful behavior. In addition, even if the optimizer can get rid of the cost of tracking this state (and even with the limitation removed I don't think you can assume that will necessarily be the case, especially once you end up with an AnyGenerator wrapper in the mix), it's still extra burden on people implementing new Generators for something that very few clients are even going to care about. If we define post-nil behavior as needing to return nil forever, I suspect it's actually more likely for implementors of new Generators to screw up and not enforce this requirement than it is for clients to even notice it.

As a side note, I just tested (using Xcode 7.3 beta 5) and AnyGenerator doesn't currently track this state itself. You can easily pass a closure to AnyGenerator() that causes it to return non-nil values after a previous call to next() has returned nil. Trivial example:

var base = someSeq.generate()
return AnyGenerator {
    guard let next = base.next() else { return nil }
    return next % 3 == 0 ? nil : next * 2
}

With this example, if `someSeq` is `1..<10`, the resulting elements from the generator are [2, 4, nil, 8, 10, nil, 14 , 16, nil, nil, ...]. And of course the whole point of AnyGenerator is to hide the implementation details from the caller, so if state is added to AnyGenerator to track this, it definitely won't get optimized away. And if the AnyGenerator is used to wrap an underlying generator, this state tracking is very likely to end up being redundantly performed in both AnyGenerator and the underlying generator.

-Kevin Ballard

···

On Tue, Mar 8, 2016, at 11:42 AM, Dmitri Gribenko wrote:

On Sun, Mar 6, 2016 at 5:58 PM, Kevin Ballard via swift-evolution > <swift-evolution@swift.org> wrote:
> That's a fair point. But I think the important sentence from my comparison to Rust is "And in practice, almost nobody ever has to actually use .fuse(), …".

The concern is that people who do need to invoke .fuse(), won't,
because all generators they are likely to try in practice will 'just
work' and return a continuous stream of nils.

I think what this really comes down to is the trade off between a
subtle correctness issue and a small performance win for a small
number of data types (which can be even non-existent as Patrick
Pijnappel shows). Given the general direction of Swift, I'm inclined
to choose correctness here.

I think it will be far easier to write generic tests of any generator than it will be to write generic tests of any function that *uses* a generic generator.

(Although I suppose you could assist in that testing by writing a wrapper around any sequence/collection/generator which enforces all rules as strictly as possible.)

You could always just test the behavior of the FuseGenerator wrapping it, if you really want to be able to rely on always-returning-nil behavior. But I'm not sure what tests you'd write that even hits this case, except for a test that's designed specifically to test the post-nil behavior (which we've already established isn't necessary to check for arbitrary generators if GeneratorType leaves that as implementation-defined).

The fuse generator wouldn't do what I'm talking about. What I mean is that it might be useful for testing purposes to be able to wrap a collection/sequence in a StrictCollection/StrictSequence (with corresponding StrictGenerators, StrictIndexes,etc.) which would do things like:

* Detect if you copy a generator and then use both copies
* Detect if you call next() on a generator after it returns nil
* StrictCollection only: Invalidate all indices retrieved before any mutation
* StrictSequence only: Prevent you from making a generator more than once

The idea is that, when you're writing a piece of code that's generic on any collection or sequence type, your unit tests could pass in a StrictCollection/StrictSequence-wrapped version of whatever your actual data is, and if your code doesn't crash you can be pretty sure it's not violating any of those protocols' unevenly enforced requirements.

···

--
Brent Royal-Gordon
Architechies

> That's a fair point. But I think the important sentence from my comparison to Rust is "And in practice, almost nobody ever has to actually use .fuse(), …".

The concern is that people who do need to invoke .fuse(), won't,
because all generators they are likely to try in practice will 'just
work' and return a continuous stream of nils.

I think what this really comes down to is the trade off between a
subtle correctness issue and a small performance win for a small
number of data types (which can be even non-existent as Patrick
Pijnappel shows). Given the general direction of Swift, I'm inclined
to choose correctness here.

Anyone who uses Generators directly has to understand their behavior and has to understand their requirements. The prohibition on not copying a Generator and invoking next() on both copies I think is actually more of a potential problem than handling post-nil behavior.

And that is another problem that we need to fix. I don't think it is
acceptable to say that we won't fix one problem because there is
another anyway.

If we define post-nil behavior as needing to return nil forever, I suspect it's actually more likely for implementors of new Generators to screw up and not enforce this requirement than it is for clients to even notice it.

I disagree. Most people just use the default IndexingGenerator.

As a side note, I just tested (using Xcode 7.3 beta 5) and AnyGenerator doesn't currently track this state itself. You can easily pass a closure to AnyGenerator() that causes it to return non-nil values after a previous call to next() has returned nil.

AnyGenerator is used to abstract away implementation details, but it
can't fix broken implementations. Any time you conform to a protocol,
you are making a statement about semantics. If the semantics are not
implemented correctly, then there are no guarantees about correctness
for things that build on top.

Dmitri

···

On Tue, Mar 8, 2016 at 12:16 PM, Kevin Ballard <kevin@sb.org> wrote:

On Tue, Mar 8, 2016, at 11:42 AM, Dmitri Gribenko wrote:

On Sun, Mar 6, 2016 at 5:58 PM, Kevin Ballard via swift-evolution >> <swift-evolution@swift.org> wrote:

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

If you're working with a generic collection/sequence, then it's still perfectly reasonable to detect if you call next() on a generator after it returns nil regardless of how the post-nil behavior is defined. After all, if you're working with a generic sequence/generator, and the post-nil behavior is implementation-defined, then your generic code probably doesn't want to invoke post-nil behavior because it doesn't know what will happen. Defining post-nil behavior as always returning nil simply lets you skip this check in your StrictGenerator, but having post-nil behavior being implementation-defined certainly doesn't make testing this any harder.

And if you want to use this same StrictSequence/StrictGenerator when you're using a sequence/generator that you explicitly know defines post-nil behavior as something useful (or if you're writing a sequence/generator adaptor where your adaptor's post-nil behavior naturally matches the underlying generator's, and you want to test your code with clients that may be relying on specific post-nil behavior) then you can parameterize your StrictSequence/StrictGenerator with a flag that tells it whether to check the post-nil behavior. In fact, you may want to do that anyway to enable disabling of any of the checks, because it's certainly possible for a concrete SequenceType/GeneratorType implementation to relax any of these requirements you're checking. For example, a specific GeneratorType implementation could explicitly document that copying the generator is safe and both copies then yield the same sequence of elements.

-Kevin Ballard

···

On Sat, Mar 5, 2016, at 10:35 PM, Brent Royal-Gordon wrote:

>> I think it will be far easier to write generic tests of any generator than it will be to write generic tests of any function that *uses* a generic generator.
>>
>> (Although I suppose you could assist in that testing by writing a wrapper around any sequence/collection/generator which enforces all rules as strictly as possible.)
>
> You could always just test the behavior of the FuseGenerator wrapping it, if you really want to be able to rely on always-returning-nil behavior. But I'm not sure what tests you'd write that even hits this case, except for a test that's designed specifically to test the post-nil behavior (which we've already established isn't necessary to check for arbitrary generators if GeneratorType leaves that as implementation-defined).

The fuse generator wouldn't do what I'm talking about. What I mean is that it might be useful for testing purposes to be able to wrap a collection/sequence in a StrictCollection/StrictSequence (with corresponding StrictGenerators, StrictIndexes,etc.) which would do things like:

* Detect if you copy a generator and then use both copies
* Detect if you call next() on a generator after it returns nil
* StrictCollection only: Invalidate all indices retrieved before any mutation
* StrictSequence only: Prevent you from making a generator more than once

The idea is that, when you're writing a piece of code that's generic on any collection or sequence type, your unit tests could pass in a StrictCollection/StrictSequence-wrapped version of whatever your actual data is, and if your code doesn't crash you can be pretty sure it's not violating any of those protocols' unevenly enforced requirements.

My intuition says the extra state & branch needed for generators like
TakeWhile could very well be optimized away in most cases if you dont
make use of its post-nil behavior. Say TakeWhile is implemented as such:

if done { return nil }
guard let element = base.next() where predicate(element) else {
  done = true
  return nil
}
return element

If the generator is then used in the common case:

let generator = TakeWhileGenerator(...)
while let element = generator.next() {
  foo(element)
}

Should give us effectively:

var base = ...
let predicate = ...
var done = false
while true {
  if done { break }
  guard let element = base.next() where predicate(element) else {
    done = true
    break
  }
  foo(element)
}

The optimizer should see (or at least could see) `done` is never read after
it's written to (thus removing the assignment), and therefore
when checking the condition it can only be false (thus it can also be
removed).

I don't have a machine to test this on atm though.

···

On Sunday, 6 March 2016, Brent Royal-Gordon via swift-evolution < swift-evolution@swift.org> wrote:

>> I think it will be far easier to write generic tests of any generator
than it will be to write generic tests of any function that *uses* a
generic generator.
>>
>> (Although I suppose you could assist in that testing by writing a
wrapper around any sequence/collection/generator which enforces all rules
as strictly as possible.)
>
> You could always just test the behavior of the FuseGenerator wrapping
it, if you really want to be able to rely on always-returning-nil behavior.
But I'm not sure what tests you'd write that even hits this case, except
for a test that's designed specifically to test the post-nil behavior
(which we've already established isn't necessary to check for arbitrary
generators if GeneratorType leaves that as implementation-defined).

The fuse generator wouldn't do what I'm talking about. What I mean is that
it might be useful for testing purposes to be able to wrap a
collection/sequence in a StrictCollection/StrictSequence (with
corresponding StrictGenerators, StrictIndexes,etc.) which would do things
like:

* Detect if you copy a generator and then use both copies
* Detect if you call next() on a generator after it returns nil
* StrictCollection only: Invalidate all indices retrieved before any
mutation
* StrictSequence only: Prevent you from making a generator more than once

The idea is that, when you're writing a piece of code that's generic on
any collection or sequence type, your unit tests could pass in a
StrictCollection/StrictSequence-wrapped version of whatever your actual
data is, and if your code doesn't crash you can be pretty sure it's not
violating any of those protocols' unevenly enforced requirements.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <javascript:;>
https://lists.swift.org/mailman/listinfo/swift-evolution

>> > That's a fair point. But I think the important sentence from my comparison to Rust is "And in practice, almost nobody ever has to actually use .fuse(), …".
>>
>> The concern is that people who do need to invoke .fuse(), won't,
>> because all generators they are likely to try in practice will 'just
>> work' and return a continuous stream of nils.
>>
>> I think what this really comes down to is the trade off between a
>> subtle correctness issue and a small performance win for a small
>> number of data types (which can be even non-existent as Patrick
>> Pijnappel shows). Given the general direction of Swift, I'm inclined
>> to choose correctness here.
>
> Anyone who uses Generators directly has to understand their behavior and has to understand their requirements. The prohibition on not copying a Generator and invoking next() on both copies I think is actually more of a potential problem than handling post-nil behavior.

And that is another problem that we need to fix. I don't think it is
acceptable to say that we won't fix one problem because there is
another anyway.

> If we define post-nil behavior as needing to return nil forever, I suspect it's actually more likely for implementors of new Generators to screw up and not enforce this requirement than it is for clients to even notice it.

I disagree. Most people just use the default IndexingGenerator.

And most people never even try to invoke next() on a Generator after it's returned nil.

I have yet to see anyone actually offer any evidence to show that this is a real problem people are hitting.

> As a side note, I just tested (using Xcode 7.3 beta 5) and AnyGenerator doesn't currently track this state itself. You can easily pass a closure to AnyGenerator() that causes it to return non-nil values after a previous call to next() has returned nil.

AnyGenerator is used to abstract away implementation details, but it
can't fix broken implementations. Any time you conform to a protocol,
you are making a statement about semantics. If the semantics are not
implemented correctly, then there are no guarantees about correctness
for things that build on top.

The point was that it's pretty easy to accidentally violate the post-nil requirement using AnyGenerator. People writing new GeneratorType implementations should hopefully pay particular attention to the requirements, but people creating ad-hoc generators with AnyGenerator are more likely to not realize that what they've written can violate the post-nil requirement. And in fact since most users of Generators don't even attempt to message a Generator after it's returned nil, they probably won't notice the problem, until such time as they happen to pass it to code that does make assumptions about the post-nil behavior of Generators.

Which is to say, if I'm writing code that accepts arbitrary Generators and wants to be resilient, and if I want to be able to message a generator after it returns nil (and get nil again), I may end up using something like FuseGenerator anyway so that way my Generator works even when the input Generator was written sloppily.

Another way to put this is the prohibition against using copies of generators is something that clients are responsible for, but defining post-nil behavior as always returning nil is something that the GeneratorType implementation itself must handle. In fact, it's the only requirement not enforced by the type system that a GeneratorType implementation must conform to. And since most clients don't even invoke post-nil behavior, it's likely that people will violate this requirement without realizing it.

-Kevin Ballard

···

On Tue, Mar 8, 2016, at 04:41 PM, Dmitri Gribenko wrote:

On Tue, Mar 8, 2016 at 12:16 PM, Kevin Ballard <kevin@sb.org> wrote:
> On Tue, Mar 8, 2016, at 11:42 AM, Dmitri Gribenko wrote:
>> On Sun, Mar 6, 2016 at 5:58 PM, Kevin Ballard via swift-evolution > >> <swift-evolution@swift.org> wrote:

This may be doable for stdlib generators, but Swift currently has a
limitation where generic types/functions defined outside the current
file (or module with -whole-module-optimization) are only specialized
for their parameters if they come from the stdlib. Any such
types/functions defined in third-party libraries (or in other files if
you're not using -whole-module-optimization) use virtual dispatch on the
generic parameters instead. This means that any generic GeneratorTypes
defined in third-party libraries won't be able to optimize away this
check as it won't be able to tell that it isn't needed.

-Kevin Ballard

···

On Sun, Mar 6, 2016, at 05:46 AM, Patrick Pijnappel wrote:

My intuition says the extra state & branch needed for generators like TakeWhile could very well be optimized away in most cases if you dont make use of its post-nil behavior. Say TakeWhile is implemented as such:

if done { return nil } guard let element = base.next() where
predicate(element) else { done = true return nil } return element

If the generator is then used in the common case:

let generator = TakeWhileGenerator(...) while let element =
generator.next() { foo(element) }

Should give us effectively:

var base = ... let predicate = ... var done = false while true { if
done { break } guard let element = base.next() where
predicate(element) else { done = true break } foo(element) }

The optimizer should see (or at least could see) `done` is never read
after it's written to (thus removing the assignment), and therefore
when checking the condition it can only be false (thus it can also be
removed).

That's a good point. Kevin, what do you think?

···

on Sun Mar 06 2016, Patrick Pijnappel <swift-evolution@swift.org> wrote:

My intuition says the extra state & branch needed for generators like
TakeWhile could very well be optimized away in most cases if you dont
make use of its post-nil behavior. Say TakeWhile is implemented as such:

if done { return nil }
guard let element = base.next() where predicate(element) else {
  done = true
  return nil
}
return element

If the generator is then used in the common case:

let generator = TakeWhileGenerator(...)
while let element = generator.next() {
  foo(element)
}

Should give us effectively:

var base = ...
let predicate = ...
var done = false
while true {
  if done { break }
  guard let element = base.next() where predicate(element) else {
    done = true
    break
  }
  foo(element)
}

The optimizer should see (or at least could see) `done` is never read after
it's written to (thus removing the assignment), and therefore
when checking the condition it can only be false (thus it can also be
removed).

--
-Dave

This is a current limitation of the compiler. The resilience effort
should make the specialization available for user-defined modules.
Thus, I don't think it makes sense to factor in the current limitation
into a long-term decision.

Dmitri

···

On Sun, Mar 6, 2016 at 6:16 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

On Sun, Mar 6, 2016, at 05:46 AM, Patrick Pijnappel wrote:

My intuition says the extra state & branch needed for generators like
TakeWhile could very well be optimized away in most cases if you dont make
use of its post-nil behavior. Say TakeWhile is implemented as such:

if done { return nil }
guard let element = base.next() where predicate(element) else {
  done = true
  return nil
}
return element

If the generator is then used in the common case:

let generator = TakeWhileGenerator(...)
while let element = generator.next() {
  foo(element)
}

Should give us effectively:

var base = ...
let predicate = ...
var done = false
while true {
  if done { break }
  guard let element = base.next() where predicate(element) else {
    done = true
    break
  }
  foo(element)
}

The optimizer should see (or at least could see) `done` is never read after
it's written to (thus removing the assignment), and therefore when checking
the condition it can only be false (thus it can also be removed).

This may be doable for stdlib generators, but Swift currently has a
limitation where generic types/functions defined outside the current file
(or module with -whole-module-optimization) are only specialized for their
parameters if they come from the stdlib. Any such types/functions defined in
third-party libraries (or in other files if you're not using
-whole-module-optimization) use virtual dispatch on the generic parameters
instead. This means that any generic GeneratorTypes defined in third-party
libraries won't be able to optimize away this check as it won't be able to
tell that it isn't needed.

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

That's not true. Once you start putting generators into a larger
context, you have:

- collection's generator should generate the same elements as
subscripting would produce;

- a generator for a sequence should produce at least
`underestimatedCount` elements;

- a generator for a sequence should produce the same elements as the
`forEach` method;

- lots and lots of similar high-level relationships.

Dmitri

···

On Tue, Mar 8, 2016 at 4:50 PM, Kevin Ballard <kevin@sb.org> wrote:

In fact, it's the only requirement not enforced by the type system that a GeneratorType implementation must conform to.

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

That's not really true; in fact I'd say it's the opposite. Any library that's not shipped with the app has the freedom to change its generator implementation in the future, and therefore you'll get even fewer guarantees about the generator than you have today.

Jordan

···

On Mar 8, 2016, at 11:38, Dmitri Gribenko via swift-evolution <swift-evolution@swift.org> wrote:

On Sun, Mar 6, 2016 at 6:16 PM, Kevin Ballard via swift-evolution > <swift-evolution@swift.org> wrote:

On Sun, Mar 6, 2016, at 05:46 AM, Patrick Pijnappel wrote:

My intuition says the extra state & branch needed for generators like
TakeWhile could very well be optimized away in most cases if you dont make
use of its post-nil behavior. Say TakeWhile is implemented as such:

if done { return nil }
guard let element = base.next() where predicate(element) else {
done = true
return nil
}
return element

If the generator is then used in the common case:

let generator = TakeWhileGenerator(...)
while let element = generator.next() {
foo(element)
}

Should give us effectively:

var base = ...
let predicate = ...
var done = false
while true {
if done { break }
guard let element = base.next() where predicate(element) else {
   done = true
   break
}
foo(element)
}

The optimizer should see (or at least could see) `done` is never read after
it's written to (thus removing the assignment), and therefore when checking
the condition it can only be false (thus it can also be removed).

This may be doable for stdlib generators, but Swift currently has a
limitation where generic types/functions defined outside the current file
(or module with -whole-module-optimization) are only specialized for their
parameters if they come from the stdlib. Any such types/functions defined in
third-party libraries (or in other files if you're not using
-whole-module-optimization) use virtual dispatch on the generic parameters
instead. This means that any generic GeneratorTypes defined in third-party
libraries won't be able to optimize away this check as it won't be able to
tell that it isn't needed.

This is a current limitation of the compiler. The resilience effort
should make the specialization available for user-defined modules.
Thus, I don't think it makes sense to factor in the current limitation
into a long-term decision.

Those are requirements for collections/sequences, I was talking strictly about GeneratorType implementations. If you want to talk about sequences/collections, then we've already bypassed the whole post-nil discussion because anyone working with sequences/collections instead of generators has already lost the ability to invoke next() after it has returned nil.

-Kevin Ballard

···

On Tue, Mar 8, 2016, at 05:03 PM, Dmitri Gribenko wrote:

On Tue, Mar 8, 2016 at 4:50 PM, Kevin Ballard <kevin@sb.org> wrote:
> In fact, it's the only requirement not enforced by the type system that a GeneratorType implementation must conform to.

That's not true. Once you start putting generators into a larger
context, you have:

- collection's generator should generate the same elements as
subscripting would produce;

- a generator for a sequence should produce at least
`underestimatedCount` elements;

- a generator for a sequence should produce the same elements as the
`forEach` method;

- lots and lots of similar high-level relationships.

What I meant is that we would allow user-defined modules to opt into
specialization. Currently it is not possible for user-defined modules
at all.

Dmitri

···

On Tue, Mar 8, 2016 at 2:52 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Mar 8, 2016, at 11:38, Dmitri Gribenko via swift-evolution > <swift-evolution@swift.org> wrote:

This is a current limitation of the compiler. The resilience effort
should make the specialization available for user-defined modules.
Thus, I don't think it makes sense to factor in the current limitation
into a long-term decision.

That's not really true; in fact I'd say it's the opposite. Any library
that's not shipped with the app has the freedom to change its generator
implementation in the future, and therefore you'll get even fewer guarantees
about the generator than you have today.

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

One minor change to what I've been proposing: Instead of merely saying that it's implementation-defined, we should expressly say that invoking next() after it has previously returned nil may return nil or it may return an implementation-defined value, but it should not fatalError() (unless some other GeneratorType requirement has been violated). Which is to say, after a GeneratorType has returned nil from next(), it should always be safe to invoke next() again, it's just up to the particular implementation to determine what value I get by doing that.

-Kevin Ballard

I'm torn about sequences that end with nil and should continue always return nil thereafter and
(pulling a name out of the air) "samples" that may return nil or non-nil values over time. I'd prefer there
to be two distinct contracts between an iterator and another construct that may return an implementation-defined
value after nil.

Otherwise, I'm agreed that there should not be a precondition or fatal error (unless another
requirement was violated.)

-- Erica

···

On Mar 8, 2016, at 7:29 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

One minor change to what I've been proposing: Instead of merely saying that it's implementation-defined, we should expressly say that invoking next() after it has previously returned nil may return nil or it may return an implementation-defined value, but it should not fatalError() (unless some other GeneratorType requirement has been violated). Which is to say, after a GeneratorType has returned nil from next(), it should always be safe to invoke next() again, it's just up to the particular implementation to determine what value I get by doing that.

-Kevin Ballard

Proposal: Create proposal to change IteratorType post-nil guarantee by PatrickPijnappel · Pull Request #213 · apple/swift-evolution · GitHub

···

On Wed, Mar 9, 2016 at 1:29 PM, Kevin Ballard via swift-evolution < swift-evolution@swift.org> wrote:

One minor change to what I've been proposing: Instead of merely saying
that it's implementation-defined, we should expressly say that invoking
next() after it has previously returned nil may return nil or it may return
an implementation-defined value, but it should not fatalError() (unless
some other GeneratorType requirement has been violated). Which is to say,
after a GeneratorType has returned nil from next(), it should always be
safe to invoke next() again, it's just up to the particular implementation
to determine what value I get by doing that.

-Kevin Ballard
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

If your sequence produces optional values, then the result of its generator should be double-optional. If next() returns `.some(nil)`, that would be a nil value in the sequence; if it returns `nil`, that's the end.

-Joe

···

On Mar 16, 2016, at 8:24 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

On Mar 8, 2016, at 7:29 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

One minor change to what I've been proposing: Instead of merely saying that it's implementation-defined, we should expressly say that invoking next() after it has previously returned nil may return nil or it may return an implementation-defined value, but it should not fatalError() (unless some other GeneratorType requirement has been violated). Which is to say, after a GeneratorType has returned nil from next(), it should always be safe to invoke next() again, it's just up to the particular implementation to determine what value I get by doing that.

-Kevin Ballard

I'm torn about sequences that end with nil and should continue always return nil thereafter and
(pulling a name out of the air) "samples" that may return nil or non-nil values over time. I'd prefer there
to be two distinct contracts between an iterator and another construct that may return an implementation-defined
value after nil.

The use case I was thinking of was real-world sampling, where there was actually a value available or not.
Using double-optionals as a sequence would work for that. Since that approach might be intuitively
obvious, maybe should be clarified through documentation?

-- E

···

On Mar 16, 2016, at 10:41 AM, Joe Groff <jgroff@apple.com> wrote:

On Mar 16, 2016, at 8:24 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

On Mar 8, 2016, at 7:29 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:

One minor change to what I've been proposing: Instead of merely saying that it's implementation-defined, we should expressly say that invoking next() after it has previously returned nil may return nil or it may return an implementation-defined value, but it should not fatalError() (unless some other GeneratorType requirement has been violated). Which is to say, after a GeneratorType has returned nil from next(), it should always be safe to invoke next() again, it's just up to the particular implementation to determine what value I get by doing that.

-Kevin Ballard

I'm torn about sequences that end with nil and should continue always return nil thereafter and
(pulling a name out of the air) "samples" that may return nil or non-nil values over time. I'd prefer there
to be two distinct contracts between an iterator and another construct that may return an implementation-defined
value after nil.

If your sequence produces optional values, then the result of its generator should be double-optional. If next() returns `.some(nil)`, that would be a nil value in the sequence; if it returns `nil`, that's the end.

-Joe

The sequence itself would have a singly-optional element type—it's only next() that adds optionality on top of that. For most use cases, next() is just an implementation detail.

-Joe

···

On Mar 16, 2016, at 9:59 AM, Erica Sadun <erica@ericasadun.com> wrote:

On Mar 16, 2016, at 10:41 AM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Mar 16, 2016, at 8:24 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Mar 8, 2016, at 7:29 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

One minor change to what I've been proposing: Instead of merely saying that it's implementation-defined, we should expressly say that invoking next() after it has previously returned nil may return nil or it may return an implementation-defined value, but it should not fatalError() (unless some other GeneratorType requirement has been violated). Which is to say, after a GeneratorType has returned nil from next(), it should always be safe to invoke next() again, it's just up to the particular implementation to determine what value I get by doing that.

-Kevin Ballard

I'm torn about sequences that end with nil and should continue always return nil thereafter and
(pulling a name out of the air) "samples" that may return nil or non-nil values over time. I'd prefer there
to be two distinct contracts between an iterator and another construct that may return an implementation-defined
value after nil.

If your sequence produces optional values, then the result of its generator should be double-optional. If next() returns `.some(nil)`, that would be a nil value in the sequence; if it returns `nil`, that's the end.

-Joe

The use case I was thinking of was real-world sampling, where there was actually a value available or not.
Using double-optionals as a sequence would work for that. Since that approach might be intuitively
obvious, maybe should be clarified through documentation?