Generators vs. errors


(Jens Alfke) #1

What should one do, when implementing a Generator, if the underlying data source can experience errors? Generator.next isn’t allowed to throw, only to return nil at the end of the sequence.

The only idea I can think of is to add an `error` property to my Generator implementation, and request that the caller check it after the iteration ends, and/or a `checkForError()` method that can throw an error if there is one:
  var q: Query = db.query(…)
  for (row in q) { … }
  q.checkForError()

As the example implies, this problem comes up when creating Swift bindings for database iterators/cursors. Generator is the obvious protocol to implement, especially since without it you don’t get the idiomatic for/in loop, but the `next` method does have to hit the database, so it has the possibility of I/O errors.

Another possibility is to do all the failable database work up front in the `query` method that creates the Generator, but this means all the results have to be cached in memory, which isn’t scaleable. (Keep in mind I work for a database company <http://couchbase.com/> whose customers often have multi-terabyte data sets.)

I know there’s been a proposal (by Brent Royal-Gordon) to allow property accessors and subscripts to throw, which I strongly endorse, but I think we also need to allow Generators to throw. Fodder for the swift-evolution list…

—Jens


(David Sweeris) #2

I think I’d either just end the sequence on an error, or do like you were thinking and have a function/computed property that gives an array containing all the rows which had an error or something.

I’m not a DB guy, though.

- Dave Sweeris

···

On Apr 12, 2016, at 1:53 PM, Jens Alfke via swift-users <swift-users@swift.org> wrote:

What should one do, when implementing a Generator, if the underlying data source can experience errors? Generator.next isn’t allowed to throw, only to return nil at the end of the sequence.

The only idea I can think of is to add an `error` property to my Generator implementation, and request that the caller check it after the iteration ends, and/or a `checkForError()` method that can throw an error if there is one:
  var q: Query = db.query(…)
  for (row in q) { … }
  q.checkForError()

As the example implies, this problem comes up when creating Swift bindings for database iterators/cursors. Generator is the obvious protocol to implement, especially since without it you don’t get the idiomatic for/in loop, but the `next` method does have to hit the database, so it has the possibility of I/O errors.

Another possibility is to do all the failable database work up front in the `query` method that creates the Generator, but this means all the results have to be cached in memory, which isn’t scaleable. (Keep in mind I work for a database company <http://couchbase.com/> whose customers often have multi-terabyte data sets.)

I know there’s been a proposal (by Brent Royal-Gordon) to allow property accessors and subscripts to throw, which I strongly endorse, but I think we also need to allow Generators to throw. Fodder for the swift-evolution list…

—Jens
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users


(Brent Royal-Gordon) #3

What I might do is use a block to make sure you can't forget the `checkForError()` method:

  let query = db.query(…)
  try query.withResults { results in
    // `results` is a SequenceType you can iterate over.
    // You can only get at it by calling `withResults` on a Query, and it becomes invalid once
    // `withResults` returns.
    // If `results` encounters an error, it terminates early.
    for row in results {
      // Use your row
    }
  }

···

On Apr 12, 2016, at 11:53 AM, Jens Alfke via swift-users <swift-users@swift.org> wrote:

What should one do, when implementing a Generator, if the underlying data source can experience errors? Generator.next isn’t allowed to throw, only to return nil at the end of the sequence.

The only idea I can think of is to add an `error` property to my Generator implementation, and request that the caller check it after the iteration ends, and/or a `checkForError()` method that can throw an error if there is one:
  var q: Query = db.query(…)
  for (row in q) { … }
  q.checkForError()

As the example implies, this problem comes up when creating Swift bindings for database iterators/cursors. Generator is the obvious protocol to implement, especially since without it you don’t get the idiomatic for/in loop, but the `next` method does have to hit the database, so it has the possibility of I/O errors.

--
Brent Royal-Gordon
Architechies