Lazy flatMap for Optionals


(Donnacha Oisín Kidney) #1

Currently, several of the methods on SequenceType in the standard library have lazy variants. flatMap, though, (seems) to have a version missing: while there’s a lazy version for nested sequences, there’s no lazy version for sequences of Optionals. Is there maybe a reason for this that I haven’t thought of? At any rate, here’s what I had in mind:

public struct FlatMapOptionalGenerator<G: GeneratorType, Element>: GeneratorType {
  private let f: G.Element -> Element?
  private var g: G
  public mutating func next() -> Element? {
    while let x = g.next() {
      if let y = f(x) {
        return y
      }
    }
    return nil
  }
}

public struct FlatMapOptionalSequence<S: LazySequenceType, Element>: LazySequenceType {
  private let f: S.Generator.Element -> Element?
  private let s: S
  public func generate() -> FlatMapOptionalGenerator<S.Generator, Element> {
    return FlatMapOptionalGenerator(f: f, g: s.generate())
  }
}

extension LazySequenceType {
  public func flatMap<T>(transform: Generator.Element -> T?) -> FlatMapOptionalSequence<Self, T> {
    return FlatMapOptionalSequence(f: transform, s: self)
  }
}

Does this seem like a good idea?

Oisin.


(Dmitri Gribenko) #2

Hi Oisin,

This is just an omission. Please submit a formal proposal as a pull
request to the swift-evolution repository.

Dmitri

···

On Fri, Dec 4, 2015 at 2:38 PM, Donnacha Oisín Kidney < oisin.kidney@gmail.com> wrote:

Currently, several of the methods on SequenceType in the standard library
have lazy variants. flatMap, though, (seems) to have a version missing:
while there’s a lazy version for nested sequences, there’s no lazy version
for sequences of Optionals. Is there maybe a reason for this that I
haven’t thought of? At any rate, here’s what I had in mind:

public struct FlatMapOptionalGenerator<G: GeneratorType, Element>:
GeneratorType {
  private let f: G.Element -> Element?
  private var g: G
  public mutating func next() -> Element? {
    while let x = g.next() {
      if let y = f(x) {
        return y
      }
    }
    return nil
  }
}

public struct FlatMapOptionalSequence<S: LazySequenceType, Element>:
LazySequenceType {
  private let f: S.Generator.Element -> Element?
  private let s: S
  public func generate() -> FlatMapOptionalGenerator<S.Generator, Element>
{
    return FlatMapOptionalGenerator(f: f, g: s.generate())
  }
}

extension LazySequenceType {
  public func flatMap<T>(transform: Generator.Element -> T?) ->
FlatMapOptionalSequence<Self, T> {
    return FlatMapOptionalSequence(f: transform, s: self)
  }
}

Does this seem like a good idea?

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Dmitri Gribenko) #3

Hi,

Thank you for the proposal.

Defining only one overload would cause the collection-ness of the input to
be lost. Please take a look at the current flatMap() overloads in
stdlib/public/core/FlatMap.swift: there's one in LazySequenceType, another
one in LazyCollectionType, and one more in LazyCollectionType with
bidirectional indices.

I'm also not a fan of introducing a specialized result type for this
operation: given that we need three overloads, this approach would add six
more types to the library (three sequences and three generators). Current
flatMap() overloads compose existing types, but they rely on intermediate
data structure being a sequence or a collection. Optional is not a
sequence, so that exact approach won't work here. Can you think of another
way we could combine existing types to express the result of this operation?

Dmitri

···

On Fri, Dec 4, 2015 at 2:38 PM, Donnacha Oisín Kidney < oisin.kidney@gmail.com> wrote:

Currently, several of the methods on SequenceType in the standard library
have lazy variants. flatMap, though, (seems) to have a version missing:
while there’s a lazy version for nested sequences, there’s no lazy version
for sequences of Optionals. Is there maybe a reason for this that I
haven’t thought of? At any rate, here’s what I had in mind:

public struct FlatMapOptionalGenerator<G: GeneratorType, Element>:
GeneratorType {
  private let f: G.Element -> Element?
  private var g: G
  public mutating func next() -> Element? {
    while let x = g.next() {
      if let y = f(x) {
        return y
      }
    }
    return nil
  }
}

public struct FlatMapOptionalSequence<S: LazySequenceType, Element>:
LazySequenceType {
  private let f: S.Generator.Element -> Element?
  private let s: S
  public func generate() -> FlatMapOptionalGenerator<S.Generator, Element>
{
    return FlatMapOptionalGenerator(f: f, g: s.generate())
  }
}

extension LazySequenceType {
  public func flatMap<T>(transform: Generator.Element -> T?) ->
FlatMapOptionalSequence<Self, T> {
    return FlatMapOptionalSequence(f: transform, s: self)
  }
}

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Donnacha Oisín Kidney) #4

(forgot to cc the mailing list)

You can define it in terms of a map-filter-map chain, like this:

extension LazySequenceType {
  
  @warn_unused_result
  public func flatMap<T>(transform: Elements.Generator.Element -> T?)
    -> LazyMapSequence<LazyFilterSequence<LazyMapSequence<Elements, T?>>, T> {
      return self
        .map(transform)
        .filter { opt in opt != nil }
        .map { notNil in notNil! }
  }
}

The version for LazyCollectionType can be done similarly:

extension LazyCollectionType {
  
  @warn_unused_result
  public func flatMap<T>(transform: Elements.Generator.Element -> T?)
    -> LazyMapCollection<LazyFilterCollection<LazyMapCollection<Elements, T?>>, T> {
      return self
        .map(transform)
        .filter { opt in opt != nil }
        .map { notNil in notNil! }
  }
}

There seems to be no performance overhead vs the custom struct version in my (very preliminary) testing.

The version for a LazyCollectionType with a BidirectionalIndexType would rely on a similar LazyFilterCollection, but a BidirectionalFilterCollection doesn’t exist (I think). Is that something that might be included in this proposal?

Oisin.

···

On 6 Dec 2015, at 09:11, Dmitri Gribenko <gribozavr@gmail.com> wrote:

On Fri, Dec 4, 2015 at 2:38 PM, Donnacha Oisín Kidney <oisin.kidney@gmail.com <mailto:oisin.kidney@gmail.com>> wrote:
Currently, several of the methods on SequenceType in the standard library have lazy variants. flatMap, though, (seems) to have a version missing: while there’s a lazy version for nested sequences, there’s no lazy version for sequences of Optionals. Is there maybe a reason for this that I haven’t thought of? At any rate, here’s what I had in mind:

public struct FlatMapOptionalGenerator<G: GeneratorType, Element>: GeneratorType {
  private let f: G.Element -> Element?
  private var g: G
  public mutating func next() -> Element? {
    while let x = g.next() {
      if let y = f(x) {
        return y
      }
    }
    return nil
  }
}

public struct FlatMapOptionalSequence<S: LazySequenceType, Element>: LazySequenceType {
  private let f: S.Generator.Element -> Element?
  private let s: S
  public func generate() -> FlatMapOptionalGenerator<S.Generator, Element> {
    return FlatMapOptionalGenerator(f: f, g: s.generate())
  }
}

extension LazySequenceType {
  public func flatMap<T>(transform: Generator.Element -> T?) -> FlatMapOptionalSequence<Self, T> {
    return FlatMapOptionalSequence(f: transform, s: self)
  }
}

Hi,

Thank you for the proposal.

Defining only one overload would cause the collection-ness of the input to be lost. Please take a look at the current flatMap() overloads in stdlib/public/core/FlatMap.swift: there's one in LazySequenceType, another one in LazyCollectionType, and one more in LazyCollectionType with bidirectional indices.

I'm also not a fan of introducing a specialized result type for this operation: given that we need three overloads, this approach would add six more types to the library (three sequences and three generators). Current flatMap() overloads compose existing types, but they rely on intermediate data structure being a sequence or a collection. Optional is not a sequence, so that exact approach won't work here. Can you think of another way we could combine existing types to express the result of this operation?

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com <mailto:gribozavr@gmail.com>>*/


(Dmitri Gribenko) #5

(forgot to cc the mailing list)

You can define it in terms of a map-filter-map chain, like this:

extension LazySequenceType {

  @warn_unused_result
  public func flatMap<T>(transform: Elements.Generator.Element -> T?)
    -> LazyMapSequence<LazyFilterSequence<LazyMapSequence<Elements, T?>>, T>
{
      return self
        .map(transform)
        .filter { opt in opt != nil }
        .map { notNil in notNil! }
  }
}

The version for LazyCollectionType can be done similarly:

extension LazyCollectionType {

  @warn_unused_result
  public func flatMap<T>(transform: Elements.Generator.Element -> T?)
    -> LazyMapCollection<LazyFilterCollection<LazyMapCollection<Elements,
T?>>, T> {
      return self
        .map(transform)
        .filter { opt in opt != nil }
        .map { notNil in notNil! }
  }
}

I like this.

The version for a LazyCollectionType with a BidirectionalIndexType would
rely on a similar LazyFilterCollection, but a BidirectionalFilterCollection
doesn’t exist (I think). Is that something that might be included in this
proposal?

Indeed. I think that would be a separate proposal.

Dmitri

···

On Sun, Dec 6, 2015 at 8:06 AM, Donnacha Oisín Kidney <oisin.kidney@gmail.com> wrote:

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/