[Review] SE-0187: Introduce Sequence.filterMap(_:)

Yeah but it seems clear from the return type so I am not sure that much confusion would really exist.

Afaics, there already is lots of confusion — that’s the reason for me to write a sequence of posts in this topic, instead of an Optional ;-)
The word „flatten“ is a quite honest description, so I wonder why words like filter, remove, ignoring or skipping should be used instead.

„Compact“ would be less irritating, but I could imagine that it indicates something like eliminating repeated occurrences.

Tino

(I’m quite close to attach a hand-drawn illustration of the flatMap process — I don’t think you want that to happen ;-)

I didn’t want to write any additional posts in this thread ;-), but as you ask a concrete question:
Imho there isn't any benefit over just using the two methods — it might be different for special cases with default values for one of the parameters (like getting rid of NaN, Zero or Infinity in a collection of floats), but I wouldn’t expect any of those to be part of the stdlib.
It’s just that this function is the natural interpretation of its name (especially for someone who knows flatMap), and I encountered people who are convinced that flatMap on Collections of Optionals is just like that: You map, you filter, you return the result (some even thought it’s the other way round, filtering being the first step).
Please note that the actual implementation of the method doesn’t matter at all for me — what matters is expectation (after all, I wouldn’t rename flatMap to joinMap), and the expectation for a filter is quite clear.

What kept the dispute alive that long is not that much that I think filterMap is a bad name per se, it is the widespread belief that it is a better description than flatMap.
Me and others strongly disagree here, and I’m quite sure no argument can change my mind on that. Therefor, the whole basis of the discussion is flawed, and I would be quite disappointed if the rationale for the acceptance of this proposal would be that flatMap is a wrong name, and filterMap is correct.
The real question should be if the renaming is beneficial for developers — and in this regard, wrong beliefs can work as good a good as true ones: As many people seem to have the same misconception about flatMap, it might be better to accept that (although I would like to see Kevin Ballards idea implemented anyways).

Concatenation is such a case: It doesn’t follow the laws of addition at all, but still, the same operator that is well established for adding numbers was chosen for it, because of the popular believe that joining two collections is like adding two numbers.

Tino

···

Am 10.11.2017 um 23:32 schrieb Max Moiseev <moiseev@apple.com>:

extension Collection {
  func filterMap<T>(transform: (Element) -> T, include: (T) -> Bool) -> [T] {
    return self.map(transform).filter(include)
  }
}

I understand the risk of diverging the discussion, but still, what’s the benefit of having this function over using the built-in `.lazy.map{}.filter{}` ?

“Flatten”, in the context here (flattening a Sequence of Optionals), only makes sense if you consider Optional as a sequence of zero or one elements. While such a sequence does act very similarly to an Optional, the fact remains that Optional is not a Sequence, and is generally not treated as one by any other part of the language. It fundamentally makes no more sense to “flatten" a Sequence of Optionals than it does to “flatten" an Optional of a Sequence.

In my mind, filter makes sense because I’m filtering out some of the results of the transformation operation before they get put into the resulting array. It’s a different kind of operation; not a map, but a filterMap. (Maybe filteringMap would be clearer?) The operation is not “map into a new array, then filter”. It’s “run this transformation for each item. If it produces something, keep it.” There’s no filter or compact that runs on the entire collection.

-BJ

···

On Nov 14, 2017, at 7:50 AM, Tino Heth via swift-evolution <swift-evolution@swift.org> wrote:

Yeah but it seems clear from the return type so I am not sure that much confusion would really exist.

Afaics, there already is lots of confusion — that’s the reason for me to write a sequence of posts in this topic, instead of an Optional ;-)
The word „flatten“ is a quite honest description, so I wonder why words like filter, remove, ignoring or skipping should be used instead.

„Compact“ would be less irritating, but I could imagine that it indicates something like eliminating repeated occurrences.

Tino

(I’m quite close to attach a hand-drawn illustration of the flatMap process — I don’t think you want that to happen ;-)

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Looks like I finally have to draw on drawings... (hope the pictures aren’t considered inappropriate content ;-)

But after all, it might be the best way to convey my point anyways: I’m not thinking in terms of type names (why should it only be possible to flatten a „Sequence“? Many people associate this operation with lists, which aren’t common in Swift at all) or implementation details (neither Optional nor Sequence actually has a flatten-method).
Instead, I tend to look at the easiest way to explain how to get from input to output, so here is what I have in my mind when I speak about Sequence-Sequence flatmap:

Imgur

There is a transform, which turns an input value (red circle) into a collection (blue basket) of output values (green squares).
map takes an array of those input values, and stores each result-collection in an array (light blue).
flatten takes that container, unwraps each sub-collection, and stores its contents together with the other elements.
Note that empty collections aren’t skipped or removed — they just have nothing that they could contribute to the final result, so no trace of them is left.
flatMap just is flatten performed on the output of map.

Now, Optionals… I could simply say „think of those as an array with a maximal size of one“ — but because there’s really no fundamental difference, I just copied the sequence-illustration, and changed one tiny bit: The transformation doesn’t return multiple items now, so that the blue basket can act as representation for an Optional.

Imgur

So conceptionally, both overrides are equivalent: The transformation always produces something — an Optional, and that can contain a value, or it can be empty, just like an array.
You also see that the flatten step really does what it says: It removes one layer of objects, and leaves a flush surface without gaps.

So, that should really be my last word in this discussion — I won’t add an animation or start singing about Optionals ;-)

- Tino

I think we understand each other; we just disagree. If you believe that it makes intuitive sense to talk about “flattening” a sequence of optionals, then the current name perhaps makes sense. If you don’t think so, then the “flatten” name is awkward in this context. I don’t think most Swift users would find it intuitive, as Optional does not conform to Sequence, and there is no ValueContainer protocol on which we might define “flatMap”. (And introducing such a protocol is outside the scope of this proposal.)

Even if “flatMap” does make conceptual sense to you on a sequence of Optionals, though, the API as it exists right now still has the other problems mentioned in the proposal and throughout this thread:
Implicit promotion of non-Optional types to Optional types makes it easy to accidentally call the wrong function.
Discoverability: Users looking to filter out ‘nil’ values have to know about something that does not say “filter”. Most new Swift users do not naturally discover this functionality.
Fragility: Adding “Collection” conformance to a type caused source incompatibility (as shown in the original proposal). In general, adding conformances should not break existing code.
Fragility: Changing a type from Optional to non-Optional (or vice versa) should not silently change behavior. But in this case, because of implicit optional promotion, it can.
Non-obvious behavior: In my experience, new Swift users do not intuitively understand what it means to “flatMap” a sequence of optionals, and could not intuitively understand the meaning of “flatten” in this context.
We can make Swift a more intuitive language by not pretending that Optionals are Sequences when they’re really not, and we can make it a more robust language by avoiding ambiguity between different versions of Sequence.flatMap in the presences of subtle type changes.

Anyway, I guess the review period ended last night, so we’ll know soon how this turns out.

-BJ

···

On Nov 15, 2017, at 8:39 AM, Tino Heth <2th@gmx.de> wrote:

Looks like I finally have to draw on drawings... (hope the pictures aren’t considered inappropriate content ;-)

But after all, it might be the best way to convey my point anyways: I’m not thinking in terms of type names (why should it only be possible to flatten a „Sequence“? Many people associate this operation with lists, which aren’t common in Swift at all) or implementation details (neither Optional nor Sequence actually has a flatten-method).
Instead, I tend to look at the easiest way to explain how to get from input to output, so here is what I have in my mind when I speak about Sequence-Sequence flatmap:

<PastedGraphic-2.png>
Imgur: The magic of the Internet

There is a transform, which turns an input value (red circle) into a collection (blue basket) of output values (green squares).
map takes an array of those input values, and stores each result-collection in an array (light blue).
flatten takes that container, unwraps each sub-collection, and stores its contents together with the other elements.
Note that empty collections aren’t skipped or removed — they just have nothing that they could contribute to the final result, so no trace of them is left.
flatMap just is flatten performed on the output of map.

Now, Optionals… I could simply say „think of those as an array with a maximal size of one“ — but because there’s really no fundamental difference, I just copied the sequence-illustration, and changed one tiny bit: The transformation doesn’t return multiple items now, so that the blue basket can act as representation for an Optional.

<PastedGraphic-4.png>
https://imgur.com/qq95b31

So conceptionally, both overrides are equivalent: The transformation always produces something — an Optional, and that can contain a value, or it can be empty, just like an array.
You also see that the flatten step really does what it says: It removes one layer of objects, and leaves a flush surface without gaps.

So, that should really be my last word in this discussion — I won’t add an animation or start singing about Optionals ;-)

- Tino

Your logic makes sense but doesn’t account for where “flatMap” came from and the expectations of how “flatMap” should work.

In functional programming, “map” must preserve structure (the container shape should stay the same), and “flatMap” must use the same structure throughout, including the return value of the higher order function it takes.

map :: (a -> b) -> m a -> m b
flatMap :: (a -> m b) -> m a -> m b

You may argue that Swift isn’t really a functional language and doesn’t have to adhere to functional rules, but as long as it adopts functional nomenclature, it should do its best to behave with functional expectations. It’s actually impossible to write “flatMap” as written above against a protocol like Sequence because Swift lacks higher kinded types, but we can at least approximate the behavior that functional programmers expect by narrowing the scope of what’s appropriate to return from that higher order function, which is in this case a sequence (of which Optional is not).

In fact, I’d rather make “flatMap” stricter by requiring that higher order function to return an Array! This of course would kill some ergonomics, but it’d at least make things easier to reason about when you always get an Array out the other side.

Stephen

···

On Nov 15, 2017, at 10:39 AM, Tino Heth via swift-evolution <swift-evolution@swift.org> wrote:

So conceptionally, both overrides are equivalent: The transformation always produces something — an Optional, and that can contain a value, or it can be empty, just like an array.
You also see that the flatten step really does what it says: It removes one layer of objects, and leaves a flush surface without gaps.

So, that should really be my last word in this discussion — I won’t add an animation or start singing about Optionals ;-)

"In functional programming, “map” must preserve structure (the
container shape should stay the same), and “flatMap” must use the same
structure throughout, including the return value of the higher order
function it takes."

I was gonna write the same, completely agree with that.
I know Swift is not purely functional and that some don't really care
about functional names, but giving names to concepts is always good
for communication and that's the only reason why FP langs use specific
names for specific types and operations. Even if we don't really carea
bout that, and we just want to pick approachable names for newcomers
Swift already picked "flatMap" (and map, reduce, etc) so the least
think we could to is at least make the name apply to operations that
follow the proper rules.

···

On Wed, Nov 15, 2017 at 8:16 PM, Stephen Celis via swift-evolution <swift-evolution@swift.org> wrote:

On Nov 15, 2017, at 10:39 AM, Tino Heth via swift-evolution > <swift-evolution@swift.org> wrote:

So conceptionally, both overrides are equivalent: The transformation always
produces something — an Optional, and that can contain a value, or it can be
empty, just like an array.
You also see that the flatten step really does what it says: It removes one
layer of objects, and leaves a flush surface without gaps.

So, that should really be my last word in this discussion — I won’t add an
animation or start singing about Optionals ;-)

Your logic makes sense but doesn’t account for where “flatMap” came from and
the expectations of how “flatMap” should work.

In functional programming, “map” must preserve structure (the container
shape should stay the same), and “flatMap” must use the same structure
throughout, including the return value of the higher order function it
takes.

map :: (a -> b) -> m a -> m b
flatMap :: (a -> m b) -> m a -> m b

You may argue that Swift isn’t really a functional language and doesn’t have
to adhere to functional rules, but as long as it adopts functional
nomenclature, it should do its best to behave with functional expectations.
It’s actually impossible to write “flatMap” as written above against a
protocol like Sequence because Swift lacks higher kinded types, but we can
at least approximate the behavior that functional programmers expect by
narrowing the scope of what’s appropriate to return from that higher order
function, which is in this case a sequence (of which Optional is not).

In fact, I’d rather make “flatMap” stricter by requiring that higher order
function to return an Array! This of course would kill some ergonomics, but
it’d at least make things easier to reason about when you always get an
Array out the other side.

Stephen
https://www.pointfree.co

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Alejandro Martinez