I guess breaking existing code will be the show stopper for this proposal — but I generally think that compatibility is a poor rationale to stop an improvement, so my personal reasons are different:
The name is just wrong.
Just have a look at this simple example
extension Int {
func justImagineError() throws -> Int {
return self
}
}
let ints: [Int?] = [nil]
let result = ints.flatMap {
return try? $0?.justImagineError()
}
print(result)
If flatMap would really filter out nil values, this should yield an empty array as result — but the actual output is [nil]
This is a wonderful example! But it’s an argument for a different discussion (of general usefulness of implicit optional promotion). Thanks to the optional promotion, what the closure returns is not nil, but instead is .some(nil), and that is not filtered out.
···
On Nov 7, 2017, at 5:23 PM, Tino Heth via swift-evolution <swift-evolution@swift.org> wrote:
This proposal is going to cause an insane amount of code churn. The
proposal suggests this overload of flatMap is used "in certain
circumstances", but in my experience it's more like 99% of all
flatMaps on sequences are to deal with optionals, not to flatten
nested sequences.>>
• Is the problem being addressed significant enough to warrant a
change to Swift?>>
I don't think so. It's a fairly minor issue, one that really only
affects new Swift programmers anyway rather than all users, and it
will cause far too much code churn to be worthwhile.>>
I'd much rather see a proposal to add a new @available type,
something like 'warning’,> Please write one, seriously!
that lets you attach an arbitrary warning message to a call (which
you can kind of do with 'deprecated' except that makes the warning
message claim the API is deprecated). With that sort of thing we
could then declare>>
extension Sequence { @available(*, warning: "Use map instead")
func flatMap<U>(_ f: (Element) -> U) -> [U] {
return map(f)
}
}
Interesting. It looks like what's going on is, with the deprecated form,
`[a, b, c]` is immediately treated as `[Any]`, but without it, it's
treated as `[Int?]` (invoking optional hoisting on `a`) and the closure
itself is inferred as returning Any.
There's probably some shenanigans we could do like adding an
_ImplicitlyPromotedOptional<T> type that takes precedence over
Optional<T> for overloading purposes, which could then be used to
reintroduce the deprecated form of flatMap, but I'm not sure if it's
worth the additional complexity unless there are more functions than
just flatMap that we'd want to kill optional hoisting for.
-Kevin Ballard
···
On Wed, Nov 8, 2017, at 11:28 AM, Max Moiseev wrote:
On Nov 7, 2017, at 3:34 PM, Kevin Ballard via swift-evolution <swift- >> evolution@swift.org> wrote:>>
And now if someone writes flatMap in a way that invokes optional
hoisting, it'll match this overload instead and warn them.>>
• How much effort did you put into your review? A glance, a quick
reading, or an in-depth study?>>
A quick reading, and a couple of minutes testing overload behavior
with availability attributes (to confirm that we can't simply use
'unavailable' for this).>>
-Kevin Ballard
On 8 Nov 2017, at 8:17 am, Stephen Celis via swift-evolution <swift-evolution@swift.org> wrote:
+1
On Nov 7, 2017, at 6:23 PM, John McCall <rjmccall@apple.com> wrote:
• What is your evaluation of the proposal?
BJ summarized my thoughts nicely. I think “flatMap” in its current form is confusing to both newcomers and functional programmers familiar with the nomenclature. I’ve been aware of this disparity and thought about it for awhile. I typically redefine “flatMap” as “mapOptional” in my projects for clarity.
• Is the problem being addressed significant enough to warrant a change to Swift?
I think so! It improves the user experience in a significant way to those that encounter the function in the wild.
• Does this proposal fit well with the feel and direction of Swift?
Yes.
• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
It fits in well. Swift’s current extra implementation of “flatMap” is the anomaly.
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
On Wed, Nov 8, 2017, at 09:29 PM, Paul Cantrell via swift-evolution wrote:> The problem in the Doodads example is that *the name flatMap is used
to identify two distinct intents*: concatenating arrays and filtering
nils. One can argue that those two operations are, in some lofty
abstract sense, if you squint, two instances of some more general
pattern — but I don’t think it’s fair to say that they represent the
same *intent*. These separate intents deserve separate names.
They absolutely represent the same intent if you think of an optional as
a collection of zero or one elements.
-Kevin Ballard
Experienced programmers (as experienced as any Swift programmer can be for a 4-year-old language) frequently use flatMap when they mean map
When you look at those examples:
What would stop those programmers to use filterMap in the exact same way, instead of switching to map?
A hypothetic migrator most likely would just replace names...
The problem in the Doodads example is that the name flatMap is used to identify two distinct intents: concatenating arrays and filtering nils.
You may have read about my issues with the prefix, but following the same logic of „filtering nils“, the other variant of flatMap just filters out empty arrays.
This proposal only suggests changing the one on sequences, and even then, not all of them. Currently, the following exist:
// Reminder: Int("someString") returns 'Int?' in the following examples
// (1) Sequence.flatMap() with a closure returning an optional, dropping anything that maps to nil
["1", "mountain", "2"].flatMap({ Int($0) }) // [1, 2]
// (2) Sequence.flatMap() with a closure returning a sequence, then joining (“flattening”) them into one array
[1, 2, 3].flatMap({ Array(1...$0) }) // [1, 1, 2, 1, 2, 3]
// (3) Optional.flatMap() with a closure returning an optional, flattening a nested optional.
("3" as String?).flatMap({ Int($0) }) // Optional(3)
The 2nd and 3rd variants are both about flattening nested structures, and would not change under this proposal. We would keep calling them “flatMap()”. There is no a proposal to add Optional.filterMap, nor to change the Sequence.flatMap variant that flattens nested sequences. The only change proposed it is to rename the first one.
-BJ
···
On Nov 11, 2017, at 9:49 AM, Michel Fortin via swift-evolution <swift-evolution@swift.org> wrote:
I think it fits well for arrays. Not sure about optionals.
I like that name too, though it’s probably my Ruby experience showing. I like the clarity of “compact” for nils, “filter” for booleans. I like how fluently “compactMap” reads.
If the core team does accept this proposal, I’d trust them to take a minute to make a considered, opinionated choice about the name.
P
···
On Nov 11, 2017, at 8:35 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:
I'd suggest `compactMap` as an alternative name, should `filterMap` find too much resistance:
Alternatively, we could think of a name for "removing optionality from a sequence" and then use that name. For example, borrowing a name from Ruby:
(Sequence where Element == T?).compact()
Sequence.compactMap(transform)
Actually, "unwrapped" might be a better name than "compact", since we use that term.
···
On Nov 8, 2017, at 9:29 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:
The problem in the Doodads example is that the name flatMap is used to identify two distinct intents: concatenating arrays and filtering nils. One can argue that those two operations are, in some lofty abstract sense, if you squint, two instances of some more general pattern — but I don’t think it’s fair to say that they represent the same intent. These separate intents deserve separate names.
This proposal is going to cause an insane amount of code churn. The proposal suggests this overload of flatMap is used "in certain circumstances", but in my experience it's more like 99% of all flatMaps on sequences are to deal with optionals, not to flatten nested sequences.
• Is the problem being addressed significant enough to warrant a change to Swift?
I don't think so. It's a fairly minor issue, one that really only affects new Swift programmers anyway rather than all users, and it will cause far too much code churn to be worthwhile.
I'd much rather see a proposal to add a new @available type, something like 'warning', that lets you attach an arbitrary warning message to a call (which you can kind of do with 'deprecated' except that makes the warning message claim the API is deprecated).
As review manager, I generally try to avoid commenting on threads, but I find this point interesting in a way that, if you don't mind, I'd like to explore.
Would this attribute not be a form of deprecation? Certainly it acts to discourage current and subsequent use, since every such use will evoke a warning.
Is the word "deprecation" just too strong? Often we think of deprecated APIs as being ones with more functional problems, like an inability to report errors, or semantics that must have seemed like a good idea at the time. Here it's just that the API has a name we don't like, and perhaps "deprecation" feels unnecessarily judgmental.
What I'm suggesting is that we don't change the API name at all. That's why I don't want to use 'deprecated', because we're not actually deprecating something. I'm just suggesting an alternative way of flagging cases where the user tries to use flatMap but accidentally invokes optional hoisting, and that's by making a new overload of flatMap that works for non-optional (non-sequence) values and warns the user that what they're doing is better done as a map. Using the 'deprecated' attribute for this would be confusing because it would make it sound like flatMap itself is deprecated when it's not.
I see. Thanks.
John.
···
On Nov 8, 2017, at 1:20 PM, Kevin Ballard <kevin@sb.org> wrote:
On Tue, Nov 7, 2017, at 09:37 PM, John McCall wrote:
On Nov 7, 2017, at 6:34 PM, Kevin Ballard via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Tue, Nov 7, 2017, at 03:23 PM, John McCall via swift-evolution wrote:
Also, more practically, it conflates a relatively unimportant suggestion — that we should call the new method in order to make our code clearer — with a more serious one — that we should revise our code to stop using a problematic API. Yes, the rename has a fix-it, but still: to the extent that these things demand limited attention from the programmer, that attention should clearly be focused on the latter set of problems. Perhaps that sense of severity is something that an IDE should take into consideration when reporting problems.
What else would you have in mind for this warning?
The main use for this warning would be for adding overloads to methods that take optionals in order to catch the cases where people invoke optional hoisting, so we can tell them that there's a better way to handle it if they don't have an optional. flatMap vs map is the obvious example, but I'm sure there are other cases where we can do this too.
But there are also other once-off uses. For example, in the past I've written a function that should only ever be used for debugging, so I marked it as deprecated with a message saying 'remove this before committing your code'. This warning would have been better done using the new 'warning' attribute instead of as a deprecation notice.
And now if someone writes flatMap in a way that invokes optional hoisting, it'll match this overload instead and warn them.
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
A quick reading, and a couple of minutes testing overload behavior with availability attributes (to confirm that we can't simply use 'unavailable' for this).
The filter function afaik doesn’t perform that kind of stripping - if you feed it [A?], you get [A?] as output, no matter what kind of predicate you use, it can only eliminate values (Optional.none), not change their type.
···
Am 08.11.2017 um 19:23 schrieb Kevin Ballard via swift-evolution <swift-evolution@swift.org>:
On Tue, Nov 7, 2017, at 05:23 PM, Tino Heth via swift-evolution wrote:
-1
I guess breaking existing code will be the show stopper for this proposal — but I generally think that compatibility is a poor rationale to stop an improvement, so my personal reasons are different:
The name is just wrong.
Just have a look at this simple example
extension Int {
func justImagineError() throws -> Int {
return self
}
}
let ints: [Int?] = [nil]
let result = ints.flatMap {
return try? $0?.justImagineError()
}
print(result)
If flatMap would really filter out nil values, this should yield an empty array as result — but the actual output is [nil]
flatMap does filter out nil values. The problem is the return type of your block is `Int??`, not `Int?`, so it's stripping off the outer layer of optionals.
On Wed, Nov 8, 2017 at 7:35 PM, Johannes Weiß via swift-evolution < swift-evolution@swift.org> wrote:
+1 and agree with Stephen on why
> On 8 Nov 2017, at 8:17 am, Stephen Celis via swift-evolution < > swift-evolution@swift.org> wrote:
>
> +1
>
>> On Nov 7, 2017, at 6:23 PM, John McCall <rjmccall@apple.com> wrote:
>>
>> • What is your evaluation of the proposal?
>
> BJ summarized my thoughts nicely. I think “flatMap” in its current form
is confusing to both newcomers and functional programmers familiar with the
nomenclature. I’ve been aware of this disparity and thought about it for
awhile. I typically redefine “flatMap” as “mapOptional” in my projects for
clarity.
>
>> • Is the problem being addressed significant enough to warrant a
change to Swift?
>
> I think so! It improves the user experience in a significant way to
those that encounter the function in the wild.
>
>> • Does this proposal fit well with the feel and direction of Swift?
>
> Yes.
>
>> • If you have used other languages or libraries with a similar
feature, how do you feel that this proposal compares to those?
>
> It fits in well. Swift’s current extra implementation of “flatMap” is
the anomaly.
>
>> • How much effort did you put into your review? A glance, a quick
reading, or an in-depth study?
>
> Gave a quick reading to the proposal.
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
This is a wonderful example! But it’s an argument for a different discussion (of general usefulness of implicit optional promotion). Thanks to the optional promotion, what the closure returns is not nil, but instead is .some(nil), and that is not filtered out.
My point is: The proposed filterMap isn’t a combination of map and filter at all — or can you build it just out of map & filter, like flatMap is constructed from map & flatten?
The migrator would already have to be aware of the return type of the closure (to avoid migrating cases where it’s actually returning a sequence as expected), so the migrator could easily only migrate the ones that are actually returning optionals in the first place.
Programmers could continue to use filterMap incorrectly, but it would be a more explicit choice. Currently, there’s a sense among some Swift users (including some that I know personally) that flatMap is the “smart” thing to use with collections because monads or something, without any real understanding of why that is. (Remember, Swift doesn’t even have a “flatten()” function anymore, so the name “flatMap" is far from obvious.) I think it’s less likely that people will continue to use “filterMap” when they know no filtering is happening.
-BJ
···
On Nov 9, 2017, at 9:13 AM, Tino Heth via swift-evolution <swift-evolution@swift.org> wrote:
Experienced programmers (as experienced as any Swift programmer can be for a 4-year-old language) frequently use flatMap when they mean map
When you look at those examples:
What would stop those programmers to use filterMap in the exact same way, instead of switching to map?
A hypothetic migrator most likely would just replace names…
But as `Optional` does not conform to collection, I would suggest that the vast majority of of developers do _not_ think of them as a collection of zero or one elements. (I certainly don’t know any who naturally think of it that way.) We don’t treat Optional as a collection anywhere else in the API, and it seems odd to do so in just this one case.
-BJ
···
On Nov 9, 2017, at 11:36 AM, Kevin Ballard via swift-evolution <swift-evolution@swift.org> wrote:
On Wed, Nov 8, 2017, at 09:29 PM, Paul Cantrell via swift-evolution wrote:
The problem in the Doodads example is that the name flatMap is used to identify two distinct intents: concatenating arrays and filtering nils. One can argue that those two operations are, in some lofty abstract sense, if you squint, two instances of some more general pattern — but I don’t think it’s fair to say that they represent the same intent. These separate intents deserve separate names.
They absolutely represent the same intent if you think of an optional as a collection of zero or one elements.
Oh right. I guess that makes most of my review beside the point then. Sorry.
···
Le 11 nov. 2017 à 14:46, BJ Homer <bjhomer@gmail.com <mailto:bjhomer@gmail.com>> a écrit :
On Nov 11, 2017, at 9:49 AM, Michel Fortin via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
I think it fits well for arrays. Not sure about optionals.
This proposal only suggests changing the one on sequences, and even then, not all of them. Currently, the following exist:
// Reminder: Int("someString") returns 'Int?' in the following examples
// (1) Sequence.flatMap() with a closure returning an optional, dropping anything that maps to nil
["1", "mountain", "2"].flatMap({ Int($0) }) // [1, 2]
// (2) Sequence.flatMap() with a closure returning a sequence, then joining (“flattening”) them into one array
[1, 2, 3].flatMap({ Array(1...$0) }) // [1, 1, 2, 1, 2, 3]
// (3) Optional.flatMap() with a closure returning an optional, flattening a nested optional.
("3" as String?).flatMap({ Int($0) }) // Optional(3)
The 2nd and 3rd variants are both about flattening nested structures, and would not change under this proposal. We would keep calling them “flatMap()”. There is no a proposal to add Optional.filterMap, nor to change the Sequence.flatMap variant that flattens nested sequences. The only change proposed it is to rename the first one.
Yet I wouldn't follow that precedent. As Erica Sadun has told us, there is something interesting in "unwappable" types. It's a clear concept. Yet it applies to *values*, not *collections*: Optional.unwrapped, Result.unwrapped, but not Array.unwrapped. Optional<Array<Result>>.unwrapped().unwrapped() is a conceptual mess.
The functional vocabulary does not help us much, here. What about calling a spade a spade? "Filtering out nils" is not general enough to fit in the functional landscape, but it happens to be a concept that has is clear, useful, and stand-alone. "Compact" fits this precise niche very well.
Gwendal Roué
···
Le 13 nov. 2017 à 04:35, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> a écrit :
On Nov 8, 2017, at 9:29 PM, Paul Cantrell via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
The problem in the Doodads example is that the name flatMap is used to identify two distinct intents: concatenating arrays and filtering nils. One can argue that those two operations are, in some lofty abstract sense, if you squint, two instances of some more general pattern — but I don’t think it’s fair to say that they represent the same intent. These separate intents deserve separate names.
I think that's the crux of it.
I think we might be better off renaming the maps on `Optional` and then naming the `Sequence` operation analogously. For instance:
You said "compact" removes nil values, so I'd name it "removeNils".
···
On Nov 12, 2017, at 10:54 AM, Paul Cantrell via swift-evolution <swift-evolution@swift.org> wrote:
On Nov 11, 2017, at 8:35 AM, Gwendal Roué via swift-evolution <swift-evolution@swift.org> wrote:
I'd suggest `compactMap` as an alternative name, should `filterMap` find too much resistance:
I like that name too, though it’s probably my Ruby experience showing. I like the clarity of “compact” for nils, “filter” for booleans. I like how fluently “compactMap” reads.
If the core team does accept this proposal, I’d trust them to take a minute to make a considered, opinionated choice about the name.
It *is* a combination of map and filter. It is purely for performance we don’t do the same thing on any Sequence, because that will allocate a temporary array or two.
···
On Nov 8, 2017, at 12:20 PM, Tino Heth <2th@gmx.de> wrote:
This is a wonderful example! But it’s an argument for a different discussion (of general usefulness of implicit optional promotion). Thanks to the optional promotion, what the closure returns is not nil, but instead is .some(nil), and that is not filtered out.
My point is: The proposed filterMap isn’t a combination of map and filter at all — or can you build it just out of map & filter, like flatMap is constructed from map & flatten?
Well, that certainly explains the nasty return type. I rather wish we
had a LazyFlatMapSequence type rather than using map(…).filter(…).map(…)
if for no other reason than it would produce a return type that's easier
to work with.
-Kevin Ballard
···
On Wed, Nov 8, 2017, at 02:29 PM, Max Moiseev via swift-evolution wrote:>
On Nov 8, 2017, at 12:20 PM, Tino Heth <2th@gmx.de> wrote:
This is a wonderful example! But it’s an argument for a different
discussion (of general usefulness of implicit optional promotion).
Thanks to the optional promotion, what the closure returns is not
nil, but instead is .some(nil), and that is not filtered out.>> My point is: The proposed filterMap isn’t a combination of map and
It *is* a combination of map and filter. It is purely for performance
we don’t do the same thing on any Sequence, because that will allocate
a temporary array or two.