Returning to an old hobbyhorse: Migrating higher order function names to comply with API guidelines

It's also worth pointing out that the guidelines explicitly leave room for terms of art. These are some of the most canonical terms of art there are. I agree that a change is not warranted here.

4 Likes

In my experience, few things cause more harm than this.

It takes a lot of effort on our (teachers) part to get students to adhere to the API guidelines. If the Standard Library itself doesn't even adhere to them (or gets its priorities wrong), most of that effort is wasted.

Also keep in mind that terms of art don't apply to new learners. To them, naming a method "filter" because other languages name it that way makes no sense.

13 Likes

When you have a collection-of-collections (eg. [[T]]) and want to perform in-place mutation of the inner elements. With a map-in-place (I like the spelling mutateEach) that takes its argument inout, you can simply do a double-map:

outerCollection.mutateEach { innerCollection in
  innerCollection.mutateEach { $0 += 1 }
}

If you write the equivalent with out-of-place map, then you’ll end up allocating new space for every inner collection as well as the outer collection. The in-place version allocates nothing.

Agreed. The form prefixes are explicitly stated as being used when the base name is a noun (eg. formUnion). The names map and filter are verbs.

• • •

I am a strong advocate of the position that functional-programming terminology is almost entirely terrible. The names for operations are abysmal. The meanings of the words that functional programmers use, do not correspond to the actual concepts which they are applied to.

However, the one advantage of those existing names, is that they are terms of art. People know them, and have written about them, and explanations exist for them.

If we think the names are bad enough to warrant changing them, then we should choose ones that we think are better. And that means choosing names that are actually meaningful.

In particular, small changes to the existing names will be even worse than no change. That would lose the benefit of term-of-art recognizability, while retaining the problem of the words meaning the wrong thing.

The existing names are locally-optimal because they are terms of art, so if we want something better that will necessarily involve moving to a different “hill” entirely.

• • •

If we are going to try to improve these method names, then we should come at it from the perspective of applying the best-practice Swift naming guidelines to replace the current term-of-art spellings:

Reduce

reduce is a bad name for its job. The operation is about combining elements. A more accurate name would be combine. As in:
let sum = values.combine(0, +).

Map

map is a reasonable name for its job, because mathematically a map is a transformation and colloquially we talk about mapping one thing to another, but it has a verb-tense that would imply mutation.

Unfortunately, mapped and mapping are also wrong. The problem is subject-object ordering. If the mapper were the subject, it would sound fine:
let topoMap = cartographer.mapping(mountainRange)

But we have a situation more like:
let topoMap = mountainRange.mapped(by: cartographer)

An alternative would be transformed:
let newValues = values.transformed{ $0 + 1 }

This also gives a nice spelling for the in-place variant:
values.transform{ $0 += 1 }

I think I like that even better than mutateEach.

Filter

Filter has a parity problem. In common parlance, we speak of both filtering out and filtering in. For example a content filter blocks things, and a high-pass filter allows things. However, the concept of filtering a list, such as search results, has become so widely understood even among non-programmers that I think filter has the strongest case of any as a term of art.

If we really want to use an unambiguous term, I would suggest selectAll. As in:
let passing = students.selectAll{ $0.grade >= threshold }

Moreover, sometimes we want to get both parities, so it might be worth considering:
let (passing, failing) = students.partitioned{ $0.grade >= threshold }

FlatMap

This is a really clunky name. Even people who know what it means have to stop and think about it. If we want to change it, I’d go with mapAndConcatenate since it’s really about concatenating.

Or, if we change map to transformed, then this would be transformedAndConcatenated.

Yes, it’s a lot longer than flatMap, but it’s meaningful. When someone reads transformedAndConcatenated, they will know what it does.

Heck, even keeping map as map, it still a lot easier to understand mapAndConcatenate than flatMap.

• • •

Thinking about map some more, I’m actually no longer convinced that it implies mutation. I see a similarity between “map” and mathematical operations like “plus” and “minus”.

• • •

So after writing all this out, I find that, even though I have significant misgivings about the names given to functional programming concepts, I don’t have a strong desire to change these methods in Swift.

map is fine.
reduce isn’t great, but it’s not actively harmful.
filter is technically ambiguous, but people know what filtering a list mean.
flatMap is weird, so I wouldn’t mind having a more clear spelling, but I can live with it.

The one thing I did decide though, is that I’d like the mutating in-place version of map to be spelled transform.

And that it would be nice to have a partition method for two-sided filtering.

5 Likes

With something like an array, sure. But not with something like optionals, unless you really stretch the definition of concatenating.

2 Likes

This pitch is predicated on the notion that this is a valuable thing to do as the terms of art stick out.

Let me run with that and try to think through this all the way:

Cloning and Discouragement

We clone functionality to a new name and allow the original name to enter a discouraged phase long before it is ever (if ever) deprecated. A large part of the community will continue to use the terms-of-art names but new learners, instructors, sample code makers, and the core team adopt the new name.

Those opposed will say this puts new learners at a disadvantage reading existing code. Those in favor will reply that the names are close enough to be instantly recognizable (with the possible exception of filter), that they now match the Swift terms of art elsewhere in the stdlib, and that the banhammer of autocomplete will push rapid adoption.

We could introduce more than one level of discouragement:

  • discouraged: no autocomplete
  • stronglyDiscouraged: no autocomplete, and warning-generation
  • deprecated: as is

Approach

The new names are produced as standard stdlib entries while the old names are actively discouraged, first by availability annotation (as below) and then some time in the distant future (perhaps) by deprecation.

This involves a simple update to the available annotation:

@available(swift, discouraged: 6.0, renamed: "mapping(_:)", message: "Please use the renamed mapping(_:) instead of map(_:)")

The discouraged element notes the language release in which the compiler and IDE are updated to discourage use of a given API. Is this one proposal or two?

Note that including a diagnostic message to issue warnings (treated widely as errors) plus the lack of autocomplete, adds strong pressure for rapid adoption of the new terms.

Implementation

Supporting available(..., discouraged:, ...):

  1. Parse a variant available or extend available where there is just one type where declarations can be deprecated, discouraged, or both.
  2. Hide a discouraged declaration from autocomplete
  3. Issue warning
  4. (Eventual/possible deprecation)

How much work would be involved in creating a prototype?

Expected Pushback and Motivation

We had the argument before and lost. In terms of community pushback, I would expect if so that a notable number of people will autocomplete to mapping and then manually rename to map, creating unhappy users (as there will undoubtedly be). Gradually increasing the level of discouragement may help acceptance of the change over time.

The measurable benefits are language consistency, better expectations for new learners. The outcome is a better designed language that is not fully locked into its own history with a path forward for changes deemed appropriate, and whose path is gradual and predictable.

Also, while we're at it, when (and if) we get past introducing "how to discourage" technology and onto "how should these be named", then filter is ripe for rename as selecting.

Update: Please see @Nevin 's renaming analysis.

This seems pretty firmly against the spirit of the principle of source compatibility under discussion. We have used the name map since Swift 1.0, and it is ubiquitous in Swift code, books, tutorials, answers on Stack Overflow. Changing the name of map---even slowly---antiquates all of that code for a very, very small win in naming consistency.

Doug

22 Likes

Note that a "form" version of filter already exists. It is removeAll(where:), introduced by SE-0197.

I agree with Doug that these renamings are unnecessarily disruptive given the mild inconsistency they represent, even if done over a long period.

6 Likes

…except that reverses the polarity of the predicate.

This was discussed during SE-0197 and I don't think it should be relitigated.

1 Like

I’m not relitigating anything, I’m saying your statement is incorrect.

1 Like

It's a form version of filter. "Well-actually"ing that isn't constructive.

1 Like

It does the exact opposite of filter.

The Swift standard library does not currently have an in-place version of filter. That’s just a fact.

Saying “Well actually, it has something that can be twisted into doing the same thing as filter by negating the predicate” isn’t constructive.

That fact that there is another way of writing the same thing is not relevant. I can write a for loop that does the same thing.

But no one would say “The in-place version of filter is called a for loop”.

4 Likes

We discussed both the polarity of the predicate and the merits of having both polarities as separate functions during SE-0197. The standard library has a form version of filter already, and proposing a second one to match filter's polarity is something already discussed and rejected in a previous review.

1 Like

That is already a method on MutableCollection.

1 Like

There is more than one kind of learner. I teach Swift more often to people with existing programming experience, and it’s so easy to lose someone’s interest when Swift is seen as having different names for want of a nail.

This converse can also apply for an actual beginner. Terms of art help a programmer who started with Swift know what to look for when they branch out to other languages. No matter how stringently Swift applies its guidelines to the stdlib, beginners will bootstrap their awareness of Swift APIs through memorization and pattern matching a lot more than they are going to deeply ingest the metaphors behind “mapping” or “selecting”. We should avoid doing them the disservice of feeling like they have to start all over again with a different language.

Swift has to fight the perception that it is just some silly Apple language that Apple forces you to learn, just as much as it needs to stick to the path it has charted for itself. The API guidelines are fabulous, but don’t exist in a vacuum.

21 Likes

We should focus on 'training" swift's code completion to suggest a new symbol to take precedence over an imported discouraged symbol when importing a new package.
There are a couple of issues that would need to be addressed:

  1. How do we "train" the code completion logic to pick an implementation over another?
  2. Raise the precedence of some symbols in overloading resolution implicitly via the same mechanism so that when somebody calls a symbol with that name then it gets picked up by a "shim" library that was imported.
  3. We still need a way to disambiguate overloads on the call site (out of scope)

Intellisence allows custom training that could function in a similar way as the bellow suggestions but instead of arbitrary being able to train code completion we would sort of "tag" our preference using an attributed.

We can address 1 and 2 with an attribute that will inform the overload resolution and code completion that should take precedence:
@symbolPrecedence(overloadRename: "mapping(_:)", higherThan: Swift , lowerThan: OtherLib)

  • overloadRename: The name of an method similar to the rename property
  • higherThan, lowerThan : a library name that would function in a similar way as "Precedence Group Declaration" . Hopefully this can be strongly typed in some form via import

Imagine I wanted to create new package that contains all the new suggested name changes, lets call this package SwiftBoost.

\\ SwiftBoost Library

import Swift
import OtherLib


extension CollectionOrArrayEtc {
...
@available(swift 5.3)
@symbolPrecedence(overloadRename: "mapping(_:)", higherThan: Swift , lowerThan: OtherLib)
@inlinable public func mapping<T>(_ transform: (Character) throws -> T) rethrows -> [T] { ... }

@symbolPrecedence higherThan: Swift , lowerThan: OtherLib)
@available(*, deprecated: 5.3, renamed: "mapping(_:)", message: "Please use mapping")
@inlinable public func map<T>(_ transform: (Character) throws -> T) rethrows -> [T] { ... }

...
}
// New Lib ABC
import SwiftBoost

var someArray = ...

someArray.mapping(...)
In ABC when I start typing map, the symbol mapping should be the first choice on code completion.

some.map(...)
In ABC if I were to choose map anyway then I will get a warning that is coming from SwiftBoost.

some.Swift::map(...)
some.SwiftBoost::map(...)

It would be really helpful if we could disambiguate which package the symbol is being picked up.

Reference:

https://docs.swift.org/swift-book/ReferenceManual/Declarations.html#ID380

Hey Erica,

Thank you so much for pushing on this. I would recommend separating the two issues:

  1. A proposal to extend availability to support "discouraging" APIs is a generally useful thing, either because they are not generally useful, the are obsolete but cannot be removed from ABI compatibility reasons, etc. I think we'd have to have a discussion about what discouraging means - clearly shouldn't show up in code completion, should it also be dropped from documentation, or just marked with a "discouraged" tag?

  2. should we rename map/filter/reduce? As you say, this will be controversial. I think it is mostly completely off the table without #1 happening, but even with #1 happening, it is still debatable (I was argue in favor of the rename FWIW, even though I originally argued against renaming them in the Swift 3 days).

If you agree with that characterization, then I think it makes sense to start with #1 and get it done. This is a modest proposal and a pretty simple change to the compiler. I suspect that someone would be willing to help prototype the compiler change, and if not, I could probably help in a few weeks.

-Chris

4 Likes

I’m for it.

It seems pretty simple to change my source code to be current, and also simple to not change my source code to be stubborn.

(Also it seems easy for Xcode / other tools to have a one-button “fix this codebase” to update away from discouraged terms, since assumedly they’d always be a simple rename?)

-Wil

2 Likes

When I google 'map filter reduce swift' Mr Google gives me over 3 million responses. It also suggests the same searches for javascript, python, java, c++. Are we going to change all those web sites and those other languages too? It's a rite of passage to read a blog or three about swift and map filter reduce. It's probably also a rite of passage to write one.

These method names have a long history as terms of art in other languages and have a five year history in Swift. I haven't heard any good reasons why they should be changed.

Somebody said yesterday that we should focus on actual improvements to the language so we can do more with it, not mess around renaming stuff. I agree with that.

11 Likes

Let's pick this up after dubs.