Returning to an old hobbyhorse: Migrating higher order function names to comply with API guidelines

Swift's higher order functions mostly predate the API guidelines and are based on terms of art. The community debated about this in the past and the "Term of Art" hammer won at that time. Perhaps we can reconsider now that Swift is entering its comfortable middle age reflective period as a missed opportunity.

It would be simple to alias map, filter, etc with API compliant names (mapped, mapping, filtered, filtering), slow-walk-deprecate the former with the gentlest touch, and move towards a more consistent dev-facing vocabulary by replacing the terms in the SPL docs and devdoc tutorials (with footnotes or sidebars) to establish a new standard long before removing the old.

Breaking changes have a high bar so it would take such a slow and cautious approach to migrate the community towards these changes. Backwards compatibility would need to be maintained for a longer period of time than usual. I'm curious as to what people think.

cc @Chris_Lattner3

41 Likes

I would love to see this. I don't think this needs to be a breaking change at all - we can introduce the new names, and use an (oft-speculated about, but never actually proposed) new attribute on the old names to hide them from code completion - without actually removing them. Such an attribute would be useful on other things, e.g. the initializers implementing the requirements for the LiteralConvertible protocols.

We should also consider adding the mutating form versions at the same time.

-Chris

20 Likes

Would that attribute be available in the Standard Library or would that be usable from other third-party libraries? I’m just concerned that with such an attribute not being a proposal on its own, we would just end up with another feature available only to the Standard Library. The standard library also hides identifiers starting with an underscore; a feature that’s nowhere to be seen in other libraries - except Apple’s.

As for the hiding ā€˜old’ methods part, I think we should be careful with the new names. As I see it we could make small changes such as filter(where:) to filter(keeping:) or something to that affect. Anything bigger than that and I don’t think it will be accepted as non-breaking. What I mean is that if on the previous version one was using method named x and then that same method was renamed to y, the user would be surprised. I know that technically it would still be non-breaking; still, I think we should not deviate much from the current method names - e.g. not rename filter to something else, just change the ā€˜where’ label. I also don’t understand from your post if you propose such a change being made in Swift 5 or 6. IMO we could add the new methods in the Swift 5, but keep the attribute for Swift 6.

Nonetheless, I like the proposed approach. I think it could also be adopted for similar changes in the future.

I’ll take the opposite line: in the absence of mandatory closure labels, there is no point in changing any of this naming and keeping the very bad, no good name ā€œfilterā€; it should be renamed to ā€œselectingā€ or similar.

8 Likes

We are still having issues with foundation extensions taking precedence over the standard library methods with similar names.

I would love to be able to hide symbols from code completion of imported libraries as a user of the library.

1 Like

I too would love to see this.
Even more so for drop than for map and filter.

2 Likes

What does this look like? Something like this?

@renamed(from: "map", from: 6.0, hidden: true)
@inlinable public func mapping<T>(_ transform: (Character) throws -> T) rethrows -> [T] { ... }

and how does that compare to the existing approach?

@available(swift, deprecated: 4.1, renamed: "compactMap(_:)", message: "Please use compactMap(_:) for the case where closure returns an optional value")

Map-in-place and filter-in-place? Is there a strong use-case? (Also, I think formMap and formFilter are terrible names.)

Further, I strongly prefer the -ing endings over -ed for these (mapped, filtered, reduced...)

The topic of an in-place map/mutating forEach has come up many times before. My understand is that we've moved away from that model and now we hope that the ownership model will solve this.

I can't remember the last time we introduced an in-place mutating algorithm in the standard library -- even CollectionDifference doesn't let you apply a diff in-place.

I was thinking about something simpler, ala:

@dontShowInCodeCompletion
func map(....

Having support for a renamed style attribute would also be great, but that is a more complex thing. The "don't show" attribute is needed for things that are not renamed, just internal implementation details (e.g. the literal convertible initializers).

-Chris

Actually, on further consideration, Erica I think you're raising a really good point:

The issue is that the new "mapping" symbol (or however it is spelled) may only be available on new deployment targets, maybe integrating this into the @available attribute makes sense to say it is "hidden" only on some deployment targets.

To directly answer your question, what I'm proposing is different than deprecated because it wouldn't produce warnings if you use the symbol (that would be crazy for things like map that are widely used).

The idea here is to provide a better "soft migration" step where we can nudge code to the new things gently at first. The Xcode auto-upgrade migrator tools could still apply the rename attributes, even though the symbols are not deprecated.

Here's a way to think about this - it is a new "discouraged" state in @availability.

-Chris

2 Likes

I would have loved to see a way to reconcile map and reduce with our naming guidelines back in the day when these decisions were made. It would have made the overall experience of using these methods more consistent.

However, I don’t think the issue rises to the bar of causing active harm, and I don’t think that anything has changed substantially enough in the ensuing years that merits revising a settled decision. To me, the drawbacks of any migration of a commonly used API like this would far outweigh any benefits to be gained.

8 Likes

It's also worth pointing out that the guidelines explicitly leave room for terms of art. These are some of the most canonical terms of art there are. I agree that a change is not warranted here.

4 Likes

In my experience, few things cause more harm than this.

It takes a lot of effort on our (teachers) part to get students to adhere to the API guidelines. If the Standard Library itself doesn't even adhere to them (or gets its priorities wrong), most of that effort is wasted.

Also keep in mind that terms of art don't apply to new learners. To them, naming a method "filter" because other languages name it that way makes no sense.

13 Likes

When you have a collection-of-collections (eg. [[T]]) and want to perform in-place mutation of the inner elements. With a map-in-place (I like the spelling mutateEach) that takes its argument inout, you can simply do a double-map:

outerCollection.mutateEach { innerCollection in
  innerCollection.mutateEach { $0 += 1 }
}

If you write the equivalent with out-of-place map, then you’ll end up allocating new space for every inner collection as well as the outer collection. The in-place version allocates nothing.

Agreed. The form prefixes are explicitly stated as being used when the base name is a noun (eg. formUnion). The names map and filter are verbs.

• • •

I am a strong advocate of the position that functional-programming terminology is almost entirely terrible. The names for operations are abysmal. The meanings of the words that functional programmers use, do not correspond to the actual concepts which they are applied to.

However, the one advantage of those existing names, is that they are terms of art. People know them, and have written about them, and explanations exist for them.

If we think the names are bad enough to warrant changing them, then we should choose ones that we think are better. And that means choosing names that are actually meaningful.

In particular, small changes to the existing names will be even worse than no change. That would lose the benefit of term-of-art recognizability, while retaining the problem of the words meaning the wrong thing.

The existing names are locally-optimal because they are terms of art, so if we want something better that will necessarily involve moving to a different ā€œhillā€ entirely.

• • •

If we are going to try to improve these method names, then we should come at it from the perspective of applying the best-practice Swift naming guidelines to replace the current term-of-art spellings:

Reduce

reduce is a bad name for its job. The operation is about combining elements. A more accurate name would be combine. As in:
let sum = values.combine(0, +).

Map

map is a reasonable name for its job, because mathematically a map is a transformation and colloquially we talk about mapping one thing to another, but it has a verb-tense that would imply mutation.

Unfortunately, mapped and mapping are also wrong. The problem is subject-object ordering. If the mapper were the subject, it would sound fine:
let topoMap = cartographer.mapping(mountainRange)

But we have a situation more like:
let topoMap = mountainRange.mapped(by: cartographer)

An alternative would be transformed:
let newValues = values.transformed{ $0 + 1 }

This also gives a nice spelling for the in-place variant:
values.transform{ $0 += 1 }

I think I like that even better than mutateEach.

Filter

Filter has a parity problem. In common parlance, we speak of both filtering out and filtering in. For example a content filter blocks things, and a high-pass filter allows things. However, the concept of filtering a list, such as search results, has become so widely understood even among non-programmers that I think filter has the strongest case of any as a term of art.

If we really want to use an unambiguous term, I would suggest selectAll. As in:
let passing = students.selectAll{ $0.grade >= threshold }

Moreover, sometimes we want to get both parities, so it might be worth considering:
let (passing, failing) = students.partitioned{ $0.grade >= threshold }

FlatMap

This is a really clunky name. Even people who know what it means have to stop and think about it. If we want to change it, I’d go with mapAndConcatenate since it’s really about concatenating.

Or, if we change map to transformed, then this would be transformedAndConcatenated.

Yes, it’s a lot longer than flatMap, but it’s meaningful. When someone reads transformedAndConcatenated, they will know what it does.

Heck, even keeping map as map, it still a lot easier to understand mapAndConcatenate than flatMap.

• • •

Thinking about map some more, I’m actually no longer convinced that it implies mutation. I see a similarity between ā€œmapā€ and mathematical operations like ā€œplusā€ and ā€œminusā€.

• • •

So after writing all this out, I find that, even though I have significant misgivings about the names given to functional programming concepts, I don’t have a strong desire to change these methods in Swift.

map is fine.
reduce isn’t great, but it’s not actively harmful.
filter is technically ambiguous, but people know what filtering a list mean.
flatMap is weird, so I wouldn’t mind having a more clear spelling, but I can live with it.

The one thing I did decide though, is that I’d like the mutating in-place version of map to be spelled transform.

And that it would be nice to have a partition method for two-sided filtering.

5 Likes

With something like an array, sure. But not with something like optionals, unless you really stretch the definition of concatenating.

2 Likes

This pitch is predicated on the notion that this is a valuable thing to do as the terms of art stick out.

Let me run with that and try to think through this all the way:

Cloning and Discouragement

We clone functionality to a new name and allow the original name to enter a discouraged phase long before it is ever (if ever) deprecated. A large part of the community will continue to use the terms-of-art names but new learners, instructors, sample code makers, and the core team adopt the new name.

Those opposed will say this puts new learners at a disadvantage reading existing code. Those in favor will reply that the names are close enough to be instantly recognizable (with the possible exception of filter), that they now match the Swift terms of art elsewhere in the stdlib, and that the banhammer of autocomplete will push rapid adoption.

We could introduce more than one level of discouragement:

  • discouraged: no autocomplete
  • stronglyDiscouraged: no autocomplete, and warning-generation
  • deprecated: as is

Approach

The new names are produced as standard stdlib entries while the old names are actively discouraged, first by availability annotation (as below) and then some time in the distant future (perhaps) by deprecation.

This involves a simple update to the available annotation:

@available(swift, discouraged: 6.0, renamed: "mapping(_:)", message: "Please use the renamed mapping(_:) instead of map(_:)")

The discouraged element notes the language release in which the compiler and IDE are updated to discourage use of a given API. Is this one proposal or two?

Note that including a diagnostic message to issue warnings (treated widely as errors) plus the lack of autocomplete, adds strong pressure for rapid adoption of the new terms.

Implementation

Supporting available(..., discouraged:, ...):

  1. Parse a variant available or extend available where there is just one type where declarations can be deprecated, discouraged, or both.
  2. Hide a discouraged declaration from autocomplete
  3. Issue warning
  4. (Eventual/possible deprecation)

How much work would be involved in creating a prototype?

Expected Pushback and Motivation

We had the argument before and lost. In terms of community pushback, I would expect if so that a notable number of people will autocomplete to mapping and then manually rename to map, creating unhappy users (as there will undoubtedly be). Gradually increasing the level of discouragement may help acceptance of the change over time.

The measurable benefits are language consistency, better expectations for new learners. The outcome is a better designed language that is not fully locked into its own history with a path forward for changes deemed appropriate, and whose path is gradual and predictable.

Also, while we're at it, when (and if) we get past introducing "how to discourage" technology and onto "how should these be named", then filter is ripe for rename as selecting.

Update: Please see @Nevin 's renaming analysis.

This seems pretty firmly against the spirit of the principle of source compatibility under discussion. We have used the name map since Swift 1.0, and it is ubiquitous in Swift code, books, tutorials, answers on Stack Overflow. Changing the name of map---even slowly---antiquates all of that code for a very, very small win in naming consistency.

Doug

22 Likes

Note that a "form" version of filter already exists. It is removeAll(where:), introduced by SE-0197.

I agree with Doug that these renamings are unnecessarily disruptive given the mild inconsistency they represent, even if done over a long period.

6 Likes

…except that reverses the polarity of the predicate.

This was discussed during SE-0197 and I don't think it should be relitigated.

1 Like