Principles for Trailing Closure Evolution Proposals

Evolution Discussion
1

Hi Swift Evolution,

During the discussion on the acceptance thread of SE-0279, the core team described how SE-0279 does not rule out future proposals aimed at improving the state of trailing closure syntax, and encouraged further discussion of follow-on proposals be broken out into new threads on the evolution forum.

One potential follow-on proposal is currently being pitched. But the design space for these follow-on proposals is large, and not all have the same characteristic of keeping future directions viable. The core team thinks a wider discussion of the different alternatives would be better had prior to moving any of them to the pitch phase.

To help guide this discussion, the core team thinks it would help to set some principles against which these proposals (or more generally, any change to the language) could be evaluated. Below are three principles the core team thinks are key to evaluating these proposals. For the most part, this is consolidating advice the core team has previously given, in the SE-0279 thread and elsewhere. We would love the community’s feedback on these principles, as well as any additional principles that should be considered.

Principles for Evolution Proposal Evaluation

Source Stability

Swift is now an established language, and there are limits to what changes we can accept.

Source compatibility between different versions of the compiler must be preserved, without any need for migration in all but very rare circumstances (such as fixing serious correctness bugs). This is critical for use cases such as Swift packages, playgrounds, or ABI-stable module interfaces, where even a tool-assisted migration is not viable.

In order to allow source-breaking changes, major versions of the Swift compiler may provide compatibility-preserving language version modes. This allows for adoption of new syntax over time. But these version modes should still be used sparingly. Because source compatibility must be preserved, each version must continue to operate indefinitely. This runs the risk of creating "old" and "new" styles of Swift, both of which may exist alongside each other for long periods of time. Having to apply changes may also discourage users from updating to the newer language version, inhibiting adoption of other features.

The impact on API surface of these versioned source breaks must also be considered. A change to Swift may encourage a “new” style of API that makes many existing APIs look outdated. This in turn may require APIs to churn more rapidly, causing knock-on problems in areas such as maintenance burden of the old and new styles of API, outdated documentation and stack overflow answers, or a need to bump major semantic versions in packages.

For these reasons, source-breaking changes, even versioned ones introduced with a migration plan, must clear a very high bar in terms of the value they add. Most source breaks can be worked around with enough energy in the compiler. So a proposer may feel they can break source if they feel the result is better, and it becomes a debate about whether it is better. But this is secondary if it’s only marginally better, if it runs the risk of bifurcating the language over a long migration.

Examples

A proposal to add language-level asynchrony to Swift is likely to render many callback-based APIs obsolete. However, the dramatic improvement in user experience brought by such a feature would certainly be considered worthwhile despite this burden on API authors to update their code to the new style. So long as the old-style callback APIs remain available, no source breakage should occur.

subscript currently deviates from regular function declaration, in that parameter names are not argument labels. Instead you must use explicit argument labels. This is confusing, and might be better made consistent, but would cause extensive source breakage. While this could be language versioned, it would cause considerable source churn and block adoption of other Swift features in the same version. The consistency benefits are therefore clearly outweighed by the cost.

Control of the Call Site

Swift tends to leave discretion about the appearance of the call site, and especially the use of a label, to the the API author rather than the caller.

This principle is key to helping API authors preserve clarity at the point of use. If there are multiple ways a client can choose to call your API, all possibilities must be considered and designed for, making the design of a clear API harder.

As stated in the acceptance of SE-0279, the core team is not satisfied with the basic design of trailing closures in Swift today, particularly in the way it deviates from this principle. If we were free to redesign the language without restrictions, we would likely do this feature differently. A more consistent design could, for example, only permit trailing closures at all when the API author uses a label of _ for the closure. But as outlined above, changing to such a design at this point would run counter to Swift’s principle of source stability.

More control at the call site also provides for more accurate method lookup, and therefore more predictable behavior and faster compile times, all things critical to the Swift developer experience.

Example

Prior to Swift 3, callers could supply defaulted arguments in any order. SE-0060 removed this ability, requiring arguments appear in the order of the function declaration. This put ordering under the control of the API author, and increased consistency at the call site.

Language Consistency and Future Directions

Evolution proposals should be consistent with the existing language. They should leave open further complimentary proposals.

The proposal for SE-0279 describes its design for multiple trailing closures as an extension of the existing rules for single trailing closures. With both single and multiple closures, the first trailing closure has no label. In the case of multiple trailing closures, additional labeled closures can then follow the initial unlabeled closure.

It also left open other future directions. Mandatory or optional labels, or enforced or banned trailing closures for a particular function, remain as feasible as they were with single trailing closures (which is to say, still not especially feasible but not ruled out).

This is a good combination: extending existing patterns in the language as an improvement that stands on its own, without requiring or closing off future directions. Depending on how these play out, SE-0279 is either the final state of this part of the language, or an acceptable resting state that we move on from in a later release. Either of those is an acceptable outcome.

By taking this approach—of extending the language in a consistent way while leaving open future directions—we are able to continue to evolve the language over time.

Examples

The original proposal for SE-0279 introduced an “outer brace” syntax for multiple trailing closures. Unlike the amended proposal, this did not extend an existing language pattern but instead introduced a new alternative calling syntax. It left no real path for resolving the discrepancy in the future.

SE-0255 extended the existing precedent of implicit return for single expression closures to be useable in all functions. As such it was a consistent extension of the existing language. It left open the future possible direction of building on it with a complimentary proposal making if and switch statements into expressions. Such a proposal would complement SE-0255, but SE-0255 stands alone without this proposal as a useful improvement to the language in itself.

–––

Applying the principles to trailing closures

In accepting SE-0279, the core team stated that it felt that the proposal did not block the path for future proposals around the labeling of trailing closures. But at the same time, we considered the path for several of those follow-on proposals to be challenging.

These principles are meant to give a framework for evaluating those challenges. Given the above principles, how might some of the candidate proposals regarding trailing closures be evaluated? Some possible proposals---including one that has been worked up as a full pitch---are evaluated below.

The application of these principles is not meant to be absolute. As shown especially in the source breaking case, a principle can be overridden if enough motivation is shown. But that motivation must be along the lines of significant new functionality or elimination of significant active harm.

The core team would welcome feedback on the principle application below, as well as any comment on the principles themselves, or further principles for consideration.

Mandating argument labels by default

A proposal to mandate the use of labels on trailing closures where the argument has a label would incur significant source breakage. So at a minimum this change could only be introduced through versioning, with a potentially lengthy migration period needed for callers to update their code to use the label. Many APIs have been designed with a label, but assuming the label will usually be omitted, and may need to be updated to avoid callers needing to supply an unnecessary label. Since this change would not bring significant new functionality to Swift, it would be hard to justify on this basis.

Allowing optional labels

Allowing the caller to optionally include a label for the first trailing closure has been pitched as an enhancement on top of SE-0279. This pitch has the benefit of allowing multiple closures of equal importance (the canonical example being Binding(get:set:)) to label both closures. But giving this option to include a label or not at the call site to the caller violates the principle that the API author should control the appearance of the call site.

Controlling trailing closures via _ labels

During the second review of SE-0279, it was suggested that multiple trailing closures be introduced using _ on the label to control if a trailing closure were allowed, or to require the label on the first trailing closure. This could only apply to multiple trailing closures: single trailing closures would still have no label. This avoids the problems with source stability (multiple trailing closures are brand new) and with giving the API author control. But it leaves the language in an inconsistent state, and making this decision around multiple trailing closures would force the language down a specific path if we also wanted to solve the problem for single trailing closures.

Controlling trailing closures via an attribute

The introduction of an attribute to block the use of trailing closures on specific APIs is another potential solution to the Binding(get:set:) problem. While not as consistent as the use of _ would have been, it does return more control to the API author. The use of this attribute would have to be versioned to avoid a break in source stability, but could be used as a guide to IDEs for code completion even in earlier language versions.

41 Likes
2

@Ben_Cohen I just want to say - thank you for writing this up. This type of messaging is exactly what I felt was missing for awhile in Swift Evolution (perhaps the only issue was the fragmented sharing).

9 Likes
3

Great topic to put a light on the evolution process! I think another possible solution is the introduction of an attribute to enforce a label on the first trailing closure for functions with multiple closures. It gives the API Author control while maintaining the benefits of trailing closure. Furthermore, it doesn't introduces the inconsistent behavior of using _ to control trailing closures and and I believe it won't affect previously designed APIs.

4

Thank you for providing this insight into the core team's thought process.

Regarding the following:

Classic trailing closures have always violated this principle. For example, the author of UIView.animate() has no control over whether the caller uses the animations label or just provides a closure after the ). Classic call-site flexibility seems congruent with the proposed optional-label flexibility.

9 Likes
5

I agree in the single closure case, but the multiple closure syntax is new and therefore has no source to break. Requiring labels only in the multi-closure case seems to meet the principles outlined. The single closure case can be left as a vestigial bit of syntax to be dealt with in the future. A small inconsistency is a small price to pay for greater overall consistency, syntactic flexibly, and better clarity. At the same time, leaving the single closure syntax as the sole unlabeled closure syntax will provide a clear path forward when the time can be spent to provide source migration or make the breaking change.

Author control is a new principle, isn't it? It was discussed during the reviews of 0279, but AFAIK, has never been a Swift design principle, and (again AFAIK) wasn't considered as part of the acceptance. It was mentioned more as something that 0279 didn't provide but could in order to meet the apparent requirement of being able to drop the first label altogether while not completely losing the meaning of the first closure. Also, 0060 was pitched as the removal of exceptional syntax from Swift and author control was never mentioned. To now apply this as a design principle seems... overly restrictive.

Personally, I'm not a huge fan of adding an attribute or other markup to the language just to reverse a syntactic default, especially when the attribute should be used more often than the default to preserve clarity.

There seem to be a variety of optional or required label paths forward available to Swift that are largely dismissed here, but that's not my main concern.

In general, I think the biggest thing missing from this document is a discussion how closure syntax can be made more clear.

Clarity is often discussed as Swift's highest design principle. On these forums, in the naming guidelines, and in WWDC presentations, clarity comes first. This makes Swift more verbose than other comparable languages, but can give it an elegance that those languages lack. Clarity makes Swift easier to learn, easier to read, and easier to reason about.

Unlabeled trailing closure syntax, as was mentioned many times in the reviews of 0279, is not clear. Yet 0279 leaves the language in an awkward place. With source compatibility on one side (unlabeled single trailing closure syntax) and the new requirement of an unlabeled first closure on the other, there is no path to clarity in the language today. And barring any changes to Swift 5.3, there won't be any way out, for users or API designers, until at least Swift 5.4 (likely in the Spring of 2021). In fact, the optional label pitch thread (as well as the reviews of 0279 and discussion of standard library API) offer a variety examples of code made less clear with the new (and existing) syntax.

So, how does the core team see unlabeled closure syntax intersecting with clarity at point of use?

Also, now that users can try the 0279 syntax in the toolchains, how will their experiences be integrated into these principles? 0279's syntax has a variety of issues beyond clarity. What's the appropriate place to bring those up?

22 Likes
6

Since this is already the case in Swift 5.2, it seems the best course of action would be to extend this to all parameters for consistency in order to fix SE-0279 ASAP, and then think about a better proposal for Swift 5.4/6.

7

It is not a new principle; it dates from some of our earliest discussions of the language with framework teams at Apple, long before Swift was announced, much less open-sourced. It was one of the things about Objective-C that people most strongly wanted to make sure we didn't lose. We may not have explicitly invoked it much, but it's always been there.

7 Likes
8

They have. Which is why the core team has previously said it is dissatisfied with the basic design of trailing closures.

The fact that they violate this principle should therefore not be considered a basis to accept other proposals that go against it too.

5 Likes
9

Single trailing closures comprise the vast vast majority of cases. They cannot be considered as “a vestigial bit of syntax.” If the labeling of trailing closures is to be tackled, it should be tackled for single trailing closures first and foremost, with multiple trailing closures following along in a consistent fashion.

I quote again what we wrote in interpreting this suggestion versus the principles:

[altering the rules only for multiple trailing closures] leaves the language in an inconsistent state, and making this decision around multiple trailing closures would force the language down a specific path if we also wanted to solve the problem for single trailing closures.

It may leave a “clear” path forward, but it’s one that is unlikely to be reconcilable with the principle of source stability. So it’s a proposal for permanent inconsistency, and as we lay out above, that isn’t ok.

10

I understand that the core team seems to prefer going the extra attribute route rather than the optional first label. That immediately raises a few questions. If source compatibility sets a new boundary, how will an attribute solve the issues for Binding(get:set:) and first(where:) in a retroactive way? As far as I can tell SE0279 will allow people to write unintentional combinations for APIs with multiple trailing closure possibilities. Adding an attribute retroactively to limit the options will break source compatibility. Having an attribute to limit the options, is one thing, but what about the API‘s with only one trailing closure that already have a label? Transforming first(where:) into firstWhere seems like a workaround for the problem without acknowledging a better solution which the design in SE0279 introduced for the last trailing closures.

3 Likes
11

I share some of the concerns touched on by @DevAndArtist just now. I'll elaborate below.

I appreciate that this conversation is happening right now. I learned a lot by reading over the OP in this thread that will make me a better API designer period. To frame up the following, I am generally in favor of the syntax settled on for multiple trailing closures. That said, I think it is incumbent upon us (the community and the core team and language maintainers) to move quickly on follow-ups for trailing closures because multiple trailing closures have already been accepted but we don't yet have a good solution to the call-site clarity or API designer control over call-site legibility.

My thesis is that the middle-ground we have now is going to result in very real downsides in the short term and I am not convinced we can clean those downsides up easily because APIs will be designed around what we have today, not what we hope to have in the future, and APIs will be used the best they can be despite future changes potentially resulting in breaking changes to those call-sites. Note that here I am not referring to breaking changes mandated by Swift Evolution, but rather breaking changes to library APIs as library authors gain hypothetical tools like attributes that force labels to be explicit.

I think it is reasonable to say that multiple trailing closures do not actually maintain the status quo even if they ostensibly follow the same rules as single trailing closure already did. Where before an API designer needed to assume that the caller might omit the argument label for the last closure, now the problem has multiplied to the API designer needing to assume that any of the argument labels for the n last closures might be omitted; note, I am not saying that all of the arguments can be omitted, but rather that any one of them might be, depending on where the call-site begins using trailing closures. This was illustrated well here under the details of Jordan's "more nebulous note."

It is really unfortunate to be faced with the questions: Do I design my API with the best possible multiple trailing closure experience or do I design my API so that call sites are never confusing? Is it responsible of me to recommend trailing closure syntax (by way of example code) in my documentation given the fact that some call sites will benefit and others will not?

I have already experienced this problem in designing my own APIs in the past weeks and trying to think about how they will interact with multiple trailing closure syntax. It puts us in an awkward position: We can design something that works really well sometimes and not very well at all other times and it is not even strictly speaking always a matter of the end-user's stylistic preference. I'll give a bit of a silly example in the expansion below.

Example

I have a function with the following signature:

public func validate<T>(
    _ description: String,
    check validate: @escaping (ValidationContext<T>) -> Bool,
    when predicate: @escaping (ValidationContext<T>) -> Bool
)

Multiple trailing closures can make the call-site quite pretty (in my opinion):

validate("Two Ones Make a Two") {
    Two($0.oneValues) != nil
} when: { 
    $0.oneValues.count == 2
}

However, if complicated validations start to become smart to write as named closures, the call-site loses the benefit of explicitly labeling the last closure as "when" unless the user decides to arbitrarily hold themselves to the rule that "calling validate() with one trailing closure is disallowed but calling it with two trailing closures is preferred":

let allOnesAdd: (ValidationContext<T>) -> Bool = ...

validate("All Ones Add Up", check: allOnesAdd) { 
    $0.oneValues.count > 0
}

Obviously, this is analogous to the existing decision an end-user must make of whether or not to use single trailing closures based on call-site clarity, but now as an API designer I can no longer just "assume any given user will not use the last closure argument label." Instead I must assume that "some users will use it and some users won't" and "some users will prefer to use it or not depending on the circumstances."

19 Likes
12

Practical Application of the No-Breakage Principle

In the context of releasing a hypothetical Swift 5.4, will this principle be enforced strictly if/when making changes to syntax newly-created in Swift 5.3? In other words, if the best path forward involves revising multiple-trailing closure syntax in a way that breaks source code written during the initial lifetime of Swift 5.3, will the Core Team view that sort of breakage in the same way that it might view a breakage of syntax that has been present since Swift 3.0?

I ask this question because I share the concerns raised by @DevAndArtist, @mattpolzin and others, in this thread and elsewhere. In those concerns, there may exist a very strong motivation for revising multiple-trailing closure syntax in a manner that may need to break Swift 5.3 code.

Experimental Features

I've noticed some other languages label new features as "experimental." For instance, see this article re: Node.js introducing features on an experimental basis. For a mature language, this practice seems like a prudent and appropriate mechanism whereby new features can be added, used, evaluated and revised or removed without worrying about source breakage resulting from users utilizing the new feature.

Might Swift adopt a similar approach for purposes of phasing in some (or perhaps all) new language features?

13 Likes
13

That the core team is so loathe to do what the entire community is asking, simply add optional labels for the first closure and leave the rest to be addressed in a future proposal, makes me think there is something else coming in WWDC that they can't divulge yet. I can't understand why else there would be so much resistance so something so obvious.

14

@ebg, obviously you're frustrated. While I'm sympathetic in concept, it is hard enough for the Core Team to do what they do without also having to bear comments so harshly and hyperbolically phrased. Let's take it easy, please.

To all, I'd like to ask the favor that, in this thread, we refrain from addressing this suppositional aspect of the matter.

19 Likes
15

If we were to add an attribute to control first-closure-label-requiredness, violations would have to be warnings (or there would have to be a warning form of the attribute) in order to allow existing APIs to adopt it while maintaining source compatibility.

This bring the case, would it actually be worse to make labels “required” with a warning (i.e. strongly recommended but not source-breaking) by default? There could be an attribute to specify that a particular label is not required, which could be adopted in most or all cases by the standard library and Apple frameworks until they have been audited on a case-by-case basis.

There might also be an attribute saying that this particular label is truly required (eliding it is an error), but I personally think it would be better to specify that at a file or module level … somehow.

Ideally, this would lead to a state where new methods intended to be used with unlabelled closures use _:, ones intended to be used with labels just have labels, and attributes are only used for “legacy” methods or when the author explicitly wants to allow call site discretion.

In the distant future it might even feel reasonable to make the warning an error, but that wouldn’t be part of the initial proposal.

12 Likes
16

If we're again wanting the API author to have control, I'd expect it to be something like this strawman syntax:

@requireClosureLabels(warning) // SemVer Minor compliant
// @requireClosureLabels(error) // SemVer Major
func foo(do workClosure: () -> Void, when conditionClosure () -> Bool))
17

I’m a bit sceptical about using attributes here.

AIUI our end goal (at least in a perfect world) is this:

func foo(bar closure: () -> ()) {}
func baz(_ closure: () -> ()) {}

foo bar: {
    // ...
}

baz {
    // ...
}

IMHO, this is intuitive behaviour and makes Swift truly consistent.

I think we can reach this, by making the first trailing closure label optional immediately, eventually deprecating the usage without a label and emitting a compiler warning, and lastly — after enough time for API adoption — making usage without a label an error.

If we used an attribute for the purpose of requiring the label now, we would make (or leave?) Swift very inconsistent in this area. We already have a feature for requiring or disallowing argument labels at the call site, we control that using _:.
I can also see problems with an attribute as an intermediate solution. Firstly it only moves the problem of actually switching to our desired behaviour into the future, and secondly I’m pretty sure, that we will have to keep this attribute in the language forever after introducing it.

In conclusion I would say that we rather go the way optional first label -> warning -> time -> error, than using some attribute.
If the core team doesn’t want any source-breakage in this context, an attribute isn’t a true alternative either, because an API adopting it is breaking sources as well.

9 Likes
18

To be clear, this proposal violates the "Source Stability" and "Control of the Call Site" principles proposed by the core team at the beginning of this thread. I think you are arguing that eventual consistency of the model is more important than those principles. I consider that a hard sell given the importance of source stability for the overall Swift ecosystem. Source stability is a hard constraint to stomach, because it means we must accept that some mistakes we made in the past can no longer be fixed, but Swift developers have told "us" (as developers of the Swift language, core team members, etc.) repeatedly that it is important that we not break their code from one release to the next.

The core team has stated that they do not want to break source compatibility here, which means that existing code needs to continue to compile. API authors are free to decide themselves whether they want to adopt a feature that will break their clients; that's outside of the scope of the language's source compatibility.

Doug

19

Would it be appropriate to make a change like this in a new language version? That way existing code can continue to compile in compatibility mode.

2 Likes
20

Going back to the OP, it does say that it's not impossible, but the bar would still be very high: