[Pitch] Elide `some` in Swift 6

Evolution Pitches
1

Hello Swift community,

I have a pitch for you about changing the default meaning for plain protocols. Please share any thoughts, questions or suggestions!

  • Proposal: SE-NNNN
  • Authors: Angela Laar, Holly Borla
  • Review Manager: TBD
  • Status: Awaiting implementation, implemented on main behind the experimental flag -enable-experimental-feature ImplicitSome

Previous Swift evolution discussion: [Discussion] Eliding `some` in Swift 6

Introduction

Currently, the default for writing a plain protocol name in type position is any, but many of these implicit any uses could be replaced with some, giving them more type information while still behaving correctly. This proposal flips the default so that when writing plain protocols, the type would default to some instead of any. Making the default some guarantees a fixed underlying type which preserves static type relationships to the underlying type, giving you full access to the protocol requirements and extension methods that use Self and associated types.

Motivation

Swift has been working towards improving UI for generics for some time with the introduction of the some keyword in Swift 5.1 to represent opaque types ā€” abstract type placeholder for a specific concrete type ā€” which was extended to parameters in Swift 5.7 type such that:

func foo<T>(_ bar: T) where T: Equatable { }

// is equivalent to 

func foo(_ bar: some T) { }

Swift 5.6 introduced explicit any, which will be required in the Swift 6 language mode, to ensure that opting-in to type erasure instead of using type parameters is explicit and deliberate. Introducing the requirement of using explicit any and encouraging writing some by default opens an opportunity to make some the default for plain protocol names.

Generic code is already simplified in more places than you think. Take protocol extensions, for example. To write generic code applicable to any concrete type conforming to a protocol, you only need to write the extension keyword and the plain protocol name. In this example we are working with the Collection protocol:

*// What you write to extend all concrete types conforming to 'Collection':*
extension Collection { ... }

In generic code, a generic signature represents generic parameters and any requirements on those parameters. In the case of protocol extensions, thereā€™s an implicit type parameter called Self and single conformance requirement Self: Collection that are added by the compiler instead of needing to be written by you. This allows you to access all protocol requirements, associated types, and other extension methods from the Collection protocol on the conforming Self type.

It is much easier for programmers to learn generic programming when there are stepping stones that donā€™t require internalizing generic signatures with where clauses. Programmers can use more straightforward and intuitive syntax to express the same thing. Swift 6 could apply this same principle to plain protocol names in other contexts. Such a practice could be invaluable for beginners learning Swift, removing the mental load of comparing tradeoffs between some and any when adding protocols to their code. Beginners donā€™t need to decide between using some vs. any, postponing the need to fully understanding the semantic differences until it becomes absolutely necessary to choose between the two. Even if youā€™re not a beginner, some as the default will still improve readability of your code by making it more concise.

Why is some a better default?

any provides a common supertype for all concrete types that meet the given requirements. Using any is a form of type erasure that allows you to store any arbitrary subtype dynamically by using a boxed representation in memory. Type erasure has a number of use cases where you need the ability to change or mix different underlying types, such as:

  • Local let -constants with different initial types across different control-flow paths and vars whose underlying type changes

  • Heterogeneous collections, e.g. [any DataSourceObserver]

  • Optionals that are assigned an underlying type later, e.g. var delegate: (any UITableViewDelegate)? = nil

  • Stored properties of types that use a protocol abstraction, but the enclosing type is not parameterized. The delegate pattern above is a common example of abstraction as an implementation detail of a concrete type.

Protocol requirements and extension methods are not available for any types. If the underlying type can vary, so can the protocol requirements that depend on that type. So there is no way to know what the generic signature of a requirement using Self or associated types is, in a contravariant or invariant position. For this reason, programmers are encouraged to write some by default and change some to any when storage flexibility is needed. This workflow is similar to writing let constants by default until mutation is necessary.

Writing any explicitly is more important because programmers need to understand the type-erasing semantics and the limitations of using an existential type, including the ability to change the underlying type, the inability to access Self and associated type requirements, etc.

In many use cases for some, itā€™s already evident in the code that the underlying type will never change. For example, a let constant or a non-inout parameter already indicates that the underlying type cannot change. Similarly, opaque result types in single-return methods and result builders clearly have one underlying return type. In these cases, writing some would be redundant.

Easing the migration to Swift 6

Swift 6 will require explicit existential syntax introduced in SE-0335. Bare protocols as types will no longer compile; they must be prefixed with either the any keyword or the some keyword:

protocol P {}

struct S {}
 
let y: P = S() // error
let x: any P = S() // okay

If, instead, we switch the plain protocol syntax to mean some instead of any, code churn will significantly decrease. This would mean in Swift 6 plain protocol name P could still be valid code in many cases, but it would mean some P instead of any P.

Consider the following function. Currently, the function parameter uses implicit any, where some would be a better fit. Since the underlying type is not expected to change, there is no need to opt-in to type erasing behavior.

public protocol BlogPost { ... }

public func controller(for post: /*implicit any*/ BlogPost) -> BlogPostDataController {
  let key = CacheKey(id: post.id)
  if let postController = cache[key] as? BlogPostDataController { return postController }
  let postController = BlogPostDataController(post: post)
  cache[key] = postController
  return postController
}

// Example call-site
let controller = controller(for: Tutorial(...))

If the default for a plain protocol name changes to some, this function will behave the same way with more type information preserved at compile-time, and notably, the code will not require any changes when upgrading to Swift 6:

public func controller(for post: /*implicit some*/ BlogPost) -> BlogPostDataController {
  let key = CacheKey(id: post.id)
  if let postController = cache[key] as? BlogPostDataController { return postController }
  let postController = BlogPostDataController(post: post)
  cache[key] = postController
  return postController
}

// Example call-site
let controller = controller(for: Tutorial(...))

Using implicit some, this codeā€™s intention is still clear; a concrete type conforming to the BlogPost protocol is needed to query the cache and create instances of BlogPostDataController.

Proposed solution

Instead of always writing some or any in Swift 6, I propose we make generic code for protocols more concise and approachable by eliding the some keyword. With this approach, type annotations will be more lightweight and readable.

For example in Swift 5.8, you can write a zip function using the some keyword for both arguments and the result type:

func zip<E1, E2>(_: some Sequence<E1>, _: some Sequence<E2>) -> some Sequence<(E1, E2)>

Eliding the some keyword, this declaration would read:

func zip<E1, E2>(_: Sequence<E1>, _: Sequence<E2>) -> Sequence<(E1, E2)>

The above declaration is conceptually the same as using explicit type parameters for each appearance of Sequence:

func zip<S1, S2, E1, E2>(_: S1, _: S2) -> some Sequence<(E1, E2)>
  where S1: Sequence, S2: Sequence, S1.Element == E1, S2.Element == E2

Detailed design

A plain protocol name P, or typealias thereof, resolves as some P in the following contexts (some of which will produce a compiler error per the restrictions on opaque types):

  • Function parameter lists, e.g. func f(_: P)
  • Function return types, e.g. func f() -> P
  • Local variables, e.g. let x: P = ...
  • Generic argument lists, e.g. let x: Generic<P> = ...
  • Primary associated type argument lists for opaque types, e.g. let x: some Collection<P> = ...
  • Same-type requirements, e.g. T.Element == P

The same restrictions on opaque types apply to plain protocol names. There are some places where you cannot use an implicit type parameter, including:

  • Primary associated type argument lists of existential types, e.g. any Collection<some P>
  • Variable declarations without an initial value, e.g. let value: some P
  • Generic requirements, e.g. T.Element == some P

Plain protocol names are still interpreted as conformance requirements in the following contexts:

  • Inheritance lists, e.g. P in struct S: P {}
  • Generic conformance requirements, e.g. P in func test<T>() where T: P
  • The right hand side of a type-alias declaration, e.g. P in typealias Alias = P

Source compatibility

On its own, this change is not source-breaking. This change will be advantageous to programmers when Swift 6 rolls out. Swift 6 will enforce SE-0335 for all existential types, blocking the use of plain protocol syntax in any Swift codebase. The change laid out in this proposal will allow such code to compile in the supported positions and improve semantics instead of causing a compiler error.

Effect on ABI stability

This proposal has no impact on ABI stability.

Effect on API resilience

Packages and libraries that have not migrated to Swift 6 will still build with their existing language mode, so they will be unaffected by this change. However, when resilient libraries adopt Swift 6, they must adopt the any keyword in public API that uses bare protocol names today, because changing an existential type to a type parameter is a non-resilient change.

Alternatives considered

A common suggestion is to formalize the heuristics for when some is most valid and use those heuristics as rules for defaulting to some in some instances and defaulting to any in others. However, this model would re-introduce the conflation between the semantics of existential and opaque types, leading to a frustrating developer experience in the face of refactoring and code evolution. Seemingly harmless changes, such as factoring a local variable into a stored property, would end up changing the type of values unexpectedly and cause programmers to re-structure code that is unrelated to the refactoring task at hand, such as moving code that operated on that local variable into a separate function accepting some.

Future Directions

  • Implicitly open existentials in more places. For example, the language could default to some for local variable assignments from an any type without a type annotation, where itā€™s possible to open the existential value:futr
    let observers: [any DataSourceObserver]
    for observer in observers {
          // The default for `observer` could be (some) `DataSourceObserver`,*
         // which means the scope of this for-loop would be a context*
         // where you have access to Self and associated types
          observer.method() // where 'method' uses an associated type
    }
  • Allow coercions to opaque types, e.g. value as some P
  • Allow some P? as a shorthand for Optional<some P> for both some and any
35 Likes
2

I really like this as the new default and the fact that it will reduce code churn.

Why this restriction though? I can understand why for a property declaration there would need to be a prefix, but within a function can this not default to some too?

Variable declarations without an initial value

3

Thanks for pushing this discussion forward @angela-laar!

I read back through the old thread to refresh my memory and I find that my opinion hasn't changed too much. It would be really nice if we could mitigate the future source break caused by requiring any for existentials, but I am not yet convinced that this is the way to do it.

There's two different questions to answer here: is having P serve as shorthand for some P right for Swift, and is Swift 6 the right time to make such a change? These questions aren't cleanly separableā€”it might be that the changed pitch here is right for Swift precisely because of the source break from SE-0335 that we have slated for Swift 6, but the change wouldn't stand on its own otherwise. And of course if the answer to the first question is "no," then the second question is moot. But let me try to elaborate on my current feelings about each of them.

Should P mean some P?

I'm sympathetic to the point that often times users 'should' use generics when they use an existential because the latter has the more available syntax. I also mostly agree with Ben's position in the previous thread that explaining to users the difference between some and any is easier once the user has actually attempted to use a function in a way that generics don't support.

However, I'm also troubled by the fact that today, generic functions (and opaque type values) do behave differently than 'normal' declarations, and while perhaps the best time for the author to discover those differences is once they try to actually use their functions, I don't know that I buy the same is true for arbitrary clients. One of my hobby horses is maintaining the ability to reason locally about code and declarations, without having to click through multiple levels of documentation. By having some P look like any other type, we lose the ability to look at a function or property and understand basic things about it's behavior (can I form a reference to this function which takes a parameter of type J? Can I assign the result of this J-typed property to this other J-typed property?)

So while it might be easier, pedagogically, to explain to the user what the difference between some and any is once they've actually used the declaration in a way that makes the difference matters, I don't believe this justifies making the language more opaque for users reading code who already understand the difference perfectly well. We have tools like educational notes to (hopefully) provide users with the context and examples they might need to understand why they have to choose between any and some. Or, we could just immediately offer a fix-it for some P alone when users try to use bare P for minimal friction, and then only expose them to the any suggestion when they try to use the declaration in a way that would require existentials.

Another downside to consider is all the existing documentation and online discussions using bare P and explaining the semantics which will suddenly become outdated and in many cases, incorrect. This is related to my discussion below about changing the meaning of existing code, but in some ways worse because even with an intervening language version, this code won't ever have a chance to migrate. IMO it is a better experience to copy code from an online resource and have it no longer compile (with an error message explaining why) than to copy code and have it behave in subtly different ways than described by the source.

Perhaps these downsides don't outweigh the benefits of bare P as some P outweighs these costs, but it's still not obvious to me that this change would make things better, even in a vacuum.

Is Swift 6 the right time?

Of course, we have to work within the bounds of the language that we have, and if we are of the mind that we really must remove the meaning of P as any P, then it might make sense for us to accept an otherwise sub-optimal solution for the benefit of easing the source compatibility story.

But I'm still not convinced on this front. I will admit that I was surprised to see this passage:

and it took me a moment to understand what you're getting at hereā€”is the idea that because SE-0335 has already 'committed' to the 'no bare P' allowed source break, this pitch would only be 'making invalid code valid' and therefore claiming itself as source compatible?

I'm not totally on board with this characterization. Since this pitch would ship in the same language version as SE-0335, we have to look at the effect of both combined. There are at least some situations where this proposal would, under Swift 6, cause the behavior of existing code to change:

protocol P {}
protocol Q: P {}
struct S: Q {}

func f(_: any P) {
    print("any P")
}
func f<T: Q>(_: T) {
    print("some Q")
}

func g(_: Q) {
    print("any Q")
}

func g(_: any P) {
    print("any P")
}

f(S()) // any P
g(S()) // any Q

Under Swift 6, the above call to g would begin printing any P instead of any Q, and this is a problem that wouldn't arise with the status quo of SE-0335. And users who wish to avoid this sort of issue have no choice but to carefully inspect every use of existentials in their program to make sure that they won't accidentally start behaving differently. I think we need to give some serious thought to such issues before we forge ahead with a solution like this. (And if that thought has already been given it would be great to see it elaborated on in the proposal. :slightly_smiling_face:)

If we instead allowed for a transition period, we would, yes, have a much larger source break, but it would be entirely mechanical and easily applied by an automated migratorā€”every use of P becomes any P, full stop, and your program behaves exactly the way it did before. The certainty of this migration is very appealing to me even if the volume of changes required is more substantial. (I for one would much prefer making the transition in this manner, in codebases that I maintain.)

I suppose that with the pitched solution we would potentially be able to offer users a choice: either change all uses of P to any P, or alternatively leave all your code as-is and let the chips fall where they may. Of course, due to the differences between generic declarations those that use existentials, it's possible users will then have individual cases they'll have to track down and convert to using any because they were forming a function reference somewhere. Maybe we would be able to automate this to a certain level of polish, but my gut tells me that this approach would on the whole require more manual intervention to make the Swift 6 migration in any reasonably large codebase than would the status quo migration under SE-0335.

Of course, I'm sure I've given less thought to this than the authors and have nothing empirical to back up my gut feeling, so if the authors have already examined the migration problem in detail I would also love to see this built upon in the proposal text!

One last random question:

So, to be clear, this means resilient libraries would be prohibited from using the 'bare P' shorthand in any public API? I think this ought to be discussed in Detailed design rather than only in the resilience section.

17 Likes
4

I have not yet read any of your post in-depth except for this point, and I wanted to quickly clarify: this point is for existing public API in resilient libraries that use a bare protocol as an existential type. New public API or public API that use the explicit some syntax can use the bare P shorthand under this proposal. The only reason why existing public API cannot take advantage of the minimized code churn that this proposal would bring is because it's an ABI break to change an existential type to use a type parameter instead.

Thank you for the detailed review, I'll spend some time on an in-depth read!

1 Like
5

Sorry if Iā€™m missing something but how does the compiler ā€˜knowā€™ when compiling a resilient library whether itā€™s compiling an existing API or not?

1 Like
6

The point in the proposal has no implication on compiler errors. Changing an existing public API from using any to some (either implicitly or explicitly) is a mistake that would be caught by other tools for resilient library authors such as the ABI checker. That said, it's possible that as part of the Swift 6 transition, we could provide a "migrator mode" for the compiler that will error when using implicit some for public API under library evolution, with an assumption that under this mode, the programmer is strictly transitioning existing code rather than adding new code.

2 Likes
7

Got it, thanks. Still donā€™t love the idea of quietly changing the ABI of a bunch of resilient interfaces and relying on other tools to catch it but I suppose the thinking here is that any resilient library author who cares at all about not breaking ABI must already be using such tools?

4 Likes
8

A variable of opaque type must be bound to a single static type. If you could declare a variable of opaque type without immediately initializing it, you could put the initialization inside an if statement, where each branch bound the variable to a different concrete type:

func myFunc(arg: Bool) - > some Sequence<Int>
  let x: some Sequence<Int>
  if arg {
    x = SequenceOfOne(42)
  } else {
    x = [42]
  }
  return x // concrete type of `x` depends on execution path!
}

IIRC, this could theoretically work if the compiler could statically prove that all execution paths through the function bound x to the same concrete type, but thatā€™s too complex of a dataflow analysis problem to do regularly.

3 Likes
9

As I've argued before, this does not seem like a very good idea, at least if we can't get rid of some everywhere. It seems from the pitch that we can't do that, but I can't really see exactly why those cases are different.

If we are keeping some uses of some, and also, as you do here, talk about the bare protocol as an "implicit some", I don't think that's a very clear or simple situation. If people will still have to imagine a some, then what is really gained. I think this change only makes sense if we can completely get rid of the whole concept of some, and just have any and bare protocol, where bare protocol doesn't mean specifically some.

Also, I think the migration argument is very weak, it seems that will only really apply to badly written code, code where implicit any shouldn't have been used in the first place. That seems like the last thing we want to optimise for.

18 Likes
10

May I plea for something more than a "future direction", and instead some kind of moral acknowledgment that the compiler shouldn't ship with this pitch but without this shorthand? In my very humble opinion the shorthand should even ship first and become a precondition for this pitch (i.e. the shorthand must be supported, instead of some non-mandatory eventual enhancement).

Forcing users to write the mouthful and arcane weak var delegate: (any MyDelegate)? is somewhere between inelegant and hostile, considering that the delegate pattern is still quite vivid (and unavoidable when you work with AppKit/UIKit frameworks - you know, the ones you must use until your app has a minimum target OS where SwiftUI catches up).

// A quick reminder:
// Before SE-0335
weak var delegate: MyDelegate?
// SE-0335
weak var delegate: (any MyDelegate)?
// SE-0335 + shorthand šŸ™
weak var delegate: any MyDelegate?
12 Likes
11

I believe @GreatApeā€™s conclusion totally makes sense. Iā€™m not a fan of simply making codes shorter unless we can achieve extra proof or functionality by doing that. Unfortunately this pitch didnā€™t seem to have such benefits.

some and any, as a pair of keywords, represents two ways and views we can work with the generic type system most of the time. Questions like "whether we want any or some for an API" generally help programmers to think in a generic way. On the other hand, when we donā€™t see the two keywords, we also know clearly that the type is either concrete or having external constraints.

The direction shown by this pitch largely breaks the balance and, despite making rooms for "simple generic codes", may be harmful for language learners and users.

19 Likes
12

While this is an interesting direction I have some concerns with this pitch. It strongly feels like we would like to revert some in almost all places and reverting several past proposals on that topic. I'm personally not in favor of such drastic change.

If you recall we want to be able to refer to a protocol as both an existential or the protocol. If that change this would not be possible.

typealias MyP = P // MyP is a protocol alias not an alias for `some P`
typealias AnyP = any P // aliasing the existential

We could also introduce the ability to have several very useful meanings in extensions when using P, any P and some P

So it's -1 for me.

8 Likes
13

I was in favor of this shorthand during the review of SE-0335 and remain so today; but in my view, some P? (or, with this pitch, just P?) and any P? shorthands can be separated out as an orthogonal pitchā€”indeed, I would be in favor of not requiring parentheses here even if we reject this pitch in its entirety.

8 Likes
14

It's certainly possible, but my two cents here are that we are not in a good place if that's what it will take.

Put another way, I think the pitch is a very intriguing idea, and it would be a great thing essentially to silently swap the meaning of naked P from any P to some P if it (a) reduces churn during the transition to Swift 6; and (b) even helps to improve the performance of users' code when they're reaching for existential types now without actually requiring itā€”all (c) without silently causing inadvertent changes in behavior except in the cornerest of corner cases. I think I'm well convinced of (a) and (b), but like @Jumhyn I'm quite worried about (c).

If we don't have confidence that we can migrate entire swaths of code (e.g., all public APIs) without special tools to avoid silently making correct code incorrect, then we're leaning pretty heavily on best-effort heuristics for correctness.

Particularly when it comes to the type system, I think that would fall firmly on the "un-Swifty" side of the spectrum from completeness to correctness that @itaifarber described so well. Yes, there'd be nothing more unsound about the underlying type system, but we'd be basically overlaying a "guess" as to whether a naked P in Swift 5 code should become any P or some P at the level of the migration tool, effectively making the meaning of P (or, if the tool will error, the validity of using it) heuristically determined.

9 Likes
15

Yeah this is a good thing to call out. Authors of public libraries who don't know all their clients (and cares about breaking source) will basically have to make the P -> any P transformation because any client might be using the declaration in a way that would be invalid with some P. Perhaps that suggests a narrower rule such as

Bare P as shorthand for some P is only available in positions that are not exposed publicly.

at least for Swift 6.

1 Like
16

It comes to my mind that the switch of implicitness from any to some has some impact on DocC - both for internal and public code.

I didn't spend more than a few minutes thinking about it, but I can feel that function prototypes displayed by DocC will become a point of confusion unless we are very careful.

In the imaginary DocC page below, are we talking about existentials, or opaque types? When was this documentation generated? Which language mode does it target? Do I have to run implicit conversions in my mind when it happens that I program Swift 6, and read the doc of a Swift 5 package? What if I don't even know the language mode of the package I use? How can I have a non-ambiguous interpretation of what I read, and which amount of knowledge/experience and energy does it take?

Declaration

func frobnicate(_ x: P1) -> P2

Parameters

etc.

Tangential question: will DocC format explicit some P in source code to bare P in the generated doc, or the opposite? Or no formatting at all?

I guess this would be the topic on another pitchā€¦ Please pardon me @Joseph_Heck to ping you directly - maybe some members of the Documentation Workgroup have something to say here.

5 Likes
17

In general I'm in favor of the pitch. I think it makes sense for some to be the default choice and therefore require the least ceremony. I do share @Jumhyn's concern about the plan for resilient libraries, though:

I feel nervous about relying on usage of the ABI checker to ensure that library owners don't break ABI during the transition because I don't feel confident that the practice is widespread enough. Personally, I'd rather see an exception requiring explicit some or any in public declarations of resilient libraries. With some well worded diagnostics I don't think it would feel that strange.

1 Like
18

No problem pinging me, but I don't have any good answers here. I'll need to defer to others in the workgroup - I'm hoping @QuietMisdreavus or @Franklin might have a clearer picture of what metadata is captured in the symbol extraction and flowed through to the end results - either the compacted JSON data structures in the doc archive or the HTML output rendered from it.

I think in either case, eliding it would seem to me that it would still be explicit in the meta-data to the compiler, and that could be reflected out into the metadata that DocC uses - but I don't know what is or isn't included there today to distinguish between any and some.

1 Like
19

Thank you for not rushing an answer :)

1 Like
20

I'm strongly against this because:

  • The introducing of any came with the educational benefit of clarifying that a protocol P is not the same as a type P. Forcing the use of some or any make us deal with it and think carefully about what our intention is when using a protocol as a parameter. I really thought this was the intention of Swift 6.

  • Since some won't go away we will keep having three different ways of using protocols, where two actually mean the same thing.

  • The mandatory use of any or some also brings clarity at the point of use. You just know that you dealing with a protocol because of these keywords.

In summary, I think eliding some will just keep a bunch of today problems of using P as type but just move theses problems to a less harmful place.

46 Likes