[Pitch] Elide `some` in Swift 6

[Ben_Cohen]

I'll post my thoughts on the pitch (of which I'm very much in favor) in another post. For now, I wanted to add some practical usage data to the discussion.

I took the latest nightly toolchain and used it to port the Mac target of NetNewsWire to implicit some. It's a fairly large-sized real world project. I only updated the Mac target, and only the app project itself, not the packages on which it depends, which stayed with the current behavior.

The experience went pretty smoothly (I filed some diagnostic improvements that would have helped along the way) and took me about half an hour. Every change was mechanical – I basically added any wherever I needed to to make it compile. In all cases this mechanical translation was "correct" – there were a few places where I could have fiddled with the code to instead make it generic, but didn't.

After migration, the code compiled and worked both with -enable-experimental-feature ImplicitSome and without it.

Overall there were 319 needed additions of any. Note that these are the requirement of of SE-0335 rather than this pitch. Incidentally, this has solidified in my mind that the parenthesis in (any P)? have got to go. This was discussed during the SE-0335 but needs revisiting based on (my :) experience in the wild.

By far the biggest cause was Result<_, any Error>, of which there were 205 occurrences, almost entirely found in callbacks. Presumably these will all go away as codebases migrate to async functions instead.

The remaining 200 or so any were mostly either:

• Storage: delegate protocols where the any is there to represent it could be any delegate (you could model this instead by making the type generic over its delegate, but that is probably not worth doing), or heterogenous collections. In these cases the any adds useful meaning. It really could be any of a number of type-erased types.
• Optional Arguments: the function was being passed an existential, and it was optional, so the concrete type may not be available when opening the existential. In some cases, some local refactoring could have eliminated the existential, improving the code slightly. In others places, use of any was the best option.
• Conformances to protocols that took any ObjCType and that cannot be generic because of the requirements of @objc. It is worth discussing a targeted carve-out to default imported ObjC protocols to be any.

Offsetting these, I counted about 40 or so places where some was in the code but could be eliminated – all uses of it in SwiftUI (some View, some Widget etc). There is currently a bug in the nightly toolchain so I didn't actually do this migration.

NetNewsWire started as a *Kit app so has relatively little SwiftUI. This will presumably increase significantly over time. An app that was all SwiftUI would likely eliminate several hundred of these (it might be interesting to try porting Ice Cubes or MovieSwiftUI). Subjectively, my view is that these some View annotations are pure noise, adding little readability benefit. This is in stark contrast to e.g. [any View] storage where the heterogenous nature of the storage is of significance. I'll say more about that when I post my pitch response.

After I did this port, I then went back and switched to -enable-upcoming-feature ExistentialAny instead. This required me to go through and add 60 additional annotations. These were mainly either on arguments where the function was now generic but functionally equivalent to an existential, or they were on as? or is where the annotation doesn't really have much effect. Again my personal opinion is these extra annotations were just noise without readability benefit (will post further on this), and added additional migration effort without much benefit.

[GreatApe (Gustaf Kugelberg)]

Thanks for the thorough reply (in old Airspeed fashion), but it seems I misunderstood your previous post. I thought you meant that the proposed change in general was good, and that's what I wanted to ask about, not the overload priority change that you where actually referring to.

[Ben_Cohen]

Heh, oops. I do also think the proposal is generally good. I think it makes Swift into what I wish it had always been. Still writing that. But in this case, yeah, I was just talking about the overload resolution priority concern.

[taylorswift]

i have been doing a lot of API and DSL design recently, so my opinion is colored by that use case.

right now i am very opposed to “hiding” generic-ness, because (as many others have already pointed out), generic APIs behave very differently from concrete APIs:

• generic APIs cannot use leading-dot syntax. (as an API designer, SE-0299 is not an attractive workaround because of the symbol pollution it causes.)
• generic APIs are incompatible with the various ExpressibleBy literal protocols. this can have serious performance implications for libraries that expose a preferred and more-efficient ExpressibleBy type, but also provide a fallback conformance for Array<Element>, Dictionary<Key, Value>, etc.

if member/conformance lookup were generalized to “just work” for generic APIs the way they work for concretely-typed APIs, i would have no issue with implicit some.

[Gerzer (Gabriel Jacoby-Cooper)]

I’m very opposed to this pitch. It wasn’t so long ago that MyProtocol meant any MyProtocol, and that code will now swap meaning some MyProtocol without any source-code changes. That level of churn is damaging, I think.

Mandating any was a big win for clarity and the mitigation of foot-guns; we threaten to undo many of those benefits if we permit the elision of some. I’m aware that there’s a narrative that developers should start with the opaque type and only switch to the existential version if the property/function/whatever needs to support heterogeneity. That’s wise advice in some scenarios, but it’s not a good reason to hide an incredibly important detail about the semantics of your generic types.

For a similar reason to what led to the any keyword, it would be totally nonobvious to someone who isn’t familiar with the intricacies of the Swift type system that func doSomething() -> MyProtocol must always return the same underlying concrete type. Novice developers will be confused about why they can’t use an if statement to return one of two different concrete types. “They both conform to the same protocol, so why does it care? I already set the return type to be the protocol, not just one of the concrete types!”

Someone will inevitably answer a relevant Stack Overflow question by just advising that the original poster add the any keyword without further explanation, and then it’ll come to be known as “the keyword that you add to make the annoying type-check error go away”, at which point we’ll be right back to where we were before the any keyword, with developers falling off the existential cliff (does that count as a pun? I think so!) of no Self or associated-type references without realizing it.

The some keyword is an indication that “there’s something more going on”, and it disambiguates between the two interpretations of using a protocol as a type. The fact that you can’t “just use the protocol as a type” is precisely the point!

The short version is that developers absolutely need to know the difference between opaque and existential types. This domain is sufficiently complex and nuanced that you can’t just hand-wave away those details and declare one as the syntactic default.

[KeithBauerANZ (Keith Bauer)]

I'm against this.

Swift 1.0 made the unfortunate decision to conflate "interfaces" (ObjC @protocols) with "typeclasses" (Swift protocols with Self constraints, etc.). Where an interface creates a type, a typeclass does not.

With some and any, we've finally undone this conflation, and have a more or less clear understanding that a protocol represents a family of types, and that some P selects one such type, and any P allows any such type (with corresponding restrictions on what you can then achieve given the lack of information).

Shifting the line in the sand to introduce a conflation in the opposite direction doesn't seem helpful, particularly given the significant number of situations where some P is disallowed or in some way does not act like a normal, concrete type.

[xwu (Xiaodi Wu)]

@Ben_Cohen

By far the biggest cause was Result<_, any Error>, of which there were 205 occurrences

[ . . . ]

Conformances to protocols that took any ObjCType and that cannot be generic because of the requirements of @objc. It is worth discussing a targeted carve-out to default imported ObjC protocols to be any.

This is intriguing to me, because it suggests a way to accomplish the goal of a smooth migration by going in a contrary direction to this pitch.

To back up a bit: One thing that concerns me about the pitch is that we already have two special-case elisions allowed by SE-0355: Any is equivalent to any Any, and AnyObject is equivalent to any AnyObject—if we adopt this pitch as-is, then we would have a scenario where everywhere a bare protocol P means some P except for Any and AnyObject. That much can be taught but seems...inelegant.

I'm not sure if I'm reading you correctly where you talk about a "targeted carve-out," but if this is to say that for an imported Objective-C protocol ObjcP, bare ObjcP should mean any ObjcP, but for a Swift-native protocol SwiftP, bare SwiftP should mean some SwiftP—well, it would seem to me that this would make the language and our migration story more confusing, as we'd be extending the special rule above beyond just two cases that the user can memorize to arbitrarily many protocols that users have to consult documentation and/or rely on tooling to clarify the provenance of.

However, there is something particular about Objective-C protocols, shared in common with Error and with marker protocols, which bears considering: namely, in these cases, the existential types conform to the corresponding protocol. I'd argue that it makes the distinction between any P and some P less salient to the end user, since of course in those particular cases any P is notionally a valid underlying concrete type where a function expects an argument or returns a value of opaque type some P.

Indeed, one might argue that in those cases specifically the distinction shades closer to being a compiler implementation detail that the user shouldn't have to worry about unless they want to. It's a very "Swifty" thing to apply a progressive disclosure approach in such scenarios; for example, we envision that users shouldn't be required to deal with ownership operators if they're using copyable types unless they want to fine-tune their code for performance.

Therefore, if we decide not to go in the direction of this pitch, an alternative could be to roll back the requirement for explicit any specifically for "self-conforming protocols" (i.e., Obj-C protocols, Error, and marker protocols, of which we have only Sendable). In those cases, bare P would continue to mean any P, with the compiler being free as it now is to optimize provably unnecessary uses to be equivalent to some P.

In the case of your migration example, this would eliminate over half of the required edits.

This would also have the salutary benefit of allowing us to stage in the proposed "flip" in overload resolution rules discussed above in a less invasive way, although this may also be too clever by half:

For "self-conforming protocols," it is arguably the case that existential P (being notionally a valid underlying concrete type) is "more specific" than some P, aligning with present-day overload resolution rules. We could therefore say that only any P when spelled out as such is regarded in Swift 6 as being "less specific" than some P, because the former explicitly specifies an existential box, leaving any continued valid uses of bare P as "more specific."

[johnno1962 (John Holdsworth)]

@Ben_Cohen

By far the biggest cause was Result<_, any Error>, of which there were 205 occurrences, almost entirely found in callbacks. Presumably these will all go away as codebases migrate to async functions instead.

It's true many of the errors trying to compile Alamofire were where any had been elided to some for closure arguments of callbacks such as the following:

/Volumes/Data2/some/Alamofire/Source/RequestInterceptor.swift:121:71: error: 'some' cannot appear in parameter position in parameter type '(Result<URLRequest, Error>) -> Void'
    public func adapt(_ urlRequest: URLRequest, for session: Session, completion: @escaping (Result<URLRequest, Error>) -> Void) {

Is there a conceptual reason why closures cannot receive an opaque arguments or is that just a current limitation of the compiler? From a low level point of view it feels like it should be possible (being just a change in calling convention) though I don't imagine it would be a small change. It just wan't expressible in the language before now.

[gwendal.roue (Gwendal Roué)]

@xwu

For "self-conforming protocols," it is arguably the case that existential P (being notionally a valid underlying concrete type) is "more specific" than some P, aligning with present-day overload resolution rules.

Would you please make the arguments explicit, if you have time? Favoring existentials can only prevent optimizations (both user-written optimizations for specific types, and compiler optimizations with specialization, guided by user annotations and internal heuristics). It seems to me that favoring existentials goes against the general spirit of this pitch which is to reward the use of generics with more efficient runtime behavior.

The presence of two overloads, one that targets existentials, one that targets generics, may be the consequence of api ergonomics (the author intends to accept both generics and existentials, for the convenience of the client, but also in order to support literals such as 123 or nil). Maybe automatic opening of existentials have made this technique a thing of the past (an assertion that needs to be checked - oops it is checked below and well it does not hold) but there is still some code like this in the wild.

EDIT: some examples

These tests: 1, 2, 3 would not compile without explicit overloads for existentials.

Note:

1. The use of the non-public @_disfavoredOverload in order to reverse the overload resolution rule.
2. Automatic opening of existentials does not make these tests compile when I remove the explicit support for existentials and rely on generics only.

[masters3d]

It would be nice if we could enhance the proposal with some examples of what other languages do with their protocol/interfaces. I believe c# interfaces (Java’s as well?) work like swift’s some protocol.

[esummers (Eric)]

C# and Java I believe just have generics. No existential types. Swift is a pretty cutting-edge language compared to most others. It would be hard to compare to other languages.

[Max_Desiatov]

I don't think that's the case at least for C#. For example, here folks discuss generics and interfaces and it looks to me like the same thing, even with similar overhead concerns: an interface boxes a struct (they don't call it an existential though), while using it as a generic constraint does not: c# - Structs, Interfaces and Boxing - Stack Overflow

[John_McCall]

Existential types are actually a very common language feature in OO languages. Java’s interface types (Collection), Objective-C’s protocol-qualified types (id<NSObject>), and so on are all kinds of existential type.

When OO languages add generics systems, you can usually also use those types as constraints on type parameters.

[esummers (Eric)]

Ah ok. I’ve never much been attracted to C# or Java. Interfaces seem similar enough. However I think the opaque some type might be too unique to Swift to compare to C#.

[CharlesS (Charles Srstka)]

-1 from me. Others have said most of what I want to say more eloquently than I could, but my basic gut reactions are:

• The explicit spelling is much clearer than implicit. some vs. any tells you exactly what you're getting. This is a good thing.
• The implicit behavior used to be any, and holy hell would that make this confusing if it now means something else. Reading some arbitrary snippet of code, you'll have to figure out what Swift version it was written for in order to figure out what it's intended to be doing. Could you imagine being a newbie learning the language and coming across older tutorials with implicit any in them? At least with the current situation this can be automatically fixed via a migrator or a fix-it.
• Is writing some really that hard? It's just four characters.

Now making some MyProtocol? shorthand for (some MyProtocol)? — that I can get behind.

[Zollerboy1 (Josef Zoller)]

I believe, the only (major) language that has a feature that works similar to Swift's existential/opaque types is Rust, where one would write any Protocol and some Protocol roughly like this: Box<dyn Trait> and impl Trait. Especially impl functions in almost the same way as some does. However, AFAIK, you cannot elide any parts of these types.

[jeremyp (Jeremy Pereira)]

@CharlesS

Is writing some really that hard? It's just four characters.

Five including the space

I'm also -1 on this.

The way Swift handles protocols was always one of my biggest bugbears with the language. Having written a fair amount of Java in the past where you can treat List<String> as a type, it always mystified me that I couldn't use Collection where Element == String in the same way. It was a huge point of friction for me.

That is, it was until I had the realisation that protocols are not types. Well, in fact, I think somebody had to point it out to me. My confusion was partially sustained by the syntax of Swift. You could write x: SomeProtocol in a lot of situations which makes it look like SomeProtocol is something it is not. I'm sure I'm not the only person with that experience.

So my position is that we should be moving away from introducing features that conflate protocols and concrete types. Explicit any was a good move in that direction I think, in spite of the migration pain it is likely to cause. Implicit some even when optional is a move in the wrong direction. Just because something is more concise doesn't mean it is better. Having to write some might be a bit of a pain and it makes the line longer and a bit more verbose, but the future reader, can see at a glance that we've got a protocol and the novice is not encouraged to think of protools as being the same as types.

[AlexisQapa (Alexis Schultz)]

-1 for me too

While I appreciate the effort to ease the migration this seems to go backward for all the reasons mentioned before. One thing that I think wasn't mentioned is that with explicit some it's far easier to google or search in the forum. This is great for progressive disclosure.

[toph42 (Topher Hickman)]

-1 from me. Speaking as someone who did not truly understand the Protocol/Existential/Opaque trifecta until any and some became ubiquitous, I hope we don’t lose that clarity. I feel like this proposal makes the language worse.

[tabsandspaces]

-1 from me.

@angela-laar

Generic code is already simplified in more places than you think. Take protocol extensions, for example. To write generic code applicable to any concrete type conforming to a protocol, you only need to write the extension keyword and the plain protocol name.

Having already made inadvertent mistakes in language design is not a reasonable argument for knowingly making more.

@angela-laar

Deciding to introduce a protocol into your code to allow for more abstraction should be considered a separate decision and this proposal would decouple those two decisions.

I may be misunderstanding here, but to me it seems like it wouldn't decouple the decisions, it would make the second decision for you automatically without telling you.

@angela-laar

I wouldn't say making bare protocols is really "making the language more opaque" though. Instead this proposal is improving the language design in such a way that enhances user experience;

As I've mentioned before, my experience is precisely the opposite. I was totally unable to form a proper understanding of generics in Swift prior to seeing code that consistently used explicit some.

@tabsandspaces

Seeing recent code examples with explicit use of some and any everywhere where they apply is one of the few times in my programming career where I've caught myself thinking "wow, I really like this language design choice". I would be sad to see it go away. Consistently providing the vocabulary to reason about use of some P or any P feels much more useful, and safer, than the alternative.

[...]

Hiding this complexity out of concern that newcomers may misunderstand may accomplish the exact opposite of your goal. Developers will be hindered in coming to a full understanding of the code they write.

Tricking people into writing generic code without realising what they're doing doesn't seem like a useful goal.

@angela-laar

Why require users who want to add abstraction to their code using protocols to deep dive into the semantic differences?

The alternative is that people will be writing a lot of code that they do not truly understand. Languages must not hide important semantic differences from code authors just because they think they know what the author probably "really" means.

In my opinion, this pitch has no benefits, only a number of known costs and fairly significant risks.