Overall +1, I'm excited to see this change, and hope that it will be a stepping stone towards existential subtyping relation and generalised existentials.
My only concern is that discouraging any to be used with Any and AnyObject makes the latter to look like existentials, and affects how Any.Type and AnyObject.Type should be interpreted.
Hello,
I have two questions regarding the transitions from P to any P, from Swift 5.5, to 5.6, to 6.
The first question is about applications (code bases that support a single build environment), and the second about libraries (code bases that support multiple build environments).
From the point of view of applications, I only consider the transition from Swift 5.5 to Swift 5.6, the release which introduces warnings about naked P existentials. Do we have an idea about the eventual fixits that will come with this warning?
From the point of view of libraries, things are more complicated. Libraries love to support multiple language versions, because dropping support for a language version is, strictly speaking, a breaking change in semantic versioning. Practically speaking, libraries relax this rule, but still generally support a range of compiler/swift versions (examples: SwiftNIO, Alamofire, GRDB). The compiler helps users upgrade their compiler version without upgrading their libraries, thanks to language modes (so that, say, a Swift 6 user can depend on a library that requires the Swift 5 mode).
Libraries like to support the latest language version, so that they can improve their api with the new shiny language features. The compiler helps libraries support users who still run older language versions with conditional compilation (#if swift and #if compiler).
In the particular case of any P, though, conditional compilation will NOT play its usual role, because the code below is not valid Swift code:
/// Doc comment
#if compiler(>=5.6)
public def foo(_ x: any P)
#else
public def foo(_ x: P)
#endif
{
// function body
}
A potential workaround is to define a typealias, but this extra indirection is very bad for the api surface and the library documentation:
#if compiler(>=5.6)
/// Doc comment
public typealias PExistential = any P
#else
/// Doc comment
public typealias PExistential = P
#endif
/// Doc comment
public def foo(_ x: PExistential) {
// function body
}
Another potential workaround which preserves the api surface and library documentation is private helper functions. Caveats are that we have to duplicate doc comments, and that this technique would be required for all public apis that have existentials in their signature:
#if compiler(>=5.6)
/// Doc comment
public def foo(_ x: any P) { _foo(x) }
private typealias PExistential = any P
#else
/// Doc comment
public def foo(_ x: P) { _foo(x) }
private typealias PExistential = P
#endif
private func _foo(_x: PExistential) {
// function body
}
And what if libraries do not want to enter this compatibility hell? Their only choice is to drop Swift 5.5 from their range of supported versions, and only use any P.
So here here is my final question: Should we amend the proposal in order to help libraries that want to support Swift 5.5...6? SwiftNIO is such a library, if I understand their intent correctly, and the "2 immediately prior non-patch versions" before Swift 6 are 5.5 and 5.6. And there likely exists other libraries in this case.
EDIT: Let's imagine a library which decides to not use the complex compatibility techniques described above. It will have to go through those steps:
P, and it can't use Swift 5.6 features (or it would get compiler warnings on naked P existentials).P, and it can use Swift 6 features.I assume it can't drop support Swift 5.5 support before Swift 6, due to the constraints of semantic versioning and the reasonable user expectations about the range of supported language versions.
See how the library can not use Swift 5.6 features until Swift 6 is out. That's a long period of time.
I can't see drawbacks for the user (except library apis that do not use Swift 5.6 features), but maybe I did not look long enough. I encourage other pairs of eyes to take a serious look at this issue.
Does this suggest the feature is best staged in over two major language versions (i.e., warning for Swift 6, error for Swift 7)?
Users (and library authors) do not like unfixable warnings that can not be silenced. A warning implies that something should be fixed. In our particular case, it is unfixable in the current state of the language and proposal. Libraries don't like warnings because they are a reputation hazard: no user likes to embed third-party code and see warnings everywhere.
I was interested in recent discussions about lifting constraints about conditional compilation (as discussed, for example, in SE-0330: Conditionals in Collections). I mean that if the code below 1. compiles, and 2. recognizes doc comments as such, I would be satisfied:
/// Doc comment (written once)
#if compiler(>=5.6)
public def foo(_ x: any P)
#else
public def foo(_ x: P)
#endif
{
// function body (written once)
}
I understand that this would be a big Xmas gift.
+1. Great improvement. It fits well into the realm with some keyword for opaque types, Any special type and AnyType type erasure pattern. And I like every decision over other alternatives considered.
There is no official guidance yet on whether the next version of Swift after 5.6 will be 5.7 or 6.0. The core team is still considering when 6.0 will be. Generally speaking, let's not make assumptions about what the next version will be unless that's been clearly and formally stated.
That said, please feel free to provide insights into the tradeoffs of this proposal interacting with major/minor language versions. Those are quite important and useful.
Hi Gwendal, thank you for your thoughtful feedback about the syntax transition and library compatibility.
This proposal/implementation does not introduce new warnings for existing code in Swift 5.6.
When a fix-it is added, it will replace the old type with it's new spelling. For a plain protocol P, the existential type P will be replaced with any P, the existential metatype P.Type will be replaced with any P.Type, the protocol metatype P.Protocol will be replaced with (any P).Type, and so on.
I completely agree with you that better #if support would greatly help library authors adopt new features while still supporting prior language modes and compiler versions, especially in the case of any P. However, I don't think the limitations #if are unique to this proposal -- the issue comes up with any new attribute or keyword that can be applied to declarations, for example -- nor do I think better support should be lumped into this proposal specifically.
Thanks! I had in mind that this proposal would drive users towards any P with a warning, but it "only" adds support for any P (and that's already a big deal).
So it looks like my concerns are postponed until naked P existential actually generates a warning or an error (Swift 6).
I respectfully disagree. This proposal states that naked P existentials are an error in Swift 6, so it is examplar of this issue, even it the pain will only be felt later.
I can provide the background here. We ended up doing this when implementing Sendable in the standard library, because it depends on the feature being in the compiler as well, like this:
#if <some check for marker protocols>
@_marker protocol Sendable { }
#else
typealias Sendable = Any
#endif
It allowed downstream clients to use Sendable as a generic constraint and also write conformances to Sendable, even with older compilers that couldn't handle the definition.
Doug
I think the answer should be no, because that would lead to really weird inconsistencies with associated type inference. You'd be able to use a plain Any existential type almost everywhere, except in a position that's a source for associated type inference, unless the implicit typealias also gets an implicit any.
I don't think the warnings would apply to any Any.Type and (any Any).Type, especially because there would be no other way to spell the latter type in Swift 6 (although I don't know if this type is useful anyway). I also don't feel too strongly about the warning.
That said, I think these are compelling points that indicate, at the very least, we shouldn't dismiss the alternative of any Value and any Object purely because Any and AnyObject already have "any" in the name. I've been thinking a little more about what any Value and any Object would mean in practice. Here are a couple thoughts:
Value isn't a great name for an empty protocol composition, but I'm sure there are better names we could come up with.Value or Object protocols or types. This is fine for the code as-is, because those declarations would shadow the ones in the language / standard library, but projects would need to fully qualify any Swift.Value or any Swift.Object when replacing Any and AnyObject, or when using Swift.Object as a layout constraint.Any and AnyObject as type aliases to any Value and any Object, respectively, to minimize the code churn. This could also provide a nicer solution to needing to write any Swift.Object when the project has its own Object, but it still requires Swift.Object when writing a layout constraint in that case.One observation is that Any is useless as a generic constraint unless you want to write an unconstrained opaque type some Any. So, I still think the confusion around Any is minimal. I admit that some Any reads pretty poorly, but I'm also not sure where it's useful. Additionally, I can convince myself that using AnyObject as both an existential type and a layout constraint makes sense. Because T: AnyObject is a layout constraint and not a normal protocol conformance requirement, I think the implications are largely the same between the two. The biggest difference would be the type system semantics, e.g. the different between <T: AnyObject> (arg1: T, arg2: T) and (arg1: AnyObject, arg2: AnyObject), and of course this means AnyObject wouldn't fit into the "protocol name as implicit type parameter" future direction.
While addition is fine, I still think that old code should still work with Swift 6 and only produce warning and not an error to avoid breaking most of todays Swift code. This will mostly affect newcomers when they try to follow a year old book and suddenly code does not compile.
Any reason why we cannot leave it as a warning in Swift 6?
I'm excited about making existential explicit, so +1 on the general direction.
I've raised the suggestion of Any<P> in the past, and I appreciate it was addressed in the proposal. I suspect that the decision is pretty much made on that front already, so I'm hesitant to dive in again, but I did want to address the comments in that section.
From the Alternatives Considered section of the proposal:
Use
Any<P>instead ofany PA common suggestion is to spell existential types with angle brackets on
Any, e.g.Any<Hashable>.any Phas symmetry withsome P, where both keywords can be applied to protocol constraints. TheAny<P>syntax is also misleading because it appears thatAnyis a generic type, which is confusing to the mental model for 2 reasons:
- A generic type is something programmers can implement themselves. In reality, existential types are a built-in language feature that would be very difficult to replicate with regular Swift code.
- This syntax creates the misconception that the underlying concrete type is a generic argument to
Anythat is preserved statically in the existential type. ThePinAny<P>looks like an implicit type parameter with a conformance requirement, but it's not; the underlying type conforming toPis erased at compile-time.
All of this is true, but it doesn't convince me that any P is better. To respond to some points directly:
any Phas symmetry withsome P, where both keywords can be applied to protocol constraints
I view this symmetry between some P and any P as a source of confusion, not clarity. While it's true that both some and any are keywords that can be applied to protocol constraints, that similarity hides an underlying dissimilarity, which is that any P creates a new distinct wrapper type:
func useSome() -> some Numeric { return 42 }
func useAny() -> any Numeric { return 42 }
let tSome = type(of: useSome()) // Int
let tAny = type(of: useAny()) // any Numeric
An existential is a wrapper type that can hold any value conforming to the protocol. The fact that any creates a new type while some just elides an existing type is an essential distinction between the two, and the fact that they are both keywords only serves to hide this fact. A new type should look like a new type. Any<P> looks like a new type. any P does not.
The
Any<P>syntax is also misleading because it appears thatAnyis a generic type.
And any P is misleading because it doesn't appear that any P is a separate wrapper type at all.
[this is] confusing to the mental model [because]:
- A generic type is something programmers can implement themselves. In reality, existential types are a built-in language feature that would be very difficult to replicate with regular Swift code.
Any is already magical; you already could not implement Any in regular Swift code. I don't see how Any growing some angle brackets would make users suddenly think it was implementable in regular Swift code, nor is it clear to be why the presence of angle brackets would make users want to implement it in regular Swift code.
- This syntax creates the misconception that the underlying concrete type is a generic argument to
Anythat is preserved statically in the existential type. ThePinAny<P>looks like an implicit type parameter with a conformance requirement, but it's not; the underlying type conforming toPis erased at compile-time.
Yes, Any<P> would act differently than a normal generic type. Any is already magical, though, so I'm not sure what practical confusion this would cause. Is the objection that someone would try to write code like this, and it wouldn't work?
func takeAny<T>(_ thing: Any<T>) {
print("You passed me some sort of \(T.self)")
}
I suppose that's a fair objection; if that's the concern, I think it would be useful to call that out directly, because I didn't see that until I sat down to write this and thought about it for a while.
Since I'm suggesting that Any<P> would not actually be generic, we should take off the <T> on takeAny<T>. Then it would look like this:
func takeAny(_ thing: Any<P>) {
print("You passed me some sort of \(P.self)")
}
But then the compiler is going to give a warning that it doesn't know what P is. What the user actually needs to do is this:
func takeAny(_ thing: Any<Numeric>) {
print("You passed me some sort of \(Numeric.self)")
}
Does this adequately capture the concern you have regarding the Any<P> syntax? My response would be that Any is already magical and I think it would be okay for it to continue to be magical in this regard, but I can understand better now what the objection is. I would be open to considering other syntax for wrapper types if it would help address the concern over confusion with generics, such as Any(P) or Any[P], but I suspect there's not much appetite for that. I do still think that we could address the concerns above with compiler diagnostics.
TL;DR
The crux of my argument is that an existential is a wrapper type, and it should look like a wrapper type. Currently it does not. I think that Any<P> looks more like a wrapper type than any P does, and I worry that switching from func x(_ p: P) to func x(_ p: any P) will not adequately convey to users that p is a wrapper type, not the original type. Understanding that it's a wrapper type is crucial to understanding why any P does not itself conform to P, and I worry that we're not adequately communicating that to users.
Iâm super excited about this feature that will clear things out and happy that we are getting it already in 5.6. The transition plan looks reasonable. A big +1.
One question. Currently one canât have nested types within protocols. My uneducated guess is that that has something to do with protocols not being types themselves, but constraints (btw, I love this new terminology).
Would this feature allow or open a future direction to allow nesting types within existential extensions?
extension any P {
struct S {} // valid or error?
}
Till some point I was even thinking that Any is an existential, not a protocol.
We could avoid having Value protocol altogether, and keep Any as a existential only. For now as a compiler built-in, and with generalized existentials make it a typealias for any<T> T.
It not only removes a redundant entity, but also eliminates self-conformance (any Value): Value.
But then, as you mentioned, until we have generalized opaque types we cannot express some<T> T. Not sure if thatâs a big loss though. I cannot think of any useful operations on values of unconstrained opaque type, which do not involve wrapping it into Any (existential). And if you end up wrapping it into an existential, you can just return an existential in the first place.
And also semantics of Any.Type changes into singletone metatype, which is not what is needed in the majority of cases. So additionally there would need to be a builtin/typealias AnyType = any<T> T.Type. Which sounds ok to me - Any, AnyType, AnyObject look pretty consistent together.
@hborla Firstly, Iâd like to thank you for all of your hard work on this feature. Iâve followed the original thread, read the proposal in full, and fully agree with the motivation of the pitch but am still -1 on the solution as the choice of the word any does not adequately address the problems stated in the motivation. TL;DR I would ask that the author consider an alternative spelling dynamic P or existential P (preferably the former).
Let's look at this excerpt from the motivation:
In addition to heap allocation and reference counting, code using existential types incurs pointer indirection and dynamic method dispatch that cannot be optimized away.
Despite these significant and often undesirable implications, existential types have a minimal spelling. Syntactically, the cost of using one is hidden
The word any is doing some serious heavy lifting to make heap allocation, ref counting, pointer indirection, and dynamic dispatch explicit or obvious. Unless someone has read over this thread, I donât think the word any is informative enough to educate a developer on the concept of existentials and their associated costs. If the goal is to make existentials explicit then something like existential P is much more direct. It explicitly names the concept and gives developers something specific to search for to learn more. Searches for âswift anyâ are likely going to produce loads of unhelpful results discussing Any which are unlikely to mention existentials at all.
any P only hints at polymorphism, that any value that conforms to P may be used here. Rustâs dyn Trait at least hints at dynamic dispatch which is a clear warning about the performance of using this value. The word âanyâ simply doesnât communicate anything about existentials explicitly or hint at the related costs. What I could see happening is many devs will apply the fix it to change P to any P without ever becoming aware of existentials or the related costs.
For these reasons I would ask for alternative spellings to be considered, specifically something like dynamic P. existential P could also be considered but may not be preferred because A. âexistentialâ is a rather esoteric word, not very user friendly and still doesnât communicate the related costs and B. dynamic dispatch is a well understood concept from many other languages, and anyone optimizing for performance will see dynamic P as an immediate red flag. dynamic P hints directly at the performance concerns rather than any or existential which require additional searching and out of band learning before the performance hit is understood.
Thanks again for your hard work!
EDIT: I just remembered that dynamic is already a keyword in as in @objc dynamic var properties, so existential P would probably be better here.
What about Any P? It's basically the same as the proposal here, except that it's capitalized. This makes it look more like a new type (because types are capitalized). It aligns with Any, which is the unconstrained existential. And it doesn't use angle brackets, so it doesn't look like a generic.
func useNumeric(_ thing: Any Numeric) {
print(thing)
}
The change from any to Any is small, but I like the alignment with unconstrained Any: it means that Any is consistently the keyword for introducing existentials.
In my understanding, existential type is existential type. It doesn't matter what structure is behind this abstraction. On the contrary, if I'm looking at Any<P> I'm reading it as:
Any is some generic wrapper, which I don't think is an equivalent for existential type.P is a placeholder for a type. Nothing suggests that P should be a protocol. And even if this would be a restriction in the grammar, it still would look like a wrapper over an existential.To me this looks like just two types. Like Struct Foo :-).
I'm positive for this proposal.
I've been reading every post about generics in this forum.
One things that I would like to get clarity on is, once this proposal is accepted and merged, will the Swift language guide be updating with good explanations of the type system (generics, existential, etc). There is a lot of information in the forums and across proposals (some of it contradictory because is not a topic very well understood) so I think it's necessary to have a definitive canonical and oficial explanation from the language guide.
And my concern is precisely that, in my experience, very few developers understand what an "existential type" is. I didn't understand it for a long time either. What finally made it click for me (assuming that I do understand it correctly) was when I realized that it was effectively this:
struct ExistentialWrapper {
var value: Any
}
Except that value can only hold values that conform to some particular protocol. Any is simply an unconstrained wrapper that can hold values of any type.
This was the "ah-ha" moment for meâI realized that an existential is just a wrapper type. If we don't help users understand this, I think they're going to continue to be confused by existentials, by the difference between existentials and generics, by the restrictions that exist on existentials, and by why existentials don't (usually) conform to their associated protocols.
Adding any, as proposed here, is a good step. I just think we may be able to do even better.
(And yes, Iâm aware that my struct above is defining existentials terms of an existential, which is circular. My point is that Any is easy to understand, even if you donât understand âexistentialsâ.)