SE-0309: Unlock existential types for all protocols

I'm sorry but I could not consider real situation that this some any P appears. For me it seems that even if existential type any P self-conforms to the P, there is no situation we have to write some any P or any some P, because any P is not protocol and some P is not protocol neither.

Please give me some example🙏

2 Likes

No, please continue. Each post highlights a different issue but you add a lot detail again and again (:+1:) and write at a level the average user can understand. Explaining it in terms of boxes and wrappers && contrasting it with all other possible kinds of types together with all of the details in one place is very helpful. It’s a nice overview written for an average user. Sometimes repetition or a different wording is needed to clearly see the connections or gain a new insight. (Now I have to internalise it.)

Thanks!

With regard to syntax. If there is to be any new syntax, I understand now (I think) why any would be less than ideal. For the last few days I have trying to get a clear model in my mind based on only the some and any keywords. Reading it like: it is some (specific) view and it can be any view. Looking at it from the caller and callee site. In the end these are similar sounding keywords and don’t really convey one is actually a box (as I now understand it to be). A lot of frustration and misunderstanding would remain I think.

Though if there is ever going to be new syntax it should indeed emphasise t’s a box or wrapper. An average user can understand boxes and wrappers far easier than terms like existentials. And then self-conformance becomes clear and the reductio-ad-absurdum proof starts to make sense too.
But in all honesty |type| would not be very readable. Bikeshedding, I know.

3 Likes

I haven't caught up with everything here, but I'd really like if we could just forget "self-conforming" is a thing and never speak of it again :sweat_smile:. IMO, it's a bit of a non-sensical notion.

The way I think of it is to consider FixedWidthInteger. It has a static requirement, bitWidth. UInt8.bitWidth returns 8, UInt16.bitWidth returns 16, etc.

What exactly is an existential container, holding any FixedWidthInteger, supposed to return as its bitWidth? There's no reasonable answer. So it follows that, even if self-conformance was a thing, there would always be protocols which an existential container could never conform to. It's a bit unfortunate that the language currently allows you to pretend that some existentials conform to their constraining protocols, but ultimately it's a red-herring - a distraction from the real issue.

That's beside the fact that, if we ever got the ability to enumerate the types conforming to a protocol, you'd have existential containers showing up alongside actual conforming types. We say that in Swift, protocols carry semantic meaning. IMO, it follows that the set of conforming types also carries semantic meaning (e.g. if you had a protocol JSONPrimitive, there are a restricted set of types which you expect to conform to it). Having existential containers also conform dilutes that idea significantly.

Uhu, but if he hadn’t mentioned it, I doubt I would have understood the nuances and the issues.

oh boy I sure hope I am not going to mess up terms in what follows.
The same name is used to denote two things: a plain old protocol and a wrapper for that protocol (i,e. existential). For the average user it looks the same and so when that dreaded warning pops up (“Self, associated type, bla bla”) MacBooks get tossed out of the window. But by explaining it’s actually a wrapper and in order to use that wrapper as the protocol type itself, the wrapper itself needs to conform to the protocol (i.e. self-conformance) the warning make sense. Finally make sense I dare say.

And then comes the reductio-ad-absurdum proof that self-conformance doesn’t really make sense
and thus shouldn’t really be mentioned. No, it shouldn’t be implemented or used. But without mentioning it that can’t be explained or understood.

Sorry if I butchered anything.

I think there's some misunderstanding here: some P is an opaque type; it is not an existential type. A variable of type any P can be bound to a value of any type that conforms to P. However, if a variable is of type some P, it can be bound only to a value of some specific type that conforms to P.

It is immaterial that some existential types can conform to their corresponding protocols; that does not make some P an existential type. It would be no more correct to say that some Hashable is a string type: the underlying type of a particular some Hashable might be String, but that does not have any impact on the "stringiness" of some Hashable.

There can be no such thing as some any P or any some P for the same reason that there can be no such thing as some Int or any Int.

15 Likes

This below compiles and runs fine, though. The some Error opaque type refers to an existential. :wink:

struct MyError: Error { }
func makeError() -> Error { MyError() }

func f() -> some Error {
    .random()
        ? makeError()
        : MyError()
}

print(type(of: f())) // "Error"

Edit: some Any is a valid type as well, and it can also return an existential:

func f() -> some Any {
    1 as Any
}

print(type(of: f())) // "Any"
2 Likes

Yup, this is a special case for those protocols where the Existential conforms to the protocol itself, where some Foo may be the protocol existential Foo.

It refers to an existential type as the underlying type, but the whole point of an opaque type is that the underlying type is opaque to you.

A variable of type some Error can never be rebound to arbitrary Error values even if you know that its underlying type is the existential [any] Error:

struct E: Error { }

func f() -> /* any */ Error { E() }
func g() -> some Error { f() }

var x = f()
x = g() // Allowed because `x` can be bound to any error (it is of existential type).

var y = g()
y = g() // Allowed because this is *always* the same underlying type.
y = f() // Compiler error. Not allowed because one cannot use knowledge of the actual underlying type.

This is an essential point to understand. The underlying type is immaterial: some Error is not an existential type.

12 Likes

Also Error is a special case with built-in compiler support, no? Might not be the best example to test with and built up a mental model of this topic.

1 Like

I think both any P and _ & P are not sufficient. They are useful as a convenience shortcut for common cases, but not sufficient to cover all the cases.

Consider the example from the proposal:

extension Sequence {
  public func enumerated() -> EnumeratedSequence<Self> {
    return EnumeratedSequence(_base: self)
  }
}

let s: any Sequence = [1, 2, 3]
let e = s.enumerated()

Even after SE-0309 this does not type check, but in theory it is possible to lift restrictions further. Then to express the type of e we would need something like any<T> EnumeratedSequence<T> where T: Sequence. For this specific example EnumeratedSequence<_ & Sequence> would work too, but in general case T may occur inside the type multiple times.

For example:

protocol P {}
extension Int: P {}
extension String: P {}
struct KeyWrapper<T: P>: Hashable {}
struct ValueWrapper<T: P> {}

let a: Dictionary<KeyWrapper<Int>, ValueWrapper<Int>> = ...
let b: Dictionary<KeyWrapper<String>, ValueWrapper<String>> = ...
let x: Array<any<T> Dictionary<KeyWrapper<T>, ValueWrapper<T>> where T: P> = [a, b]

Alternatively, this can be expressed in a top-down fashion using constraints if we can refer to generic parameters as associated types:

...
let x: [any Dictionary where Key == some KeyWrapper, Value == some ValueWrapper, Key.Wrapped == Value.Wrapped, Key.Wrapped: P] = [a, b]

See Syntax for existential type as a box for generic type for more.

This will become even more import once Swift gets unboxing existential. Unboxing gives a generic type of local scope, and to escape this scope, any value needs to either have a non-generic type, or be wrapped into an existential container.

1 Like

"It is often that people reach for existential types when they should be employing generic contraints — "should" not merely for performance reasons, but because they truly do not need or intend for any type erasure."

Using of generic functions lead to increasing of code size. What about cases where we use

func foo(_ value: SomeProtocol)

instead of

func foo<T: SomeProtocol>(_ value: T)

to reduce binary size?

All “self-conformances” are special cases: there can be no implicit self-conformance in the general case. We are talking about them because @1oo7 is concerned about how these special cases work. There is nothing special about Error here (for the purposes of this discussion) other than it’s a self-conformance that you can actually try for yourself on all platforms.

4 Likes

I hope that the compiler did the right thing in most cases when considering generics.
However, I wish some kind of annotation to overwrite the decision of the compiler.

Using generics or not shouldn't fix boxing or code generation for the backend.

The compiler can specialize either form—they are isomorphic—so this is not a reliable way of controlling code size.

7 Likes

In first case instance of SomeProtocol can have different underlying types. In second case Type is known at compile time. What do you mean, saying "The compiler can specialize either form"?

"this is not a reliable way of controlling code size" – totally agree here, I only handle that such idea exists. Does Swift 5.4 compiler generates binaries of the same size for this two functions, or may be it will do in future?

In cases where SomeProtocol existentials are statically resolvable, i.e. you know the underlying type, and the underlying type doesn't change in the function body you can specialize functions accepting runtime existentials, but not always.

Neither of these statements is always true. If you pass a value of known static type into an existential, then the compiler can see that and still specialize the function based on the type going in. Similarly, if you call the generic function with a dynamic type, the compiler does not necessarily know what that type is, and may not be able to specialize. Both functions accept a generic type and a value of that type; the existential form just packs them into a single value instead of two separate arguments.

There is an "existential specializer" pass that turns some functions with existential arguments into their generic forms, which allows generic specialization passes to further optimize them. You could maybe use @inline(never) to suppress specialization, but there still isn't a fully designed and developed suite of attributes for controlling optimizer behavior.

8 Likes

Thanks for sharing these interesting details.

4 Likes

This would not really happen
A protocol should never have the prefix Any, normally it’s given to a type-erasing wrapper type (which normally is a struct and could never have another any in front of it).
So the Array type would instead be written either as [any Cancellable] or as [AnyCancellable] (depending on whether you want an array of existentials or an array of wrapper types), but never [any AnyCancellable] (unless you aren’t following standard naming rules and give your protocol the name AnyCancellable).

1 Like

The discussion of some P vs. any P has gone off the rails. It’s conflating type names with types. Look at this example:

func getSauce() -> some Sauce {
    return SecretSauce()
}

Here, the returned value has type SecretSauce, whether the caller knows it or not. There are two different names, “SecretSauce” and “some Sauce”, but the names really do refer to the same type (well, they do in this context — “some Sauce” can refer to a different type when it’s the return type of a different function).

It makes no sense to ask whether SecretSauce is an opaque type, whatever that might mean. The opaqueness here belongs to the type name “some Sauce”, and indicates that the caller typically doesn’t know what it names.

OTOH it makes no sense to ask whether type name “SecretSauce” is existential. A type SecretSauce is existential or not, regardless of how it’s named or referred to.

“Opaque” is a qualifier of type names, not of types. “Existential” is a qualifier of types, not type names. Asking “Is a given type name existential?” is of course nonsensical.

7 Likes