Introduction of new keyword 'any' for protocol conforming arguments

Evolution Discussion
1

SE-0335 has been accepted and on it’s current trajectory will eventually impose a source breaking change to the Swift language where any methods that have argument types which are a protocol will need to be rewritten to prefix the protocol name with a new keyword any. The review breezed through and this is on all our horizons and already available in Xcode 13.3.

The specifics of the change are rather long winded and revolves around the overhead associated with the use of “existential containers” which some feel should be surfaced in the language and made explicit.

To cut a long story short, there are two nearly equivalent ways to pass something conforming to a protocol to a function as discussed in these great, articles

// First currently "existential" form
func startTraveling(with transportation: Drivable) { }

// Second "generic" form
func startTraveling<D: Drivable>(with transportation: D) { }

The second form has a couple of advantages. To begin with the second form runs faster (though I feel these concerns may have been overblown) but more importantly it is more flexible and you don’t encounter the dreaded protocol can only be used as a generic constraint because it has Self or associated type requirements error which is trying to tell you to change the function signature to the second form if your protocol has these requirements.

The approach taken by SE-0335 seems to be to introduce friction to the first form so people are aware they are producing slower and more limited code in the hope that they move to the second far less intuitive form. This seems a very inefficient use of a lot of peoples time.

I’d like to propose a different approach:

As the second form is effectively an equivalent and better form of the first, I feel the compiler should automatically, internally rewrite functions in the first form into the second form from the point of view of the rest of the program and libraries importing the module containing that code. The source file would retain the succinct, first from. If you look at it this way the necessity to even discuss existential containers disappears. In a sense this is the way the compiler should have implemented passing protocols in the first place given the relationship between constrained generics and protocols in the implementation of Swift. View the first half of this video if you want to understand what I mean. Generic functions are not always specialised and faster as a result but typically use dispatch through “protocol witness tables” as does the first form of function discussed above. This is another thing developers just don’t need to know about.

If this approach were taken Swift 6 would not include a source breaking change but an internal code rewriting optimisation that makes less optimal code run faster. Is there a reason I'm missing why this has not been considered or is this the long term plan and why couldn't it be introduced now? You could still have the option of using the any keyword if you needed to go existential.

1 Like
2

These forms are not equivalent, though — if they were, we wouldn't need two different ways of spelling the same thing. (i.e., there's a reason we have both existentials and generics in the language)

Calling a generic method requires compile-time knowledge of the generic parameter type, even if the function isn't specialized for the type. But, you don't always have that:

struct Car: Drivable {}
struct Bus: Drivable {}
struct RaceCar: Drivable {}

func startTravelingE(with transportation: Drivable) {}
func startTravelingG<D: Drivable>(with transportation: D) {}

let vehicles: [Drivable] = [Car(), Bus(), RaceCar()]
for vehicle in vehicles {
    startTravelingE(with: vehicle) // ✔
    startTravelingG(with: vehicle) // error: protocol 'Drivable' as a type cannot conform to the protocol itself
}

If the existential-taking startTravelingE were rewritten into the generic startTravelingG, you wouldn't be able to call with a variable of type Drivable — which is the driving need for the existence of existentials.

Functions can't safely be rewritten in this way without being aware of all callsites: unless you know for a fact that a method taking an existential is called with non-existential values 100% of the time, you wouldn't be able to rewrite this. Perhaps it would be possible for internal/private methods where you might be able to make such a guarantee, but it feels to me that you might be better off using a linter which could suggest converting existential-taking methods into generic ones.

5 Likes
3

I'm currently reviewing GRDB code for the next major version, Swift 5.6+, and it just happens that I'm dealing with SE-0335. I look for all protocol uses, add some any, or refactor into generic functions when relevant.

In a few places, I want to expose apis that accept existentials, and I do not convert them to generic functions. And I don't want the compiler to perform such refactoring automatically.

This happens when the user is expected to call those apis with existentials. This happens, for example, when the user is expected to define collections of heterogenous values (collections of existentials).

With Swift 5.6, we have currently:

protocol P { }

func frobnicate1(_ value: any P) { }
func frobnicate2<T: P>(_ value: T) { }

var values: [any P] = []
for value in values {
    // OK
    frobnicate1(value)
    
    // Compiler error
    // Protocol 'P' as a type cannot conform to the protocol itself
    frobnicate2(value)
}

Maybe [Pitch] Implicitly opening existentials will change this. Maybe the compiler will learn to automatically open existentials when a generic function is called.

But I'm not 100% sure, and maybe @Douglas_Gregor will shed some light.

Now, unless my intuition is wrong, I think that the Core Team currently wants the language to migrate to:

// First form
func startTraveling(with transportation: any Drivable) { }

// Second form
func startTraveling(with transportation: some Drivable) { }

In a way, the runtime behavior of our methods would be made explicit.

Clearly, this does not come without friction! Some people just don't care about this level of explicitness.

So maybe your idea, along with implicit existential opening, will allow us to keep naked protocol names eventually :-)

--

EDIT: @itaiferber you were first :-)

2 Likes
4

Interesting, I've never seen that error before. Is that a compiler constraint that could be lifted I wonder? In terms of bits and bytes the generic invocation has all it needs from the existential container you're using (see the video I mentioned above): the pointer to the object, the meta-type and witness table of the conformance of the type to the protocol. If not, using any Drivable could inhibit the rewrite I suggest.

5

ISWYDT ;)

2 Likes
6

This diagnostic has been rewritten a few times, so you may have previously seen alternate forms:

  • Swift 5.2: error: value of protocol type 'Drivable' cannot conform to 'Drivable'; only struct/enum/class types can conform to protocols
  • Swift 5.1: error: protocol type 'Drivable' cannot conform to 'Drivable' because only concrete types can conform to protocols
  • Swift 4.2.4: error: cannot invoke 'startTravelingG' with an argument list of type '(with: Drivable)'

In any case, this method call has never been possible.

Perhaps! As @gwendal.roue points out above, there is some exploration in [Pitch] Implicitly opening existentials for formalizing the existential → generic conversion, but I think it's something that requires a lot of thought in terms of the language design, from the top down. (e.g., if it's possible to open an existential, should generic methods be implicitly callable with existential parameters, assuming valid constraints? And if so, what becomes the effective difference between existential types and concrete types, if any at all? Maybe nothing!)

My post above had more in mind the state of things today, though things could definitely change in the future. It's a question worth asking.

One thing I wanted to point out about generic methods regarding their capture of the static, compile-time type to be exposed at runtime:

class A {}
class B: A {}

func f<T>(_ t: T) {
  print(T.self, "<=>", type(of: t))
}

f(A()) // A <=> A
f(B()) // B <=> B
f(B() as A) // A <=> B     <<---------

Part of the reason that you can't call a generic method with an existential is that the static type isn't known, but if the waters between existential and concrete types above do get muddied a bit, and you could call a generic method with an existential parameter, you can imagine the generic being instantiated with the runtime type inside of the existential container.

If that does become possible, API consumers may need to be a lot more careful and aware of the types they're passing around: if a method goes from returning some P to any P and the result is passed to another generic method, the results may change considerably!

4 Likes
7

I swear this wasn't intentional, but I will take credit. :slight_smile:

1 Like
8

i don’t think this would be good for the teachability of the language. “any” and “some” are synonyms, but they’re being used to distinguish between completely different declarations here.

9

There are places where "any" and "some" are interchangeable, but no, they're not synonyms.

5 Likes
10

i meant in the English language sense. if i say “bring me some pizza” as opposed to “bring me any pizza”, both are expressing the idea that i don’t care about the specific kind of pizza i am receiving.

now, it’s not like we don’t already have examples of this in the ecosystem. for example, we’ve mostly settled on type meaning a compiler type, and kind meaning ‘type’ when not referring to a compiler type. but that is something that exists to workaround an existing name collision, whereas we already have a syntax for generics.

1 Like
11

We've had thorough discussion of the design space in SE-0244, SE-0335, and SE-0341 over the past three years. It's really not necessary to relitigate this well trodden ground particularly now that the review processes are complete.

As to the original question, it has always been tempting to imagine if the bare protocol spelling could be repurposed for some P, but because some and any are not synonyms, this is not possible without first having some explicit spelling for an existential type, which is SE-0335. The compiler already optimizes away existential boxes in circumstances where it can determine their use to be unnecessary; this is orthogonal to the fact that some and any are necessarily distinct concepts that cannot be conflated in one spelling.

3 Likes
12

relitigate? it’s only been a few days since any shipped.

“well-trodden” for swift evolution veterans == “barely surfaced” for 99.9% of swift users. personally, i don’t consider that just because something is perceived as settled among a tiny group of language designers means that the full implications of a decision have already been accounted for.

2 Likes
13

Precisely—the feature’s not only been discussed, pitched, reviewed, implemented, and previewed in beta, it’s been shipped. Your point has been made on these forums in the past—including by me—and considered in the context of vibrant discussion. You can search the forums for extensive relevant commentary; these comments don’t expire and you don’t need me or anyone else to replay the discussion. Certainly if there’s something you notice has been missed in the discussion or not appreciated until widespread use, do bring it up.

3 Likes
14

as i understand it, shipping a feature is the first step in acquiring meaningful feedback. so far, very few people have had a chance to evaluate this feature, and i’m willing to bet nobody has tried to teach it to someone who isn’t already a language expert or co-author.

15

As the second form is effectively an equivalent and better form of the first, I feel the compiler should automatically, internally rewrite functions in the first form into the second form from the point of view of the rest of the program and libraries importing the module containing that code.

It seems like the ExistentialSpecializer SIL pass is meant to do some of this, though not exactly sure to what extent — probably only when the compiler can see the function body, given that it is done as specialization but not rewriting.

16

I'm not sure where you got that impression, but since ABI and source stability, shipping a feature is usually the last step in the evolution process. Barring "active harm" it's essentially impossible for language features like any or some to be changed. Even easier changes, like deprecating a function, have a high bar. Unfortunately Swift doesn't really have a full public preview period. Xcode betas could function like that but they're so often broken and unusable, and you can't ship apps with them, that few people actually try them out.

6 Likes
17

As Coolidge once said, “You lose.” There’s been a veritable cornucopia of explainers for SE-0335 in written and video form directed towards users, such as:

1 Like
18

and 1990 was thirty years ago; congrats on the quote tweet. but, i gotta ask, how would you know if you’re wrong about SE-0335?

at what point would you say that “yeah, this is probably making it harder to teach swift and the harm to the growth of the community outweighs my desire to perfect the language?” do we care if someone decides swift is just too mathy for them despite a tutorial author making their best effort to explain it to them through a blog post?

1 Like
19

The beta period for new Xcode major versions also just isn't long enough relative to the number of changes shipped. A pretty common pattern is that something is broken in beta 1, fixed in beta 3, and then by the time I have enough experience trying to use it to give useful feedback we're in late August and it's too late for changes to make it into .0.

4 Likes
20

Yes, implicit in a useful beta period is a release cadence that gets fixes out in a useful amount of time.