Very confused about some compiler warnings related to protocols—bugs or intended?

Using Swift
1

Problem description:

I keep getting this compiler error when trying to use variables whose type is declared as conforming to a protocol:

Protocol type FooProtocol cannot conform to FooProtocol because only concrete types can conform to protocols.

Could someone please let me know if this is a bug, or expected behavior? If it's expected, why? I mean, what is the point of protocols if you cannot use a given instance purely on the basis of its conformance to a protocol?

Example:

Given:

protocol FooProtocol { var bar: Int { get set } }

class FooClass: FooProtocol { var bar = 42 }

struct Bar<F: FooProtocol> {
    typealias MyFoo = F
    var foo: MyFoo
    init(foo: MyFoo) { self.foo = foo }
}
struct Whee {
    var someSpecificFoo: FooProtocol
    func process() { let bar = Bar(foo: someSpecificFoo) } // error line
}
var whee = Whee(someSpecificFoo: FooClass())
print(whee.someSpecificFoo.bar)

Strangely, the compiler fails with:

Protocol type FooProtocol cannot conform to FooProtocol because only concrete types can conform to protocols.

This seems like a buggy error, because this statement is basically incorrect. The compiler ought to be able to recognize that someSpecificFoo is a concrete instance conforming to FooProtocol because that's what var someSpecificFoo: FooProtocol means.

If we change the declaration to, var someSpecificFoo: some FooProtocol, all this does is to change the error to:

Property declares an opaque return type, but has no initializer expression from which to infer an underlying type

Why does the compiler now fail to auto-synthesize an initializer expression like normal?

Even if we add an explicit initializer, it will still fail. Adding:

    init<F: FooProtocol>(foo: F) { self.foo = foo }

... just changes the error message to:

Cannot assign value of type F to type some FooProtocol

Why can't the first error tell us how to fix the problem, instead of suggesting a corrective action that (if you try it) will just change the error to something else?

More importantly, why can't F be considered some FooProtocol? F conforms to FooProtocol by definition.

And in my codebase there is only one type that conforms to FooProtocol, which means that even if the compiler just guessed, it would have a 100% chance of being correct about what the actual underlying type must be. So why do we get these errors?

Workarounds:

Just on a hunch I tried adding @objc to the protocol declaration and import Foundation to the top line, and boom, now the code works perfectly.

But why?

And are there downsides to adding @objc here? If so what are they? If not, why isn't it just like that by default?

1 Like
2
3

This is not a full answer with respect to some of the why's, but the gist of the failure here is that

var someSpecificFoo: FooProtocol

declares a variable which contains some unknown type which you know to conform to FooProtocol, while

init<F: FooProtocol>(foo: F)

declares an initializer which takes a specific known type F which you know to conform to FooProtocol. Effectively, you cannot pass a value of "unknown" (dynamic) type where you must pass in a value of "known" (static) type.

There is also a sharp distinction between someSpecificFoo: FooProtocol which is that it can be assigned any value conforming to FooProtocol, while some FooProtocol means "this value conforms to FooProtocol and has a known static type, but that type is hidden behind an opaque protocol constraint".

As for the wording of the diagnostic specifically: you're right that it's not very descriptive, but this is a very difficult-to-describe area of the language that's been very difficult to boil down into a one-statement error message that's easy to digest.

This diagnostic comes up a lot and there might be merit to rephrasing it — although it is technically correct in a very compiler-y way, it's not terribly helpful. It might be good to better recognize this case and simplify the message to: "cannot pass value of dynamic protocol constraint 'FooProtocol' to function expecting generic type 'F: FooProtocol'", though this may be non-trivial.

1 Like
4

However, a protocol type does not conform to protocols

I am not talking about a protocol type, I'm talking about an instance of a protocol type—which does, by definition, conform to that protocol.

If an instance of a protocol type did not conform to the protocol then it would just be an instance of Any and protocols would be useless.

For protocols with static method, initializer, or associated type requirements, the implementation of these requirements cannot be accessed from the protocol type

No, but all of those can be accessed from an instance of a protocol type—to wit, the following compiles and runs just fine:

protocol FooProtocol { 
    var bar: Int { get set } 
    init(bar: Int)
    static func perform()
}

extension FooProtocol {
    func instancePerform() {
        Self.perform()
    }
}

struct Foo: FooProtocol {
    var bar = 42
    init(bar: Int) {
        self.bar = bar
    }
    static func perform() {
        print("performing...")
    }
}

struct Bar {
    let foo: FooProtocol = Foo(bar: 96)
}

Bar().foo.instancePerform()
type(of: Bar().foo).perform()
type(of: Bar().foo).init(bar: 104)
5

But we are never passing an unknown type—when it gets passed in there, it will be known, because the runtime knows the type (otherwise how would it pass it in).

Right?

Or is there a situation where the runtime won't know the type?

6

The key here is that by design generics require static type information; dynamic type information (which is indeed provided at runtime) is insufficient.

See also "Existential Types and Generics" for some specifics on this (and in general, all of Generics.rst may be helpful, including "Implementation Model" and "Specialization").

2 Likes
7

I can see that there is a technical difference under the hood, regarding a difference between how existentials and generics are handled, with relation to vtable lookup.

However lets look at what this means for Swift developers:

  • cryptic, counter-intuitive, and often self-contradictory compiler warnings
  • just resort to @objc and the problem goes away (but at what cost?)
  • the only alternative seems to be kludgy type-erasure schemes
  • so people just avoid adding Equatable or Hashable like the plague since it breaks everything

Seems to me, if we're going to be forced to resort to type-erasure anyway, couldn't the compiler just auto-synthesize the type erasure code behind the scenes? Would there be a drawback to that?

1 Like
9

I'm not sure what you mean here; only types conform to protocols, not instances.

The function type(of:) gives you the dynamic type of a value. This is distinct from the static type.

The distinction between the dynamic type and the static type is not an "under the hood" matter. It is part of the user-facing semantics of Swift.

In the general case, it is impossible to conform every existential type to its own protocol automagically. This is not a limitation of Swift.

Consider the Comparable protocol:

You can conform type A to Comparable, providing an implementation such that (a1 < a2) == true. I can conform type B to Comparable, providing an implementation such that (b1 < b2) == true. However, if the existential type Comparable conforms to itself, then that implies we could write:

let a1c = a1 as Comparable
let b1c = b1 as Comparable
print(a1c < b1c)
// If Comparable conforms to itself, then we can compare the two values.
// But what is the result?

The compiler cannot pull an implementation of such an operation out of thin air. Yes, we can arbitrarily create a rule which sorts every value of type A before every value of type B by lexical ordering of type names, but someone would need to actually make that arbitrary decision and write it out in the form of a concrete implementation of static func < for the existential type.

(Then, someone else would complain that it would be a very silly implementation, because it would imply (100 as Int) < (0 as UInt); thereby demonstrating that any reasonable implementation requires specific knowledge of the semantics of the requirement being implemented as well as of any relationships among conforming types. You can peruse the multiple revisions of AnyHashable to see how difficult it is to implement a hash value that fulfills all the required semantics.)

The same applies to static methods, initializers, and Self or associated type requirements in every protocol.

Sure, we can (and eventually should, IMO) relax the rules around which protocols can be used as existential types, change the spelling of existential types from P to any P, and even allow users to implement self-conformance by writing extension any P: P { ... }, but the implementation itself (the part that's elided here with ...) will have to be manually written. There is no "auto-synthesis" here in the general case.

10 Likes
10

To expand on what @xwu said below: this is very much not an under-the-hood difference, but a semantic difference, and even more so, a feature difference. Existential types and generic types in Swift are two separate but very important language features that give you different tools for your toolset. Even though they overlap in some significant aspects, there are important use-cases that the other cannot resolve:

  • Existential types allow you to represent a dynamic range of types which are unknown and cannot be known at compile time
  • Generic types give you access to associated types, specialization, (occasionally) static dispatch, different optimizations, and more

So, a bit of a frame shift: given two separate feature sets that happen to have some overlap, instead of trying to work around where they are disjoint, is it possible for your use-case to lean into one or the other?

My instinct for your (likely simplified) code example would not be to work around the issue but to lean into generics and make Whee generic on the specific type F: FooProtocol that you want to store (and eventually create a Bar from). If this is not possible, consider why not; for instance, if Whee.someSpecificFoo needs to by dynamically assignable to any FooProtocol instance, then you're using the core feature that existential types exist for; in that case, is it possible for Bar.init(foo:) to take an existential instead of a generic type? (If not, then you have more thinking to do, because clearly in some cases you need explicitly dynamic type instances, and in others you need explicitly static type instances. There are ways to solve this, but it will be specific to your use-case.)

Can you share a little bit more about your concrete use-case? There should definitely be ways to resolve this without workarounds.

2 Likes
11

I definitely take @1oo7's point that the error message and underlying failure here are quite confusing—a quick search of StackOverflow turns up countless results for the same issue. It's also true that in the 'easy' cases, such as FooProtocol, it's conceptually sound for a protocol to conform to itself. After all, the static type FooProtocol obviously trivially satisfies the requirements of FooProtocol.

But as soon as the protocols start getting more complex (such as in the ways @xwu mentioned), there's a lot of tricky questions to be answered, and the language currently doesn't have the expressive power to cover those cases. Adding a one-off rule of "'simple' protocols may self-conform" without the additional features needed to generalize the rule may leave the language in a worse resting place than we have currently (or, maybe not! I'm unsure whether there's been extensive discussion around allowing self-conformance for a very limited subset of protocols).

To your point about being able to access static or init requirements from an instance, @1oo7, it's worth pointing out that Swift's generics model doesn't require that a generic parameter be attached to an instance at all. Consider:

protocol FooProtocol { 
  var bar: Int { get set } 
  init(bar: Int)
  static func perform()
}

struct FooCreator<T: FooProtocol> {
  func giveMeAFoo() -> T { T.init(bar: 0) }
}

Right now, there's simply no way for a FooCreator<FooProtocol> to function. Yes, if we had some foo: FooProtocol we could do type(of: foo).init(bar: 0), but that's not going to be possible generally since generic functions and types can be parameterized by just the type.

1 Like
12

What you are looking for is "opening existential type". This was discussed in Improving the UI of generics, but not yet implemented.

With the hypothetical syntax from the the discussion it would look like this:

func process() {
   let <X: FooProtocol> openedFoo = someSpecificFoo 
   let bar = Bar<X>(foo: openedFoo)
}

Current workaround for that is to move code depending on the opened type to the protocol extension, like you did with instancePerform().

1 Like
13

On master, the diagnostic is slightly improved:

error: protocol 'FooProtocol' as a type cannot conform to the protocol itself; only concrete types such as structs, enums and classes can conform to protocols
    func process() { let bar = Bar(foo: someSpecificFoo) } // error line
                               ^
note: required by generic struct 'Bar' where 'F' = 'FooProtocol'
struct Bar<F: FooProtocol> {
       ^
3 Likes
14

After reading this thread I have some trouble with my understanding of the definition of existential type.

Quoting from this link that was posted in the first comment:

https://github.com/apple/swift/blob/master/userdocs/diagnostics/protocol-type-non-conformance.md

However, a protocol type does not conform to protocols - not even the protocol itself. Allowing existential types to conform to protocols is unsound. For protocols with static method, initializer, or associated type requirements, the implementation of these requirements cannot be accessed from the protocol type - accessing these kinds of requirements must be done using a concrete type.

Please correct where I'm wrong:

  1. In my understanding an existential type in Swift is a type that can – at runtime – contain different concrete types that all conform to the same protocol.
  2. The existential type implementation is implicitly generated by the compiler and uses some form of dynamic dispatch.
  3. For protocols that have associated type requirements it's not possible to create an existential type.

This would imply that existential types always conform to their protocol, otherwise they couldn't be created in the first place.

Example from this thread:

let a1c = a1 as Comparable
let b1c = b1 as Comparable
print(a1c < b1c)
// If Comparable conforms to itself, then we can compare the two values.
// But what is the result?

I think this already fails before the comparison, because Comparable cannot be used to create an existential:

let a1c = a1 as Comparable
// Protocol 'Comparable' can only be used as a generic constraint
// because it has Self or associated type requirements

Coming back to @1oo7's initial question, if existential types always conform to their protocol this would imply that allowing them to be used to specialize generic parameters shouldn't create any soundness issues.

1 Like
15

Any type-level requirements (initializers, static methods) also become problematic as @xwu mentioned and as I illustrated above. You won’t always have access to a dynamic type via an instance, so these requirements need to be invokable via the static type, which isn’t compatible with the way protocols/existentials work (currently).

1 Like
16

Ah, thanks, missed that on the first read.

Adding a one-off rule of "'simple' protocols may self-conform" without the additional features needed to generalize the rule may leave the language in a worse resting place than we have currently (or, maybe not! I'm unsure whether there's been extensive discussion around allowing self-conformance for a very limited subset of protocols).

I think it would be a good idea to allow "simple" protocols to self-conform, it's not an uncommon case. Maybe with a slightly adapted error message:

error: protocol 'FooProtocol' as a type cannot conform to the protocol itself
 because it has static method or Self requirements;
 only concrete types such as structs, enums and classes can conform to 'FooProtocol'
1 Like
17

For the past month or so, several people here have requested to have existential types for protocols with associated types when all of said associated types are fully specified. This still won't help with protocols with Self requirements, though.

1 Like
18

Sorry to come back to this thread after so many months, but I need further clarification regarding a particular aspect of the original question.

Generally this is true however I found there is actually an exception to this, which was mentioned in the "Exceptions" section of the documentation @xwu linked above, but which I had missed.

Namely, the exception is that adding @objc to a protocol will allow its protocol-type to conform to the protocol, as long as there are no static or initializer requirements of that protocol.

So, the point I would like clarification on, is whether there is any reason to avoid this?

For example, would there be any deleterious consequences to updating the original example by adding @objc to the protocol, like so:

import Foundation

@objc // witnesseth the power and magic of @objc
protocol FooProtocol { var bar: Int { get set } }

class FooClass: FooProtocol { var bar = 42 }

struct Bar<F: FooProtocol> {
    typealias MyFoo = F
    var foo: MyFoo
    init(foo: MyFoo) { 
        self.foo = foo 
    }
}
struct Whee {
    var someSpecificFoo: FooProtocol
    func process() -> String { "\(type(of: Bar(foo: someSpecificFoo)))" } 
    // no more compiler errors! thanks @objc
}

var whee = Whee(someSpecificFoo: FooClass())
print(whee.process()) // prints "Bar<FooProtocol>" 
print(whee.someSpecificFoo.bar) // prints 42

Note: adding a static func win() or init() to @objc protocol FooProtocol above will cause the compiler to throw the following error at the line of func process():

FooProtocol cannot be used as a type conforming to protocol 'FooProtocol' because 'FooProtocol' has static requirements.

As mentioned in the Swift repo documentation about existentials under the "Exceptions" section at the bottom, @objc protocols are not subject to the following restrictions:

(thanks to @xwu for providing that!)

Discussion

For easy nomenclature:

  1. SIRs— @objc protocols with Static and/or Initializer Requirements
  2. no-SIRs— @objc protocols without SIRs ("simple" according to @Jumhyn's opinion—see also below discussion)
  3. PATS— protocols with Associated Type and/or Self Requirements (aka "PATS")
  4. POPs— all other protocols ( Plain Old Protocols )

Regarding @Jumhyn's opinion:

Given that no-SIRs (see above) already may self-conform, what is the compelling reason why there could not be an equivalent category or subset of POPs? Just curious what is so different about a POP that it cannot also have the capability of its existential type conforming to it.

I'd also like to better understand what additional differences there might be between the four categories mentioned. Is there a compelling reason not to add @objc to a protocol in order to get the capability of having its existentials conform to it?

Table of Differences Between SIRs, no-SIRs, PATs, and POPs

|   |                                  | noSIRs | SIRs    | PATs | POPs |
|---|----------------------------------|--------|---------|------|------|
| 1 | compatible with Obj. C code      | Yes    | Yes     | No   | No   |
| 2 | allowed in a generic argument    | Yes    | No      | No   | No   |
| 3 | `foo = [BarProtocol]` is allowed | Yes    | Yes     | No   | Yes  |
| 4 | Dispatch type                    | Dyn.   | Dyn.    | Stc. | Dyn. |
| 5 | can require `func x<T>()`        | No     | No      | Yes  | Yes  |
| 6 | req.s must be Obj.C compatible   | Yes    | Yes     | No   | No   |
| 7 | optimizability                   | Less   | Less    | More | Less |
| 8 | ...? any other differences?      |        |         |      |      |

Notes:

  • Row 2 means, given let bar: Barrable = Bar() and a function func foo<T: Barrable>(_ bar: T), the following is valid code: foo(bar)
  • Row 5 is obviously a consequence of row 6, insofar as Obj. C does not support generics of any kind.

Please reply and let me know any other differences between these four protocol types, and I will update the table accordingly. Thanks!

1 Like
19

This was noted in the education note which I linked to months ago for your reference. It works in this way for Objective-C compatibility reasons.

As to why non-Objective-C Swift protocols don’t behave similarly, this has already been answered above, and in particular by @Jumhyn, and it is also touched on in the educational note.

1 Like
20

Ah, thanks. I must have not scrolled down to that "Exceptions" section. But sure enough there it is:

@objc protocols with no static requirements can also be used as types that conform to themselves.

I've updated my above post to reflect this (and made some other improvements as well). Thanks for all that you do on the Swift.org documentation, it is really top-notch... I'm just trying to understand the underlying behaviors and exceptions better.

Also, thanks for the tip regarding the question of why POPs (or a subset of them) couldn't be updated to allow such a protocol to have an object that conforms to itself. I do see where your team added this clear explanation in the documentation:

Currently, even if a protocol P requires no initializers or static members, the existential type P does not conform to P (with exceptions below). This restriction allows library authors to add such requirements (initializers or static members) to an existing protocol without breaking their users' source code.

This does give me an idea, but it's more of a pitch so I'll save that for another thread.

That being the case, I'm still curious about what other differences there might be between @objc protocols and non-@objc protocols that would cause us not to want to use them, besides what I listed in the table in the previous post.

For example is there a runtime cost associated with @objc or does it mean that all objects that conform to the protocol automatically get NSObject conformance or similar? Thanks.

21

Well, if you care about cross-platform, @obj protocols are not supported for anything other than Darwin. The obj runtime get involved would be another difference